Azure Data Engineer

RESPONSIBILITIES:

• Design, Develop, and Maintain scalable ETL pipelines using Azure Data Factory, Databricks, and related services.
• Implement data ingestion, transformation, and storage solutions on Azure Cloud.
• Optimize data workflows for performance and cost efficiency.
• Write and maintain Python scripts for data processing, including advanced Regex-based parsing and validation.
• Collaborate with data scientists and analysts to ensure data availability and quality.
• Monitor and troubleshoot data pipelines, ensuring reliability and accuracy.
• Implement best practices for data security, governance, and compliance.

Required Skills & Qualifications:

• Bachelor's degree in computer science, Information Technology, or related field.
• Minimum 5+ years in Data Engineering roles.

Technical Skills:

• Strong experience with Azure Data Services (Data Factory, Data Lake, Synapse).
• Hands-on experience with Databricks for Big Data Processing and Analytics.
• Proficiency in Python, including advanced Regex for text processing.
• Knowledge of SQL and relational database concepts.
• Familiarity with CI/CD pipelines and version control (Git).