Job Description
Job Description:
We are seeking a skilled Data Engineer to design, build, and maintain scalable data solutions on the Azure platform. The ideal
candidate will play a key role in developing robust data pipelines, ensuring data quality, and contributing to enterprise-wide
analytics initiatives.
Key Responsibilities:
Design and develop scalable data pipelines using Spark SQL and PySpark on Azure Databricks
Build and orchestrate ETL workflows using Azure Data Factory (ADF)
Develop and maintain a modern Lakehouse architecture leveraging ADLS and Databricks
Perform data preparation tasks including data cleaning, normalization, deduplication, and data type transformations
Collaborate with DevOps teams to deploy and manage solutions in production environments
Monitor data pipelines, identify issues, and implement corrective actions, including root cause analysis and resolution
Actively contribute as part of the global Analytics team, delivering data-driven insights and solutions
Collaborate with Data Science and Business Intelligence teams to share best practices and drive innovation
Lead data engineering projects and support cross-functional initiatives led by other team members
Apply change management practices, including documentation, communication, and training, to support system
upgrades and data migrations
Must-Have Skills
Strong experience with Azure ecosystem (Azure Databricks, Azure Data Factory, ADLS)
Proficiency in PySpark and Spark SQL
Solid understanding of ETL processes and data pipeline development
Advanced SQL skills