Senior Azure Data Engineer

Job Description

We are looking for a highly skilled Azure Data Engineer with strong hands-on experience in Azure Databricks, PySpark, Azure Data Factory (ADF), ADLS Gen2, Delta Live Tables (DLT), and Unity Catalog to build and maintain enterprise-scale data platforms and modern Lakehouse architectures.

The ideal candidate should have experience designing scalable ETL/ELT pipelines, implementing Medallion Architecture, optimizing Spark workloads, and driving data governance through Unity Catalog.

Key Responsibilities

Design, develop, and maintain scalable ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
Build and implement Medallion Architecture (Bronze, Silver, Gold layers).
Develop data transformation logic using PySpark and Spark SQL.
Build and manage Delta Live Tables (DLT) pipelines.
Implement data governance, security, and access controls using Unity Catalog.
Design and manage data ingestion frameworks using ADLS Gen2 and Auto Loader.
Optimize Databricks jobs and Spark workloads for performance and cost efficiency.
Develop metadata-driven and reusable data pipelines.
Implement monitoring, logging, and data quality checks.
Work closely with business and technology teams to gather requirements and deliver data solutions.
Support CI/CD deployment using Azure DevOps.
Participate in production support, troubleshooting, and performance tuning activities.

Mandatory Skills

Azure Databricks
Azure Data Factory (ADF)
PySpark
SQL
ADLS Gen2
Delta Live Tables (DLT)
Unity Catalog
Delta Lake
Medallion Architecture
ETL / ELT Development

Good to Have

Auto Loader
Azure Synapse Analytics
Microsoft Fabric
Databricks Workflows
Structured Streaming
Azure Event Hub
Kafka
Azure DevOps CI/CD
Data Quality Frameworks
Azure Key Vault

Experience: 6.5 to 15 Years

Location: Bangalore

Azure Data Engineer

Job Description

No Referrers Available