Job Summary

Designed and implemented a scalable enterprise Lakehouse platform on Azure using Azure Data Factory, Delta Lake, ADLS Gen2, and Databricks. Migrated legacy ETL processes to cloud-native ELT pipelines supporting both batch and near real-time workloads. Established reusable data frameworks, metadata-driven orchestration, data quality validation, and automated CI/CD pipelines.

Responsibilities

Designed and implemented scalable data pipelines using Azure Data Factory, Databricks, Delta Lake, Python, and SQL.
Architected Lakehouse solutions supporting enterprise analytics, AI/ML, and Generative AI workloads.
Developed structured and unstructured data pipelines for RAG, vector search, and AI-powered applications.
Built reusable frameworks, improving code quality, scalability, and development productivity.
Optimized ETL/ELT processes, reducing data latency and improving performance.
Implemented CI/CD, automated testing, monitoring, and DataOps best practices.
Established data quality, governance, security, and metadata management standards.
Collaborated with cross-functional teams to deliver business-driven data and AI solutions.
Mentored engineers and led technical design and architecture discussions.
Drove cloud modernization initiatives, improving reliability, scalability, and operational efficiency.

Skills

Must have

5+ years of experience
Azure Data Factory (ADF)
Databricks
Delta Lake
Python
SQL / pl-sql
PySpark
Azure Data Lake (ADLS)
CI/CD (Azure DevOps)
Data Modeling
ETL/ELT Pipelines

Nice to have

Azure AI Search
Microsoft Fabric
Kafka
Snowflake
Terraform
Docker / Kubernetes
Microsoft Purview

Location - pune,mumbai,chennai,banagalore

Azure AI Data Engineer

Job Description

No Referrers Available