Experience
4 - 10 yrs
Job Location
Bengaluru, India
Vacancy
1
Designation
Azure Data Engineer
Job Type
ONSITE
Job Description
Azure Data Engineer
Please find below the JD for Databricks Lead:
- Databricks Lead, will be responsible for designing, developing, and managing scalable data pipelines and lakehouse architectures using Databricks.
- Will lead data engineering efforts, ensure data quality and goverce, and collaborate with crossfunctional teams to support analytics and business intelligence initiatives.
- Design and implement batch and streaming data pipelines using Databricks (PySpark/Scala).
- Ingest data from APIs, FTP/SFTP, Event Hub, Azure Data Factory, etc.
- Build reusable frameworks for data ingestion, transformation, and enrichment.
- Implement Medallion architecture (Bronze, Silver, Gold layers).
- Create Operational Data Models and Data Marts.
- Ensure data lineage, traceability, and quality across layers.
- Develop realtime ingestion using Structured Streaming and Delta Live Tables.
- Enable near realtime data availability for downstream systems.
- Monitor performance and optimize for cost, speed, and reliability.
- Implement alerting, logging, and observability tools.
- Ensure data goverce using Unity Catalog, Azure Purview.
- Maintain security, privacy, and compliance standards.
Also, as secondary skill, good to have basic knowledge in Power BI
Location Bangalore
Yrs of Exp6Yrs