Job Description
We are seeking an experienced Azure Databricks Engineer with 6+ years of experience in Data Engineering and strong expertise in Azure cloud technologies. The ideal candidate should have hands-on experience building scalable data pipelines using Azure Databricks, PySpark, and Azure Data Factory, along with a solid understanding of US Healthcare data models, claims, provider, member, and clinical datasets.
The candidate will be responsible for designing, developing, and optimizing enterprise-scale data solutions while ensuring compliance with healthcare data standards and governance requirements.
Key Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks and Azure Data Factory.
- Build and optimize data ingestion, transformation, and processing frameworks using PySpark and Spark SQL.
- Develop and maintain data solutions using Azure Data Lake Storage (ADLS Gen2), Azure Synapse Analytics, and Azure SQL.
- Implement Lakehouse architecture using Delta Lake and Databricks best practices.
- Work with large healthcare datasets including Claims, Membership, Provider, Clinical, EHR/EMR, HL7, and FHIR data.
- Ensure data quality, governance, security, and compliance with healthcare regulations.
- Perform performance tuning and optimization of Databricks clusters, notebooks, and Spark jobs.
- Collaborate with Data Architects, Business Analysts, Data Scientists, and Healthcare SMEs.
- Develop reusable frameworks and CI/CD pipelines using Azure DevOps.
- Support production deployments, monitoring, troubleshooting, and incident resolution.
Technical Skills
- 6+ years of experience in Data Engineering.
- 5+ years of hands-on experience with Azure Databricks.
- Strong expertise in:
- Azure Databricks
- PySpark
- Spark SQL
- Python
- SQL
- Azure Synapse Analytics
- Delta Lake
- Experience with Data Warehousing concepts, Data Modeling (Star/Snowflake Schema), and ETL/ELT development.
- Knowledge of Medallion Architecture (Bronze, Silver, Gold layers).
- Experience with Git, Azure DevOps, and CI/CD implementation.
- Strong understanding of performance tuning and optimization techniques in Databricks.
US Healthcare Experience (Mandatory)
- Minimum 2+ years of experience working in the US Healthcare domain.
- Strong understanding of:
- Claims Data
- Member Data
- Provider Data
- Clinical Data
- EHR/EMR Systems
- HL7/FHIR Standards
- HIPAA Compliance
- PHI/PII Data Handling
- Experience supporting healthcare analytics and reporting initiatives.
Educational Qualification
- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
Key Competencies
- Strong analytical and problem-solving skills.
- Excellent communication and stakeholder management skills.
- Ability to work independently in a fast-paced environment.
- Experience collaborating with global teams and business stakeholders.
Interview Rounds 3 rounds
Shift Timings 2 PM to 11 PM
Location: Bangalore
Work Mode: Hybrid - 2 days WFO and 3 days WFH
Regards,
naveen.vediyappan@sagility.com
No Referrers Available
There are currently no referrers available for this job. You can still apply, will let you know once there is any referrer available.