Data Engineer (Databricks)

OSI Digital is looking for a Data Engineer with strong expertise in Azure Databricks, PySpark, SQL, and Power BI to build scalable data pipelines and analytics solutions. The ideal candidate will design modern Lakehouse architectures, transform raw data into actionable insights, and develop high-performance reporting solutions.

Responsibilities:

• Design and develop end-to-end data pipelines using Azure Databricks and PySpark

• Build and optimize Delta Lake tables using Medallion Architecture (Bronze, Silver, Gold)

• Develop scalable ETL/ELT workflows for large-scale data processing

• Create interactive Power BI dashboards and reports using DAX and Power Query

• Write and optimize SQL queries for data transformation and analytics

• Collaborate with business analysts to translate business requirements into technical solutions

• Ensure data quality, performance, and scalability across data platforms

• Support deployment, monitoring, and troubleshooting of data engineering solutions

1. 36 years of experience in Data Engineering

2. Strong expertise in Python, PySpark, and Spark SQL

3. Hands-on experience with Azure Databricks, Delta Lake, Unity Catalog, and Delta Live Tables

4. Experience with Azure Databricks Workflows and Lakehouse architecture

5. Strong SQL development and query optimization skills

6. Expertise in Power BI, DAX, and Power Query

7. Strong analytical, troubleshooting, and communication skills

• Experience with Azure Data Factory, Azure Synapse Analytics, or Azure Data Lake

• Knowledge of CI/CD and DevOps practices for data platforms

• Microsoft Azure Data Engineering Certification

• Experience working in Agile/Scrum environments

• Has never worked directly with business stakeholders

• Lacks hands-on experience with Azure Databricks and PySpark

• Has limited exposure to Power BI and SQL-based data analytics