Databricks Consultant

Educational Requirements

Bachelor of Engineering,BTech,BCA,BSc,ME,MTech,MCA,MSc,MBA

Service Line

Data Analytics Unit

Responsibilities

• Design, develop, and maintain scalable Databricks-based data engineering solutions.
• Build and optimize complex ETL/ELT pipelines using Databricks, PySpark, Spark SQL, and Python.
• Develop robust data ingestion and transformation frameworks for structured and unstructured data.
• Implement and manage Delta Lake and Lakehouse Architecture solutions.
• Design, monitor, and optimize enterprise-grade data pipelines for performance, scalability, and reliability.
• Perform performance tuning and troubleshooting of Databricks and Apache Spark workloads.
• Develop reusable frameworks and best practices for Data Engineering and Data Integration.
• Collaborate with architects, analysts, and business stakeholders to translate business requirements into technical solutions.
• Ensure data quality, governance, security, and compliance standards are followed.

Technical and Professional Requirements

• 68 years of experience in Data Engineering and Big Data technologies.
• Strong hands-on experience with Databricks and PySpark.
• Expertise in Apache Spark, Spark SQL, and distributed data processing.
• Strong programming skills in Python and SQL.
• Experience building and managing enterprise-scale ETL/ELT pipelines.
• Hands-on experience with Delta Lake and Lakehouse Architecture.
• Strong understanding of Data Warehousing, Data Modeling, and Data Integration concepts.
• Experience working with large-scale structured and unstructured datasets.

Preferred Skills

• Technology->OpenSystem->Python - OpenSystem->Python
• Technology->Data Engineering->Databricks