Databricks Architect

Experience: 1015 Years
Notice Period: Immediate to 20 Days Preferred

Job Description

We are looking for a Databricks Architect with strong hands-on experience in Python, PySpark, SQL, and Databricks to design, build, and optimize scalable cloud-based data engineering solutions. The ideal candidate should have extensive experience in architecting modern data platforms while remaining actively involved in development and technical leadership.

Key Responsibilities

• Design and architect enterprise-scale data platforms using Databricks.
• Develop and optimize ETL/ELT pipelines using Python, PySpark, and SQL.
• Build scalable data ingestion, transformation, and data processing frameworks.
• Optimize Spark jobs for performance, scalability, and cost efficiency.
• Design and implement Delta Lake, Medallion Architecture, and data governance best practices.
• Collaborate with business stakeholders, data engineers, and analytics teams to deliver high-quality data solutions.
• Lead technical discussions, conduct code reviews, and mentor engineering teams.
• Ensure data quality, security, monitoring, and performance optimization.
• Work closely with DevOps teams for CI/CD implementation and deployment automation.
• Troubleshoot production issues and provide technical guidance.

Required Skills

• 10+ years of experience in Data Engineering.
• 4+ years of hands-on experience with Databricks.
• Strong programming expertise in Python.
• Strong hands-on experience with PySpark.
• Excellent SQL skills with experience in query optimization.
• Experience designing data lakes and modern data platforms.
• Expertise in Delta Lake, Spark optimization, and Databricks Workflows.
• Experience with cloud platforms such as Azure, AWS, or GCP.
• Strong understanding of data modeling, ETL frameworks, and distributed computing.
• Experience with Git, CI/CD, and Agile methodologies.

Preferred Skills

• Experience with Azure Data Factory, Azure Synapse, or Microsoft Fabric.
• Knowledge of Terraform or Infrastructure as Code.
• Experience with Kafka/Event Streaming.
• Databricks Certification is preferred.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.

Notice Period

• Immediate Joiners or candidates with a notice period of less than 20 days will be given preference.