Lead Data Engineer

# Lead Data Engineer

**Experience:** 58 years | **Location:** Bangalore/India | **Type:** Full-time

ROLE

Lead the design, build, and operations of a production-grade data platform. Own end-to-end pipelines, set engineering standards, and mentor a team of Data Engineers delivering data for analytics, ML, and product use cases.

RESPONSIBILITIES

• Lead & mentor a team of Data Engineers — code reviews, design reviews, growth plans
• Design and operate ETL/ELT pipelines in Python for batch and streaming workloads
• Own data modelling across PostgreSQL / Supabase (OLTP) and Cassandra (wide-column)
• Build and run orchestration on Apache Airflow — DAG design, SLAs, retries, backfills
• Deploy and operate the platform on AWS (EC2-based, self-managed stack)
• Establish data governance: catalog, lineage, ownership, MDM, data quality, profiling, observability
• Drive engineering excellence — Agile delivery, CI/CD, testing, IaC, code review

PRODUCT MINDSET

Treat the data platform as a long-lived product, not a project:

• Own outcomes across the full lifecycle — design, delivery, operations, iteration
• Optimise for scale, reliability, and maintainability with measurable SLAs and cost targets
• Prioritise based on user value (analytics, ML, product teams as customers)
• Ship iteratively; measure adoption, quality, and reliability continuously

MUST-HAVE

• 5–8 years in data engineering with proven experience leading and mentoring engineers
• Expert Python and strong SQL
• Production experience with PostgreSQL, Supabase, and Apache Cassandra
• Hands-on ownership of Apache Airflow at scale
• Comfortable running data workloads on AWS EC2 (Linux, networking, self-managed services)
• Strong ETL, data modelling, orchestration fundamentals
• Practical experience implementing data governance — MDM, data quality, data profiling, observability, lineage/catalog
• Agile, CI/CD, and coding best practices as everyday habits

GOOD-TO-HAVE

• Data security and compliance — encryption (KMS/at-rest/in-transit), PII handling, GDPR/PDPA
• Streaming (Kafka, Kinesis) and lakehouse (Iceberg, Delta) exposure
• IaC (Terraform, Ansible) for self-managed AWS stacks

EDUCATION

Bachelor's or Master's in Computer Science, Engineering, or related field.