Job Summary

We are looking for a Lead Data Engineer to lead the effort for designing, building, and maintaining robust, scalable, and high-performance data infrastructure. In this role, you will work on end-to-end data pipelines, ensuring reliable data availability for analytics, machine learning, and business applications.

Responsibilities

Lead mentor a team of Data Engineers - code reviews, design reviews, growth plans
Design and operate ETL/ELT pipelines in Python for batch and streaming workloads
Own data modelling across PostgreSQL / Supabase (OLTP) and Cassandra (wide-column)
Build and run orchestration on Apache Airflow - DAG design, SLAs, retries, backfills
Deploy and operate the platform on AWS (EC2-based, self-managed stack)
Establish data governance: catalog, lineage, ownership, MDM, data quality, profiling, observability
Drive engineering excellence - Agile delivery, CI/CD, testing, IaC, code review

Product Mindset

Treat the data platform as a long-lived product, not a project:

Own outcomes across the full lifecycle - design, delivery, operations, iteration
Optimise for scale, reliability, and maintainability with measurable SLAs and cost targets
Prioritise based on user value (analytics, ML, product teams as customers)
Ship iteratively; measure adoption, quality, and reliability continuously

Must Have

5-8 years in data engineering with proven experience leading and mentoring engineers
Expert Python and strong SQL
Production experience with PostgreSQL, Supabase, and Apache Cassandra
Hands-on ownership of Apache Airflow at scale
Comfortable running data workloads on AWS EC2 (Linux, networking, self-managed services)
Strong ETL, data modelling, orchestration fundamentals
Practical experience implementing data governance - MDM, data quality, data profiling, observability, lineage/catalog
Agile, CI/CD, and coding best practices as everyday habits

Nice to Have

Data security and compliance - encryption (KMS/at-rest/in-transit), PII handling, GDPR/PDPA
Streaming (Kafka, Kinesis) and lakehouse (Iceberg, Delta) exposure
IaC (Terraform, Ansible) for self-managed AWS stacks

Education

Bachelor's or Master's in Computer Science, Engineering, or related field.

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

Lead Data Engineer

Job Description

No Referrers Available