Job Description
Lead and mentor a team of data engineers, providing technical direction and career guidance.
Define the target data platform architecture for migrating from on-prem HDFS/Hive to Cloud Object Storage (e.g., AWS S3, Azure Data Lake Storage, or GCP Cloud Storage).
Select and integrate cloud-based compute and query engines (e.g., Spark).
Lead the design of ingestion, transformation, and storage patterns optimized for scalability, cost-efficiency, and performance in the cloud.
Define security, encryption, and compliance controls for sensitive enterprise data in the cloud.
_x000D_ Essential functions_x000D_
_x000D_
-
Develop and own the migration roadmap, including phased transition from on-prem to cloud while minimizing business disruption.
-
Oversee data migration strategies (bulk historical loads, incremental sync, and cutover).
-
Define and enforce coding standards, CI/CD pipelines, and automated testing for data pipelines.
-
Partner with Data Architects, Cloud Engineers, and Security teams to align platform design with enterprise standards.
_x000D_ Qualifications_x000D_
_x000D_Mandatory
-
Proven experience leading data engineering teams, including distributed teams across multiple geographies and time zones.
-
Effective in managing cross-team collaboration with architects, product managers, and operations.
-
Scala and Python
-
Apache Spark (batch & streaming) must!
-
Deep knowledge of HDFS internals and migration strategies.
-
Experience with Apache Iceberg (or similar table formats like Delta Lake / Apache Hudi) for schema evolution, ACID transactions, and time travel.
-
Running Spark and/or Flink jobs on Kubernetes (e.g., Spark-on-K8s operator, Flink-on-K8s).
-
Experience with distributed blob storages like Ceph or AWS S3 and similar
-
Building ingestion, transformation, and enrichment pipelines for large-scale datasets.
-
Infrastructure-as-Code (Terraform, Helm) for provisioning data infrastructure.
-
Strong communication skills
_x000D_ Would be a plus_x000D_
_x000D_Nice to have:
-
Experience with Apache Flink
-
Apple experience preferred (to enable him/her to get up to speed on our tooling set quickly and more independently)
_x000D_ We offer_x000D_
- Opportunity to work on bleeding-edge projects
- Work with a highly motivated and dedicated team
- Competitive salary
- Flexible schedule
- Benefits package - medical insurance, sports
- Corporate social events
- Professional development opportunities
- Well-equipped office
No Referrers Available
There are currently no referrers available for this job. You can still apply, will let you know once there is any referrer available.