We are seeking a talented individual to join our Mercer Team at Marsh. This role will be based in Noida/Gurgaon. This is a hybrid role that has a requirement of working at least three days a week in the office.

Senior Principal Engineer - Data Engineering

We will count on you to:

As a Data Engineer, you will be responsible for designing and implementing scalable data pipelines and AI Based solution using Databricks. You will handle end-to-end ETL/ELT processes, manage large datasets, and work with tools like Python, PySpark, and AWS S3 to ensure data is transformed and optimized for analytical use.

You ll work on cutting-edge cloud and hybrid data projects, transforming raw data into meaningful insights and AI Analytics. You ll be hands-on from day one, collaborating closely with architects and business stakeholders.

What you need to have:

Develop and maintain data pipelines using Databricks and the Medallion Architecture (Bronze, Silver, Gold layers).
Design AI Based Solution using Databricks Genie and E2E integration.
Knowledge of exposing/consuming Databricks features via API using cloud-native tools or other application.
Write data transformation scripts using Python and PySpark.
Store and manage real time data in AWS S3 and integrate with other cloud-based services.
Use SQL to query, clean, and manipulate large datasets.
Collaborate with cross-functional teams to ensure data is accessible for business intelligence and analytics.
Monitor and troubleshoot data pipelines for performance and reliability.
Document data processes and follow best practices for scalability and maintainability.
Ingest and process structured and unstructured data across batch and streaming sources.

What makes you stand out

Experience with Databricks components like : Pipeline, scheduled / event based job , Genie , Unity Catalog and Datawarehouse.
Proficiency in Python, PySpark, and SQL for data processing and transformation using AWS S3 data.
Experience in Data Governance , data access security , and information of configuring Job compute for different Jobs in Databricks.
Familiarity with version control using Git.
Understanding of Databricks API and its integration with different Tools and application.
Bulk data and real time data streaming understanding.
Experience with Delta Lake and other Databricks technologies. Knowledge of additional AWS services (e.g., Athena, Glue, Lambda, S3, DMS ).

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

Senior Principal Engineer - Data Engineering

Job Description

No Referrers Available