Lead Data Engineer - AWS Glue (5-10 yrs)

Job Summary :

NetConnect Global is hiring a Lead Data Engineer for a reputed client. We are looking for a highly skilled professional with expertise in AWS Data Services, PySpark, Data Warehousing, NoSQL Databases, and Enterprise Data Platforms to drive large-scale data transformation initiatives.

Key Skills :

- AWS Glue

- Amazon S3

- Amazon Redshift

- Amazon EMR

- PySpark / Apache Spark

- Data Engineering

- ETL / ELT Pipelines

- Couchbase

- DynamoDB

- MongoDB

- Data Warehousing

- CI/CD Pipelines

- GitLab

- AWS CodePipeline

- Data Governance

- Data Quality

- Microservices Architecture

Responsibilities :

- Design, develop, and optimize large-scale data pipelines using AWS services and PySpark.

- Build and maintain scalable ETL/ELT frameworks and data ingestion pipelines.

- Design and implement ODS layers using NoSQL databases such as Couchbase, DynamoDB, or MongoDB.

- Develop high-performance data processing solutions using Spark.

- Implement CI/CD pipelines and deployment automation.

- Ensure data quality, governance, metadata management, and security compliance.

- Support microservices-based architectures and enterprise data platforms.

- Collaborate with business, analytics, and engineering teams to deliver data-driven solutions.

- Mentor junior engineers and drive engineering best practices.

Required Experience :

- 5+ years of Data Engineering experience with 2+ years in a Lead role.

- Strong hands-on expertise in AWS Glue, S3, Redshift, and EMR.

- Advanced knowledge of PySpark/Spark for large-scale data processing.

- Experience with Couchbase, DynamoDB, MongoDB, or similar NoSQL databases.

- Strong understanding of ETL/ELT design patterns and data architecture.

- Experience with GitLab, AWS CodePipeline, or similar CI/CD tools.

- Exposure to Retail, POS, MMS, or Transaction Data environments is highly preferred.

- Excellent communication and stakeholder management skills.