EY - GDS Consulting - AI And DATA -AWS Data Engineer - Manager

Role Experience Guide Primary Skill Area The opportunity Your key responsibilities AWS Data Engineering

• Design and build production-grade AWS data pipelines using Amazon S3, AWS Glue, PySpark, Athena, Redshift, EMR, MWAA/Airflow, Step Functions, Lambda, EventBridge, CloudWatch, IAM, and KMS.
• Develop reusable ingestion frameworks supporting batch, streaming, event-driven, CDC, API-based, file-based, database, and third-party service integration patterns.
• Build curated raw, standardised, trusted, and consumption layers using scalable lakehouse design patterns, partitioning, metadata management, and file-format optimisation.
• Optimise Spark/Glue/EMR workloads for performance, cost efficiency, scalability, and operational stability.

AWS Platform Engineering

• Create reusable AWS platform accelerators for onboarding, pipeline templates, orchestration, monitoring, reconciliation, deployment, logging, and support runbooks.
• Implement Git connectivity, branching strategy, pull requests, code reviews, CI/CD, Infrastructure as Code, controlled releases, and environment promotion.
• Integrate AWS data platforms with enterprise APIs, source applications, messaging/event services, governance tools, security platforms, and downstream analytics consumers.
• Partner with infrastructure, IAM, network, DBA, application, and support teams to resolve connectivity, access, deployment, and production issues.
• SageMaker, AI Integration Agentic Enablement
• Integrate AWS data platforms with Amazon SageMaker for data preparation, feature engineering, model training, deployment, MLOps workflows, and inference-ready data products.
• Support SageMaker Pipelines, Feature Store, Model Registry, Model Monitor, Bedrock where relevant, vector stores, semantic search, and RAG-ready data products.
• Apply AI-assisted and agentic operations for anomaly detection, schema drift detection, failed-job diagnosis, data quality recommendations, documentation generation, and incident summarisation.

Governance, Security Data SRE

• Implement AWS data security controls including IAM least privilege, KMS encryption, Secrets Manager, VPC endpoints, Lake Formation, Glue Data Catalog, Macie, CloudTrail, and audit-ready access patterns.
• Integrate with Immuta, Microsoft Purview, Collibra, enterprise IAM, monitoring platforms, data quality tools, and downstream analytics/AI consumers.
• Own Data SRE responsibilities including CloudWatch observability, SLA/SLO tracking, alerting, retry logic, restartability, root-cause analysis, incident response, and production reliability management.

Skills and attributes for success

• Core platform: AWS Glue, Amazon S3, Athena, Redshift, EMR, MWAA/Airflow, Step Functions, Lambda, EventBridge, CloudWatch.
• AI and GenAI: Amazon SageMaker, SageMaker Pipelines, Feature Store, Model Registry, Model Monitor, Bedrock optional, RAG-ready data products, Agentic AI.
• Engineering: Python, PySpark, SQL, APIs, Git, CI/CD, Shell scripting, unit testing, integration testing, data pipeline testing.
• Cloud and DevOps: Terraform/OpenTofu, CloudFormation, GitHub Actions, Azure DevOps, Jenkins, Docker, Kubernetes/EKS, policy-as-code.
• Governance and reliability: IAM, KMS, Lake Formation, Glue Data Catalog, Macie, Immuta, Purview, CloudTrail, data quality, observability, Data SRE, FinOps.

To qualify for the role, you must have

• 8-11 years of experience in data engineering, data platform operations, analytics engineering, platform engineering, or AI platform enablement.
• Strong hands-on implementation experience with cloud data platforms, APIs, Git connectivity, CI/CD, governed access patterns, SRE practices, and production operations.
• Preferred certifications aligned to the relevant cloud/platform stack, data engineering, DevOps, security, governance, and AI/

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.