Lead AI Engineer

Job Description

We are looking for an AI-Native Developer with deep expertise in building agentic systems, LLM-powered workflows, and autonomous data pipelines on AWS.

This role goes beyond traditional data engineering you will design and build AI-driven ingestion and transformation systems, leveraging LLMs, code-generation agents, and orchestration frameworks to accelerate development, improve reliability, and reduce manual effort.

You will work extensively with tools like Claude Code, Cursor, and AWS AgentCore, and will be responsible for building production-grade AI agents that can interpret specifications, generate pipelines, validate outputs, and continuously improve through feedback loops.

Agentic System Design Development

• You will be #LI-hybrid based in Hyderabad and reporting to Director Engineering
• Design and implement AI agents for data engineering workflows (spec parsing, code generation, validation, reconciliation).
• Build multi-step agent orchestration pipelines using LLMs and tool integrations.
• Develop autonomous systems capable of reverse engineering legacy pipelines and generating cloud-native equivalents.

AI-Augmented Engineering

• Leverage Claude Code, Cursor, and similar tools to accelerate development cycles.
• Build feedback loops for self-healing pipelines and auto-correction of data issues.
• Integrate LLMs for schema inference, transformation logic generation, and anomaly detection.

AWS-Native Development

• Build scalable pipelines using S3, Glue, Lambda, DynamoDB, EventBridge, and streaming services (MSK/SQS).
• Work with AWS AgentCore (or equivalent frameworks) to deploy and manage agentic workloads.
• Design systems that combine event-driven architectures with AI decision layers.

Intelligent Ingestion Transformation

• Develop AI-driven ingestion frameworks supporting batch and near real-time processing.
• Implement automated data validation, reconciliation, and lineage tracking using AI agents.
• Ensure parity with legacy systems using AI-assisted validation techniques.

CI/CD AI Integration

• Embed agentic workflows into CI/CD pipelines (Jenkins/Harness).
• Automate testing using LLM-generated test cases and validation strategies.
• Integrate security scanning (SonarQube, Veracode) with AI-assisted remediation suggestions.

Data Governance Control (AI-Enhanced)

• Implement AI-powered data quality, lineage, and compliance monitoring.
• Build intelligent dashboards with automated evidence generation.
• Enable proactive detection of data issues using anomaly detection agents.

Collaboration Innovation

• Partner with Product, Architecture, and AI teams to define agentic engineering standards.
• Work closely with legacy system experts to translate logic into AI-understandable specifications.
• Contribute to building an AI-first engineering culture within the organization.

QualificationsCore AI Agentic Development

• 5+ years of overall experience
• Experience building AI agents / agentic workflows (multi-step reasoning, tool use, orchestration).
• Hands-on experience with Claude Code, Cursor, or similar AI coding environments.
• Experience with AWS AgentCore or equivalent agent orchestration frameworks.
• Understanding of prompt engineering, evaluation, and LLM optimization techniques.

Programming Data Engineering

• Python (advanced), PySpark
• Strong experience in data pipeline design and distributed processing

AWS Cloud

• S3, Glue, Lambda, DynamoDB, EventBridge, MSK/SQS
• Experience building event-driven and serverless architectures

DevOps Automation

• Jenkins/Harness, Bitbucket
• SonarQube, Veracode
• Experience integrating AI into CI/CD workflows

Data Governance Security

• Data quality frameworks, lineage tracking
• IAM, encryption, secure configuration

Core Outcomes Success Metrics

• Production-grade AI-driven ingestion and transformation systems deployed on AWS
• Measurable improvement in developer productivity through AI-assisted engineering
• High accuracy in AI-generated pipelines with validated parity to legacy systems
• Automated data quality and governance with minimal manual intervention
• Robust agentic workflows embedded in CI/CD pipelines

Stakeholder Interactions

• Engineering Leadership: Technical direction and innovation
• Product Owners BAs: Requirements and prioritization
• AI/ML Engineers: Agent design and optimization
• Data Stewards: Governance validation
• DevSecOps: Secure and scalable deployment