Experience
10 - 15 yrs
Job Location
Pune, India
Vacancy
1
Designation
Data Engineering Manager
Job Type
Not specified
Job Description
Positions: 1 Full Time
Experience
10 15 Years
Job Description Engineering Manager Data Engineering Position Overview
Job Title: Engineering Manager Data Engineering
Location: Pune, Maharashtra, India
Work Mode: Hybrid
Experience Required: 10-15 Years
Open Positions: 1
Job Summary We are seeking an experienced and dynamic Engineering Manager Data Engineering to lead cross-functional data engineering teams and drive the design, development, and delivery of enterprise-scale data platforms. This is a 50-50 blend of technical leadership and delivery management role where you will combine hands-on technical expertise with strategic people leadership. You will be responsible for leading multiple squads (20+ members), ensuring exceptional delivery quality, and building scalable data solutions using Databricks on cloud platforms. The ideal candidate will have a proven track record of end-to-end delivery, strong stakeholder management capabilities, and deep expertise in modern data engineering practices, particularly in the Banking/Financial Services sector.
Key Responsibilities Technical Leadership Architecture (50%) - Lead the design and implementation of enterprise-scale data platforms using Databricks as the primary technology stack.
- Define and enforce technical architecture, coding standards, and engineering best practices across all data engineering initiatives.
- Drive adoption of modern data engineering patterns including Medallion Architecture, Delta Lake, and Delta Live Tables (DLT).
- Conduct comprehensive solution design reviews and provide technical guidance on complex data engineering challenges.
- Review code quality, performance optimization strategies, and ensure adherence to data security, governance, and compliance standards.
- Mentor team members on advanced Databricks features, PySpark optimization, and cloud-native data architectures.
- Stay current with emerging technologies and industry trends; champion innovation within the team.
- Ensure implementation of robust data modeling strategies (Star Schema, Snowflake, Data Vault) and ETL/ELT design patterns.
- Own end-to-end delivery accountability for multiple data engineering initiatives, from requirements gathering through production deployment and post-go-live support.
- Lead and manage 2+ squads (20+ members) with a strong focus on delivery excellence and quality outcomes.
- Develop comprehensive project plans, timelines, and resource allocations with accurate estimation, budgeting, and costing.
- Execute sprint planning and management; track progress against milestones and KPIs.
- Proactively identify, assess, and mitigate delivery risks and dependencies.
- Ensure production readiness through rigorous testing, validation, and release management processes.
- Drive continuous improvement in engineering processes, tools, and team productivity.
- Manage stakeholder expectations, communicate progress transparently, and prioritize business requirements effectively.
- Lead, mentor, and develop a high-performing team of Data Engineers and Technical Leads.
- Conduct regular one-on-one meetings, performance reviews, and career development discussions.
- Support hiring initiatives, onboarding programs, and capability development plans.
- Foster a culture of collaboration, innovation, accountability, and continuous learning.
- Identify and nurture talent; create pathways for career growth within the team.
- Ensure platform reliability, availability, and optimal performance in production environments.
- Drive root cause analysis for production incidents and implement preventive measures.
- Improve monitoring, alerting, observability, and incident response capabilities.
- Optimize cloud infrastructure costs and resource utilization across Azure, AWS, or GCP.
- Collaborate effectively with Product Owners, Solution Architects, Business stakeholders, and Platform teams.
- Communicate technical decisions, risks, and delivery status to senior leadership and business partners.
- Databricks Expertise: Databricks Workspace, Delta Lake, Delta Live Tables (DLT), Unity Catalog, Databricks Workflows, Auto Loader, Structured Streaming
- Programming: Python, PySpark, Spark SQL (SQL is mandatory; Python/PySpark is good-to-have)
- Data Engineering Patterns: ETL/ELT design, Medallion Architecture, Batch and Streaming data pipelines
- Data Modeling: Star Schema, Snowflake Schema, Data Vault methodologies
- Database Technologies: SQL Server, Oracle, Snowflake, PostgreSQL
- Hands-on experience with Microsoft Azure (primary) and/or AWS
- Cloud Storage: ADLS (Azure Data Lake Storage), S3, or GCS
- Understanding of cloud-native architectures and infrastructure optimization
- Git and version control best practices
- Azure DevOps or GitHub for CI/CD pipeline management
- Infrastructure as Code (Terraform preferred)
- Release management and deployment automation
- Engineering leadership with proven experience leading multiple squads (20+ members)
- Agile delivery and project management expertise
- Strong planning, estimation, budgeting, and costing skills
- Stakeholder management and executive communication
- Risk management and conflict resolution
- Coaching, mentoring, and team development capabilities
- Solution and design thinking with end-to-end delivery ownership
- Excellent communication and interpersonal skills
- Banking/Financial Services industry experience (highly preferred)
- Understanding of financial data governance, compliance (GDPR, SOX), and security requirements
- Total Experience: 10-15+ years in Data Engineering and related roles
- Leadership Experience: 3-5+ years leading engineering teams or squads
- Databricks Experience: Hands-on, production-grade experience with Databricks on Azure, AWS, or GCP (mandatory)
- Enterprise-Scale Platform Development: Proven experience building and delivering enterprise-scale data platforms
- End-to-End Delivery: Demonstrated ability to own complete delivery lifecycle from requirements gathering, design, development, testing, production deployment, and post-go-live support
- Performance Tuning Optimization: Experience optimizing data pipeline performance, cost management, and production support
- Solution Architecture: Strong background in designing and implementing complex data solutions
- Databricks Certified Data Engineer Professional
- Azure Data Engineer Associate or Azure Solutions Architect Expert
- AWS Certified Data Analytics Specialty
- Bachelor s degree in Computer Science, Engineering, or related field
- MLflow and ML pipeline orchestration
- Databricks Asset Bundles (DABs)
- Apache Kafka or Azure Event Hubs experience
- AI/ML pipelines and Generative AI knowledge
- Data Mesh or Data Fabric architecture experience
- Databricks Genie or AI-assisted development tools
- NoSQL databases (MongoDB, Cassandra)
- Advanced monitoring and observability tools
- Technical Expertise: Deep hands-on knowledge of modern data engineering and cloud technologies
- Leadership: Ability to inspire and lead high-performing teams toward ambitious goals
- Delivery Excellence: Demonstrated track record of on-time, quality delivery of complex projects
- Strategic Thinking: Capability to align technical solutions with business objectives
- Communication: Exceptional verbal and written communication skills; ability to articulate complex concepts to diverse audiences
- Problem-Solving: Strong analytical and troubleshooting capabilities
- Adaptability: Comfortable in fast-paced, dynamic environments with evolving requirements
- Accountability: Takes ownership of outcomes and drives results
- A hands-on technical leader who can code and architect solutions while managing teams
- Someone with proven expertise in Databricks and enterprise data platform development
- A delivery-focused professional with strong planning and estimation capabilities
- A leader who has successfully managed multiple squads and driven complex, multi-initiative programs
- Banking/Financial Services industry experience is a significant plus
- An individual with excellent communication skills who can engage effectively with all stakeholder levels
- A champion of quality, continuous improvement, and operational excellence
No Referrers Available
There are currently no referrers available for this job. You can still apply, will let you know once there is any referrer available.