Data Engineer - Lead

Data Engineer - Lead

Company: Iris Software

Location: Noida, UP, India

Job Description

a) Mandatory Skills

• Databricks
• Apache Spark
• Python and SQL
• 8+ years strong hands-on experience with:
• Experience building scalable ETL/ELT data ingestion, Transformation and curation pipelines
• Strong understanding of Delta Lake (ACID, schema evolution, versioning)
• Experience with Lakehouse architecture (ingestion + curated layers)
• Ability to implement source-to-target mappings and physical schemas
• Experience implementing data quality, validation, and monitoring frameworks
• Strong understanding of distributed data processing
• Experience with CI/CD, Git, and pipeline automation
• Strong banking/financial services domain experience
• Ability to deliver production-grade, scalable data products end-to-end
• Experience working in Agile, cross-functional teams
• Great interpersonal and communication skills
• Can work independently with minimum supervision
• Flexible and adaptable working style to work with multiple stakeholders
• Open to work in UK Shift (10 AM to 9 PM IST with 8 hours of productive work)

b) Good to Have Skills

• Exposure to Snowflake (hybrid environments)
• Knowledge of Scala
• Experience with data governance and catalog tools
• Exposure to workflow orchestration tools (Airflow, etc.)
• Experience in regulated environments
• Familiarity with financial domains (Payments, FX, Accounts)
• Experience in data product-oriented delivery approaches
• AI-assisted data product development experience

Mandatory Competencies

• Data AI - Data Engineering - Data Quality Validation
• Big Data - Big Data - Pyspark
• Data AI - MLOPS - Python
• Database - Database Programming - SQL
• Data AI - Data Engineering - Apache Kafka
• Beh - Communication and collaboration