Experience :- 5-12 Years
Work Type:- Onsite(Bengaluru, India)
Job Type:- Contract

Skills Required:-

Python (Strong) Advanced SQL Airflow/Spark/Kafka Cloud (AWS/Azure/GCP) Modern DW (Snowflake/Databricks) Data Pipeline ETL/ELT

Job Description

Key Responsibilities

Design, develop, and maintain scalable data pipelines using modern orchestration frameworks (Apache Airflow, Apache Spark, Kafka)

Build data ingestion solutions from diverse sources including ERP systems (SAP, Oracle), CRM platforms (Salesforce, Workday), and cloud data warehouses

Implement data quality frameworks, validation rules, and monitoring to ensure high-quality data across all applications

Optimize data processing workflows for performance, reliability, and cost-efficiency in cloud-native environments

Collaborate with BI Engineers and Data Modelers to ensure seamless data flow from source systems to analytical dashboards

Write clean, reusable, and well-tested code following software engineering best practices (code reviews, unit testing, CI/CD)

Required Qualifications:-

5-12 years of proven experience in building production-grade data pipelines and ETL/ELT solutions

Strong proficiency in Python for data engineering (pandas, PySpark, SQLAlchemy, data processing libraries)

Advanced SQL skills including complex joins, window functions, query optimization, and performance tuning

Hands-on experience with cloud platforms (AWS, Azure, or GCP) and cloud-native data services

Familiarity with orchestration frameworks such as Apache Airflow, Apache Spark, Kafka, or similar tools

Experience with modern data warehouses (Snowflake, Databricks, BigQuery, Redshift) including data modeling and optimization

Knowledge of ERP/CRM systems integration (SAP, Oracle, Salesforce, Workday) - extracting data via APIs, connectors, or database access

Strong understanding of data quality principles, testing frameworks, and validation methodologies

Experience in leveraging or critically thinking about how to integrate AI into work processes, decision-making, or problem-solving.

Nice-to-Have Qualifications

Experience with AI-powered productivity tools (Cursor, Windsurf, Claude, GitHub Copilot) for accelerated development

Hands-on Java experience and understanding of enterprise application design patterns

Experience working with ServiceNow platform or similar enterprise workflow platforms

Knowledge of real-time streaming architectures (Kafka Streams, Flink) for industrial/IoT analytics

Experience with containerization (Docker, Kubernetes) and modern CI/CD practices

Familiarity with AI/ML pipeline integration, feature engineering, and model serving infrastructure

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

Senior Data Engineer

Job Description

No Referrers Available