Job Description
An ETL developer in Azure is responsible for designing, implementing, and maintaining data pipelines that securely move, transform, and store data while ensuring performance, quality, and traceability, utilizing services like Azure Data Factory, Azure Data Warehouse, Key Vault, Azure SQL, Azure DevOps, and Git Flow.
Core Responsibilities
Data Extraction and Loading
ETL developers extract data from multiple sources, including relational databases, APIs, flat files, or event streaming platforms (Event Hubs, Kafka, IoT Hub). They ensure secure connections via Azure Key Vault for credentials and secrets and perform incremental or full loads into centralized storage solutions like Azure Data Lake Storage (ADLS Gen2) or Azure SQL / Synapse Data WarehouseSQL Server
Data Transformation
During transformation, data is cleansed, normalized, deduplicated, and enriched to meet business logic. ETL developers implement these transformations using Azure Data Factory mapping data flows, Azure Databricks (PySpark/Scala/Python/SQL notebooks), T-SQL, or custom scripts. Transformations include calculations, aggregations, joining multiple datasets, and handling Slowly Changing Dimensions (SCDs) for historical tracking SQL Server
Pipeline Orchestration and Automation
ETL developers design robust, modular, and parameterized ETL/ELT pipelines using Azure Data Factory, leveraging control flow, triggers, and integration runtimes. They implement retry mechanisms, error handling, and incremental updates to maintain idempotency and pipeline reliability. Pipelines can be batch-based or real-time streaming, incorporating Databricks Structured Streaming.
No Referrers Available
There are currently no referrers available for this job. You can still apply, will let you know once there is any referrer available.