Experience
7 - 12 yrs
Job Location
Noida, India
Vacancy
1
Designation
Azure Data Engineer
Job Type
ONSITE
Job Description
Job Title:
Job Summary : We are looking for a Data Engineer who is passionate about building scalable, secure, and highperformance data architectures. In this role, you will be the primary architect and developer for our Azure Data Lake ecosystem. Your expertise in Scala and Apache Spark will be critical in transforming massive raw datasets into actionable data products for our analytics
Responsibilities:
Data Lake Architecture: Design and implement ADLS Gen2 storage structures (Bronze/Silver/Gold layers) ensuring optimal partitioning and security.
Pipeline Development: Build and maintain complex ETL/ELT pipelines using Scala on Azure Databricks or HDInsight.
Performance Tuning: Optimize Spark jobs written in Scala to ensure low latency and costeffective compute usage.
Data Orchestration: Use Azure Data Factory (ADF) to schedule and monitor endtoend workflows.
Streaming Batch: Develop realtime data ingestion pipelines using Azure Event Hubs or Kafka integration.
Goverce Security: Implement rolebased access control (RBAC), encryption, and data masking using Microsoft Purview .
MustHave Skills / Requirements:
7+ years overall backend development experience in production environments
Primary Language: Advanced proficiency in Scala (Functional Programming, Spark APIs).
Azure Big Data Stack: Expert knowledge of Azure Data Lake Storage Gen2 , Databricks, and Data Factory.
Distributed Computing: Deep understanding of Apache Spark (Data Frames, Datasets, Spark SQL, and optimization techniques like Caching and Broadcast joins).
SQL Mastery: Ability to write and optimize complex SQL queries for Synapse or Azure SQL.
DevOps: Experience with Azure DevOps , CI/CD pipelines, and version control (Git).
Data Modeling: Familiarity with Star/Snowflake schemas
, Job Summary : We are looking for a Data Engineer who is passionate about building scalable, secure, and highperformance data architectures. In this role, you will be the primary architect and developer for our Azure Data Lake ecosystem. Your expertise in Scala and Apache Spark will be critical in transforming massive raw datasets into actionable data products for our analytics
Responsibilities:
Data Lake Architecture: Design and implement ADLS Gen2 storage structures (Bronze/Silver/Gold layers) ensuring optimal partitioning and security.
Pipeline Development: Build and maintain complex ETL/ELT pipelines using Scala on Azure Databricks or HDInsight.
Performance Tuning: Optimize Spark jobs written in Scala to ensure low latency and costeffective compute usage.
Data Orchestration: Use Azure Data Factory (ADF) to schedule and monitor endtoend workflows.
Streaming Batch: Develop realtime data ingestion pipelines using Azure Event Hubs or Kafka integration.
Goverce Security: Implement rolebased access control (RBAC), encryption, and data masking using Microsoft Purview .
MustHave Skills / Requirements:
7+ years overall backend development experience in production environments
Primary Language: Advanced proficiency in Scala (Functional Programming, Spark APIs).
Azure Big Data Stack: Expert knowledge of Azure Data Lake Storage Gen2 , Databricks, and Data Factory.
Distributed Computing: Deep understanding of Apache Spark (Data Frames, Datasets, Spark SQL, and optimization techniques like Caching and Broadcast joins).
SQL Mastery: Ability to write and optimize complex SQL queries for Synapse or Azure SQL.
DevOps: Experience with Azure DevOps , CI/CD pipelines, and version control (Git).
Data Modeling: Familiarity with Star/Snowflake schemas
No Referrers Available
There are currently no referrers available for this job. You can still apply, will let you know once there is any referrer available.