Lead Data Engineer

Job Summary

This mail is with reference to your profile on Job board. We have wonderful opportunity for Lead Data Engineer professionals.

Experience

Experience: 10 yrs - 20 yrs

Location

Bangalore/Chennai/Hyderabad, India

Key Qualifications

• Data Architecture and Design: Collaborate with data architects and stakeholders to design and implement scalable data architectures on the Azure platform. Define data storage, data ingestion, data transformation, and data processing strategies that align with business requirements and best practices.
• Data Pipeline Development: Develop robust and scalable data pipelines for ingesting, processing, and transforming data from various sources using Azure services such as Azure Data Factory, Azure Databricks, or Azure Logic Apps. Ensure data pipelines are reliable, performant, and scalable to handle large volumes of data.
• Data Security and Governance: Implement data security and privacy measures to ensure compliance with regulations and industry best practices. Configure access controls, encryption, and auditing mechanisms for data protection. Establish data governance practices and enforce data quality standards.
• Performance Optimization: Optimize data processing and query performance by fine-tuning Azure services, utilizing appropriate data partitioning and indexing techniques, and implementing caching mechanisms. Monitor and troubleshoot performance issues to ensure optimal system performance.
• Collaboration and Leadership: Collaborate with cross-functional teams, including data scientists, business analysts, and software engineers, to understand data requirements and deliver effective data solutions. Provide technical guidance and mentorship to junior data engineers. Lead and contribute to data engineering projects and initiatives.
• Continuous Learning and Innovation: Stay updated with the latest Azure technologies and data engineering trends. Evaluate and recommend new Azure services, tools, or methodologies that can enhance data engineering capabilities and drive innovation.

Role Details

Role: Data Engineer

Industry Type: IT Services Consulting

Department: Engineering Software QA

Employment Type: Full Time, Permanent

Role Category: Software Development

Education

UG: Any Graduate

PG: Any Postgraduate

Experience

Key Skills

• Pyspark
• Big Data
• Azure Databricks
• Ci/Cd
• MS sql server
• python
• Azure Data Factory
• Data Modeling
• SQL Scripting

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.