Azure Data Engineer

Mode of Work: (5 days work from office)Interview :1st virtual & 2nd F2FMust skills: ADF, ADB, Python & SQL (Complex Queries)Job Role:

• Develop long-term vision for a highly scalable data platform, data management and Data Opspractices.
• Design and architect data flows, data management in Hadoop or Cloud environment which are
• scalable, repeatable and eliminate time consuming steps
• Promote Data Ops approach to automate the provision of data, testing and monitoring and to
• shorten development cycles and increase deployment frequency
• Establish development and data governance processes to build mature data pipelines, CI/CD,test coverages, etc.
• Evaluate, provide insights and recommendations on tools and technology strategy for analytics
• data platforms and applications in conjunction with Enterprise Architecture team
• Ability to lead data engineering workstreams with a product mindset

Who are we looking for?

• Bachelors or master’s degree in computer science, Information Systems or equivalent field.
• At least 5+ years of experience in building data flows and data management on modern bigdata tech stack
• Data Strategy: Understands, articulates, and applies principles of the defined strategy to routinebusiness problems that involve a single function.
• Data Transformation and Integration: Extracts data from identified databases. Creates datapipelines and transform data to a structure that is relevant to the problem by selectingappropriate techniques. Develops knowledge of current analytics trends.
• Data Source Identification: Supports the understanding of the priority order of requirements and service level agreements. Helps identify the most suitable source for data that is fit for purpose.
• Demonstrates expertise in writing complex, highly optimized queries across large data sets
• Strong experience in using ETL framework (eg. Airflow, Oozie, Jenkins etc.) to build and deployproduction-quality ETL pipelines.
• Experience in ingesting and transforming structured and unstructured data from internal andthird-party sources into dimensional models.
• Knowledge of data structures and distributed computing. Should be comfortable inmanipulation and analysis of high-volume data from variety of internal and third-party sources.
• Experience in one or more programming languages like Python or PySpark and moderate
• knowledge on unix scripting