Sr. Cloud Engineer

What you'll do:

• Experience in building and managing end-to-end analytics workloads using Microsoft Fabric, OneLake, Lakehouses, and Warehouses. Implement Direct Lake Connectivity for high-performance Power BI reporting.
• Experience design and develop scalable data processing engines using Azure Databricks. Leverage PySpark for complex transformations, streaming, and large-scale data wrangling.
• Experience in architecting multi-stage data pipelines using Azure Data Factory (ADF) and Synapse Pipelines. Focus on metadata-driven frameworks and dynamic orchestration to minimize hard-coding.
• Advanced Scripting Transformation:
• SQL Scripts: Write high-performance T-SQL and Spark SQL for complex business logic, data validation, and performance tuning in Synapse Dedicated/Serverless pools.
• Python: Develop custom Python modules for API integrations, automation scripts, and advanced data manipulation beyond standard ETL tools.
• Experience in Implementing the Medallion Architecture (Bronze/Silver/Gold) using Delta Lake formats to ensure ACID transactions, data lineage, and schema evolution.

About You:

• Primary Toolset: Extensive hands-on experience with Microsoft Fabric, Azure Databricks, Azure Synapse Analytics, and Azure Data Factory.
• Expert Scripting: Python: Deep proficiency in Python and PySpark (Spark Core, Spark SQL, Structured Streaming).
• Advanced SQL: Expert-level SQL scripting (Window functions, CTEs, stored procedures, and query optimization).
• Delta Lake OneLake: Strong understanding of Delta Lake table formats and the unified storage principles of OneLake.
• Orchestration Patterns: Proven ability to build reusable, parameter-driven pipelines in ADF that handle diverse data sources (REST APIs, Parquet, Delta).
• DevOps CI/CD: Good to have experience with Git-based version control (Azure DevOps/GitHub) and deploying data infrastructure as code (Bicep/Terraform)

Disclaimer: This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.