Job Description:

Implement and enhance end to end data pipelines (batch and/or streaming) to ingest data from diverse source systems into the enterprise data platform, following agreed architecture and patterns.

Engineer robust ETL/ELT workflows to transform, cleanse, and standardize data, ensuring conformance with canonical data models and business rules.

Build and optimize data layers (raw, curated, semantic) that enable self-service analytics, BI, and data science use cases, with particular focus on performance, scalability, and cost efficiency.

Industrialize data solutions by implementing re usable frameworks, templates, and components for ingestion, quality checks, logging, and monitoring.

Apply best practices for code management, CI/CD, environment promotion, and automated testing for data pipelines and related assets.

Implement data quality and data validation checks, reconcile data across systems, and resolve data issues in collaboration with business and platform teams.

Contribute to data modeling activities (conceptual, logical, physical) and translate models into physical structures in the target data platform/warehouse.

Tune queries, jobs, and storage layouts to meet SLAs for latency, throughput, and concurrency, leveraging partitioning, indexing, caching, and other optimization techniques supported by the platform.

Implement and adhere to security, privacy, and governance standards, including role based access controls, data masking, and lineage/metadata capture.

Produce technical documentation for pipelines, data sets, job flows, and operational procedures, and hand over solutions into BAU/support as they are industrialized.

Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.

Sr. Data Engineer (Mandatory Python, SQL, BI tools)

Job Description

No Referrers Available