Job Description
ETL/Data testing
cloud data platforms
and CI/CD integration within enterprise-grade analytics ecosystem
Shift timing will be 1:00 PM to 10:00 PM IST.
Key Responsibilities
Data Pipeline Testing:
Validate end-to-end data ingestion, transformation, and consumption pipeline across bronze, silver, and gold layers.
Perform source-to-target data validation for structured.
Validate business rules and data transformations.
Ensure data consistency, completeness, uniqueness and referential integrity.
Automation Testing:
Design and develop automated test frameworks for data validation and regression testing.
Automate ETL and data quality testing using python, SQL, PySpark, or testing frameworks.
Build reusable validation scripts for schema validation, reconciliation, and data drift detection.
Integrate automated test suites into CI/CD pipelines.
Databricks and Cloud Validation:
Validate Databricks notebooks, workflows, Delta Tables and job orchestration.
Test Delta Lake functionalities include ACID compliance, schema evolution and time travel.
Validate performance and scalability of Spark-based transformations.
Required Skills:
SQL
Python
PySpark
Databricks
Batch Pipelines
Incremental Loads
CDC pipelines
Knowledge of Medallion Architecture concepts
Preferred Qualifications:
6-8 Years of experience in QA/Data Testing.
2+ years of experience in Databricks/Data Lakehouse projects.
Experience working in Agile/Scrum delivery models.
Certification in Databricks, Cloud or Testing tool is an advantage.
Disclaimer : The job description & location mentioned in this description is based on publicly available information or company headquarters. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.