SDET API Python Automation

Job Summary

We are looking for a highly skilled Data SDET (Software Development Engineer in Test) with strong expertise in data engineering validation, automation, and cloud-based testing frameworks. The ideal candidate will have experience working with large-scale data platforms, ETL pipelines, and modern cloud-native architectures while ensuring data quality, reliability, and performance.

Key Responsibilities

• Design, develop, and execute automated test strategies for data pipelines and ETL/ELT workflows
• Validate data integrity, transformation logic, and end-to-end data flow across systems
• Work closely with data engineering teams to test batch and real-time data pipelines
• Perform CDC (Change Data Capture) validation and data reconciliation across heterogeneous systems
• Build scalable test automation frameworks using Python and Pytest
• Write and optimize complex SQL queries for validation and analysis
• Test distributed systems leveraging Kafka, Kinesis, and streaming platforms
• Validate data pipelines on AWS serverless services (Lambda, Glue, Step Functions, etc.)
• Ensure data quality using monitoring and observability tools like Grafana
• Participate in CI/CD pipelines integration for automated testing (GitLab/Jenkins)
• Collaborate with cross-functional teams to define test strategies, test plans, and quality metrics
• Identify performance bottlenecks and ensure high data throughput and reliability

Mandatory Skills

• Strong programming skills in Python
• Expertise in Advanced SQL
• Hands-on experience with AWS ecosystem
• Experience with Kafka / streaming platforms
• Strong knowledge of Unix/Linux environments
• Expertise in test automation frameworks (Pytest or similar)
• Experience with AWS Serverless services

Secondary Skills (Good to Have)

Data Engineering & ETL

• Ab Initio / ETL / ELT testing
• AWS Glue
• CDC validation techniques

Databases & Data Platforms

• Amazon Redshift, Oracle, SQL Server, Teradata
• AWS Athena, DynamoDB

Big Data & Workflow Tools

• Databricks, EMR
• AWS Step Functions
• Understanding of data pipelines and orchestration

DevOps & Automation

• Experience with CI/CD pipelines
• Tools: GitLab, Jenkins

Monitoring & Quality

• Grafana / monitoring frameworks
• Data Quality Engineering practices

Test Management

• Experience with Jira, X-Ray or similar tools