Senior Software Engineer- Data

Key Responsibilities

• Build scalable batch and streaming pipelines using Python, PySpark, and SQL.

• Develop and maintain ETL/ELT workflows and data ingestion frameworks.

• Design dimensional data models (fact/dimension, star schema, 3NF).

• Write and optimize complex SQL transformations for large datasets.

• Ensure data quality, validation, lineage, and governance best practices.

• Tune Spark jobs and SQL queries for performance and cost.

• Work with AWS services (S3, Glue, EMR, Lambda, Step Functions).

• Implement CI/CD, testing, and production deployment of data workflows.

• Monitor, log, and troubleshoot data pipelines in production.

• Participate in architecture discussions and drive technical decisions.

• Collaborate with analytics, product, and data science teams.

• Mentor junior engineers and promote best practices.

Required Skills Qualifications

• 5+ years of experience in data engineering or related roles.

• Strong Python development experience.

• Advanced SQL (analytical queries, performance tuning).

• Expertise in PySpark (DataFrame API, RDD concepts, optimization).

• Solid understanding of data modeling for OLTP/OLAP.

• Proven ETL/ELT pipeline design experience.

• Cloud experience (AWS preferred; similar GCP/Azure OK).

• Knowledge of data warehousing concepts and distributed systems.

• Git and CI/CD experience.

• Strong debugging, optimization, and problem-solving skills.

• Ability to work with large-scale datasets.

Preferred Skills

• Airflow/Dagster/Prefect

• Kafka/Kinesis or streaming systems

• Docker/Kubernetes

• IaC: CDK, Terraform, CloudFormation

• Delta Lake, Iceberg, or Hudi

• ML feature pipelines

• Monitoring tools: CloudWatch, Datadog, Prometheus, ELK

• System design and architectural experience

Soft Skills

• Clear communication and documentation

• Ownership mindset

• Mentorship capability

• Collaboration with cross-functional teams

• Detail orientation and structured thinking

Why Work With Us

• High-impact ownership of modern data platforms

• Exposure to advanced cloud, Spark, and lakehouse technologies

• Collaborative culture focused on learning, innovation, and engineering excellence

• Opportunity to shape data architecture and strategy

Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.