Data Engineer II

Data Engineer II Overview
Who is Mastercard
Mastercard is a global technology company in the payments industry. Our mission is to connect and power an inclusive, digital economy that benefits everyone, everywhere by making transactions safe, simple, smart, and accessible. Using secure data and networks, partnerships, and passion, our innovations help individuals, financial institutions, governments, and businesses realize their greatest potential.
The Mastercard Services organization is a key differentiator, delivering cutting-edge solutions used by some of the worlds largest organizations to make critical business decisions. Focused on innovation and scale, Services provides data-driven capabilities across consulting, analytics, experimentation, and risk management.
Team Overview
As part of Mastercards Data Platform Orchestration team, you will contribute to building next-generation data platforms that are critical to our global data ecosystem.
Our team develops and operates scalable platform capabilities, including:
Cloud-native infrastructure and application provisioning
Standardized CI/CD pipelines and engineering tooling
Reusable frameworks and data platform components
These platforms enable teams to build, deploy, and operate data-driven solutions efficiently and securely at scale.
Role Overview
Data Platform Orchestration is seeking a Data Engineer II to design and build next-generation, cloud-native data platforms supporting Mastercards global data ecosystem.
In this role, you will lead the development of scalable batch and real-time data pipelines, enabling efficient data processing across Data Lakes and Data Warehouses. You will work at the intersection of data engineering, cloud platforms, and distributed systems, contributing to high-impact initiatives and driving engineering excellence.
This role is ideal for someone who thrives in a fast-paced, collaborative environment, enjoys solving complex data challenges, and is passionate about building resilient, high-performance systems at scale.
Role Overview
As a Data Engineer II, you will design and develop scalable batch and near real-time data pipelines that power Mastercards analytics and operational systems.
You will work across data engineering, cloud platforms, and distributed systems, building robust data solutions on top of Data Lakes and Data Warehouses. This role is highly hands-on and requires strong engineering fundamentals, with opportunities to influence design decisions and mentor junior engineers.
You will contribute to building cloud-native data platforms, enabling reliable, high-performance data processing at scale.
________________________________________
Key Responsibilities
Design and build scalable data pipelines and microservices using Java (Spring Boot), Spark, and cloud-native technologies
Develop high-throughput, low-latency data processing systems for real-time and batch workloads
Design and develop batch and near real-time data pipelines using Spark, Kafka, and Java-based frameworks
Build and maintain ETL/ELT pipelines for structured and unstructured data
Develop data processing solutions across Data Lakes and Data Warehouse environments
Contribute to stream processing use cases (Kafka, and optionally Flink or Spark Structured Streaming)
Ensure data quality, validation, and reliability across pipelines
Optimize data processing workloads for performance and scalability
Develop and deploy data pipelines on AWS, Azure, or GCP
Leverage cloud-native services such as S3/ADLS/GCS, EMR/Databricks, BigQuery/Redshift/Snowflake
Contribute to Infrastructure as Code (Terraform, CloudFormation, or equivalent)
Build solutions with high availability, fault tolerance, and scalability
Follow best practices for secure data processing and cloud resource utilization
Follow best practices in coding, testing, and CI/CD pipelines
Contribute to automation, monitoring, and observability of data pipelines
Develop reusable components to improve engineering efficiency and consistency
Participate in code reviews and design discussions
Collaborate with architects, product teams, and cross-functional stakeholders
Support production deployments and troubleshoot issues in distributed systems
Contribute to a culture of continuous improvement and technical excellence
Mentor junior engineers and share knowledge within the team
Required Skills Qualifications
Strong proficiency in Java (JDK 8+), OOP/OOAD principles, Python is a plus. Experience with Spring Boot, REST APIs, Spring Security, Hibernate
Hands-on experience with distributed systems, multithreading, and messaging systems (Kafka preferred)
Hands-on experience with Apache Spark (Core, SQL, or Structured Streaming). Experience with Kafka or similar messaging/streaming platforms
Understanding of ETL/ELT pipelines, batch and streaming architectures
Familiarity with data formats (Parquet, Avro, ORC)
Basic understanding of data modeling (star/snowflake schemas). Understanding of distributed systems and multithreading concepts
Hands-on experience with at least one cloud platform: AWS, Azure, or GCP
Experience using cloud storage and compute services (e.g., S3, ADLS, Databricks, EMR)
Familiarity with containerization (Docker) and basic Kubernetes concepts
Exposure to Infrastructure as Code tools is a plus
Strong SQL skills and experience working with relational and analytical databases
Exposure to Data Lakes and Data Warehousing platforms
Familiarity with workflow orchestration tools (Airflow or similar)
Experience with CI/CD tools (Jenkins, GitHub Actions, or similar)
Good understanding of unit testing (JUnit or equivalent)
Familiarity with monitoring tools (Splunk, Prometheus, Grafana, etc.)
Awareness of secure development practices
Experience with real-time processing frameworks -Spark Streaming, good to have knowledge on Apache Flink
Exposure to performance testing tools (JMeter, Gatling)
Familiarity with DevSecOps or SRE concepts
Experience improving automation and developer productivity
Strong problem-solving and analytical skills. Ownership mindset with ability to deliver independently
Good communication and collaboration skills. Passion for learning new technologies and improving engineering practices
Ability to work effectively in a fast-paced, global environment
Education
Bachelors degree in Computer Science, Information Technology, Engineering, or a related field

• Abide by Mastercards security policies and practices;
• Ensure the confidentiality and integrity of the information being accessed;
• Report any suspected information security violation or breach, and
• Complete all periodic mandatory security trainings in accordance with Mastercards guidelines.