GCP Data Engineer

Job Description

We are looking for a Data Engineer with 4+ years of experience in designing and developing scalable data pipelines on Google Cloud Platform (GCP). The ideal candidate should have hands-on experience with BigQuery, Dataflow, and workflow orchestration using DAGs.

Required Skills

• 3+ years of experience in Data Engineering.
• Strong hands-on experience with Google BigQuery.
• Good understanding of BigQuery Partitioning and Clustering for query and cost optimization.
• Experience in designing and developing ETL/ELT pipelines using Google Cloud Dataflow.
• Hands-on experience with Apache Airflow/Cloud Composer and creating, managing, and troubleshooting DAGs.
• Strong SQL skills with experience in performance tuning and query optimization.
• Good understanding of data modeling and large-scale data processing.
• Experience with Python for developing data pipelines is preferred.
• Familiarity with GCP services such as Cloud Storage, Pub/Sub, IAM, and Monitoring is an added advantage.

Roles & Responsibilities

• Design, develop, and maintain scalable data pipelines using Dataflow.
• Build and optimize BigQuery datasets, tables, partitioning, and clustering strategies.
• Develop and manage Airflow/Cloud Composer DAGs for workflow orchestration.
• Optimize query performance and reduce BigQuery processing costs.
• Monitor, troubleshoot, and improve data pipeline reliability and performance.
• Collaborate with cross-functional teams to understand business requirements and deliver data solutions.
• Ensure data quality, security, and governance best practices are followed.

Preferred Skills

• Experience with CI/CD for data pipelines.
• Knowledge of Git and Agile development methodologies.
• Exposure to streaming data pipelines is a plus.