GCP Data Engineer

We are looking for an experienced GCP Data Engineer (Technical Lead) to design, build, and manage scalable data pipelines on Google Cloud Platform. The ideal candidate will have strong expertise in cloud-native data services, big data technologies, and DevOps practices within an agile environment.

Key Responsibilities:

• Design and develop scalable data pipelines using GCP services
• Work within an agile, DevOps-driven environment
• Build and manage data processing workflows using Dataflow, Pub/Sub, and BigQuery
• Develop solutions for real-time and batch data processing
• Integrate data from multiple sources using tools like Kafka/NiFi
• Optimize data storage and querying performance in BigQuery
• Manage orchestration workflows using Cloud Composer (Airflow)
• Implement infrastructure automation using Terraform and scripting (Shell/Python)
• Ensure data quality, reliability, and performance across pipelines
• Collaborate with cross-functional teams for data architecture and delivery

Required Skills & Expertise:

• Strong hands-on experience with Google Cloud Platform (GCP)
• Expertise in:
• Python
• Dataflow, Pub/Sub, BigQuery
• Experience with GCP components:
• GCS (Google Cloud Storage)
• BigQuery
• Cloud Composer (Airflow)
• Cloud SQL
• Pub/Sub or Kafka
• Dataflow
• Google Cloud SDK
• Knowledge of Hadoop ecosystem and tools like NiFi/Kafka
• Experience with Terraform and Shell scripting
• Strong understanding of RDBMS systems
• Experience working in Agile/DevOps teams

Preferred Qualifications:

• GCP Data Engineer Certification
• Experience in large-scale data platforms and streaming architectures
• Exposure to data governance and security best practices