Job Description
-
Building RAG Pipelines: Developing and optimizing ETL/ELT processes to prepare unstructured data (documentation, logs, knowledge bases) for LLM usage (chunking, embedding generation, ingestion into Vector DBs).
-
Data Architecture: Designing scalable data schemas in Google Cloud Platform (BigQuery, Cloud Storage) to support both transactional and analytical queries from AI agents.
-
Performance Optimization: Tuning vector search and retriever optimization to reduce latency during response generation.
Tech Stack
-
Cloud: Google Cloud Platform (GCP) deep knowledge.
-
Data Processing: Python, VertexAI
-
Orchestration: Cloud Composer (Airflow).
-
Infrastructure: Terraform, Kubernetes (GKE), Docker.
_x000D_ Essential functions_x000D_
_x000D_-
5+ years of experience in Data Engineering, with proven experience designing complex architectures from scratch.
-
GCP Expertise: Deep knowledge of Google Cloud services (BigQuery, Dataflow, Pub/Sub, Vertex AI).
-
Pipeline Development: Expertise in building both batch and streaming (real-time) data processing pipelines.
-
Vector Search & RAG: Understanding of RAG (Retrieval-Augmented Generation) principles and experience setting up pipelines for data vectorization.
-
Python Mastery: Excellent command of Python for writing complex data transformation logic.
-
System Design: Ability to design fault-tolerant and secure data systems (Security, GDPR/CCPA compliance).
_x000D_ Qualifications_x000D_
_x000D_-
5+ years of experience in Data Engineering, with proven experience designing complex architectures from scratch.
-
GCP Expertise: Deep knowledge of Google Cloud services (BigQuery, Dataflow, Pub/Sub, Vertex AI).
_x000D_ Would be a plus_x000D_
_x000D_-
Experience with AgentOps and understanding the lifecycle of LLM applications.
-
Experience in the E-commerce domain or with MACH architecture.
-
Experience mentoring junior engineers and conducting technical audits of solutions.
-
Excellent communication skills (working with US stakeholders) and the ability to translate business requirements into technical data tasks.
_x000D_ We offer_x000D_
_x000D_-
Opportunity to work on bleeding-edge projects
-
Work with a highly motivated and dedicated team
-
Competitive salary
-
Flexible schedule
-
Benefits package - medical insurance, sports
-
Corporate social events
-
Professional development opportunities
-
Well-equipped office
No Referrers Available
There are currently no referrers available for this job. You can still apply, will let you know once there is any referrer available.