Data Engineer

Role & responsibilities

• Data Pipelines & ETL/ELT  
• Develop, schedule, and monitor ELT/ETL pipelines in Python to ingest data from APIs, files, and databases into Snowflake. 
• Implement transformations (SQL & Python) following modular, reusable patterns. 
• Leverage AI-assisted tooling (e.g., GitHub Copilot) to accelerate development, improve readability, and reduce boilerplatewhile ensuring correctness through reviews and testing 
• Snowflake Engineering  
• Build tables, views, stages, and secure data shares. 
• Tune queries and warehouse configs; manage roles, RBAC, and resource usage. 
• Use AI assistance to analyze query performance, refactor SQL, and identify optimization opportunities 
• Cloud & Orchestration  
• Use cloud services (AWS/GCP/Azure) for storage, compute, and event triggers (e.g., S3/Blob/GCS; Lambda/Functions). 
• Orchestrate jobs (Airflow/Cloud Composer/Prefect/Azure Data Factory) with proper alerting & SLAs. 
• Apply AI tools to speed up pipeline scaffolding, DAG generation, and configuration validation 
• DevSecOps & Reliability  
• Apply CI/CD for data code (linting, unit/data tests, IaC review gates). 
• Embed security scanning (dependencies, IaC policies), secrets management, and leastprivilege IAM. 
• Contribute to observability (logging, metrics, lineage, data quality checks). 
• Use AI responsibly to assist with test generation, documentation, and failure analysis 
• Collaboration  
• Work with Analytics/ML teams to productize models and enable selfservice. 
• Write clear documentation (readme, runbooks, data dictionaries). 
• Actively learn and upskill in AIdriven engineering practices, sharing learnings with the team 

Preferred candidate profile

• 3 - 6 years of experience in data engineering, analytics engineering, or backend engineering (internships/co-ops included). 
• Programming: Python (pandas, SQLAlchemy/DB APIs, typing, testing with pytest). 
• SQL & Warehousing: Strong SQL; experience with Snowflake (warehouses, stages, tasks, Snowpipe, RBAC). 
• Cloud Basics: Familiarity with one major cloud (AWS/Azure/GCP) and storage + compute + IAM fundamentals. 
• DevSecOps Mindset: Git, CI/CD (GitHub Actions/Azure DevOps/GitLab CI), code reviews, secret handling, dependency scanning. 
• Data Quality: Exposure to unit tests, data validation (e.g., Great Expectations/dbt tests), and monitoring principles. 
• AIAware Engineering:  
• Experience or willingness to adopt GitHub Copilot or similar AI tools 
• Ability to critically evaluate AIgenerated code and ensure correctness, security, and performance 
• Strong problem-solving, curiosity, and ability to learn quickly. 

Additional information

Nice-to-Have 

• Orchestration: Airflow/Prefect/ADF/Composer. 
• IaC: Terraform/Azure Bicep/CloudFormation; policy-as-code (e.g., OPA/Conftest). 
• Streaming: Kafka/Kinesis/PubSub; Change Data Capture. 
• dbt (models, tests, exposures) and semantic layer concepts. 
• Basic SCD/CDC design patterns; Dimensional modeling (Kimball). 
• Security: OWASP, least privilege, key rotation, secret stores (AWS Secrets Manager/Azure Key Vault). 
• Observability: OpenLineage, DataDog/CloudWatch/Log Analytics; SLA/SLO thinking. 
• Exposure to MLOps (feature stores, model serving) is a plus