Experience
Up to 2 yrs
Job Location
India
Vacancy
1
Designation
Data Engineer
Job Type
Not specified
Job Description
As a Data Engineer on the data foundation team, well rely on your expertise across multiple disciplines to develop, deploy and support data systems, data pipelines, data lakes, and lakehouses. Your ability to automate, operate, performance tune, and scale the data platform will be key to your success.
You will:
- Collaborate with stakeholders to make effective use of core data assets
- Leverage a combination of tools, engines, libraries, and code to build data integrations
- Build integrations and pipelines loading both streaming and batched data
- Development of data platform and data pipeline operational and administrative tools in Python
- Load data to support defined data patterns and use cases
- Build integrations and pipeline unit, integration, quality, and performance tests
- Automate continuous deployment and testing of integrations and data pipelines
- Compose data engineering infrastructure as code and build automated infrastructure deployment workflows
- Monitor, operate, and optimize tools, systems, integrations, and pipelines
- Participate in peer code reviews, code approvals, and pull requests
- Help ensure appropriate data privacy and security
- Identify, recommend, and implement opportunities for improvement in efficiency, resilience, scale, security, and performance
What You Need to Get the Job Done (if you don't have all, apply anyway!)
- Understanding of data lake, lakehouse, and data warehouse systems, and related technologies
- Knowledge and understanding of data formats, data patterns, models, and methodologies
- Experience developing data integrations, tools, and pipelines in Python
- Experience working with APIs
- Experience working with streaming technologies such as Kafka
- Ability to work with cloud infrastructure including resource scaling, S3, RDS, IAM, security groups, AMIs, cloudwatch, cloudtrail, and secrets manager
- Experience with expressing infrastructure as code leveraging tools like Terraform
- Experience with GitFlow and GitHub automation
- Git-based team coding workflows
No Referrers Available
There are currently no referrers available for this job. You can still apply, will let you know once there is any referrer available.