Experience
6 - 11 yrs
Job Location
Bengaluru, India
Vacancy
1
Designation
Senior Big Data Engineer
Job Type
Not specified
Job Description
Core Responsibilities
- Ability to oversee and guide junior and mid-level engineers, providing technical direction and mentoring to support their growth.
- Delegating tasks effectively, ensuring clarity of expectations and alignment with delivery goals.
- Ownership for the delivery of assigned workstreams, ensuring quality, timeliness, and accountability across the team.
- Build pipelines for processing big amount of medical data
- Participate in analysis, system design, development, deployment and maintenance
- Conduct peer code reviews
- Contribute to the definition and adoption of technical standards
- Create comprehensive automated unit and integration tests
Bachelor s Degree in Information Technology, Software Engineering, Computer Science, Mathematics, or another related field
- 6+ years of the following language: SQL, Python, Scala
- 6+ years of experience in data engineering
- Strong understanding of Databricks platform
- Experience with Big Data Batch processing and Azure Cloud Services
- Practical knowledge of Apache Spark ecosystem, including Delta and Parquet data formats
- Experience with complex ETL design and implementation
- Experience with data analysis, data profiling, data integration, data quality, data warehousing, data reporting
- Experience with CI/CD automation processes and tools such as Jenkins/GitLab
- Familiarity with AI-powered productivity tools such as Copilot or Cursor, and their practical use in day-to-day software development workflows (e.g. code generation, documentation, task automation, or debugging support).
- Experience working in an Agile environment
- Experience with database design and modelling
- Previous experience with healthcare or pharmaceutical data would be an advantage
- Knowledge of algorithms design patterns and how to apply them effectively
- Experience with large data volume (TBs ) processing
Personal skills and behaviors
- Excellent analytical troubleshooting skills
- Strong collaboration, written, and verbal skills
- Familiarity with project management concepts, specifically Agile/Scrum
- Ability to present complex results in a clear language for both technical and non-technical audience
- Experience collaborating with teams from Europe, with an understanding of cross-cultural communication and remote delivery dynamics
- A great team player mindset