Experience
7 - 10 yrs
Job Location
Gurugram, India
Vacancy
1
Designation
Senior Big Data Engineer
Job Type
ONSITE
Job Description
Seeking a Big Data Cloudera Engineer with expertise in PySpark, Hadoop, SQL, Teradata, and Python. Responsible for developing and optimizing ETL pipelines, database queries, and automation solutions. Experience with performance tuning, troubleshooting, and workflow scheduling tools like Airflow or Oozie is required. Work closely with cross-functional teams to deliver scalable, high-quality data solutions.
Key Responsibilities - Database SQL Development: Write, optimize, and debug complex SQL queries on Teradata, Hive, and other RDBMS systems. Ensure data consistency, quality, and performance tuning of databases.
- Automation Scripting: Develop reusable Python scripts for automation, data validation, and process scheduling. Work with Airflow, Oozie, or similar workflow schedulers.
- Performance Optimization Troubleshooting: Monitor and tune PySpark jobs and Hadoop cluster performance. Debug and optimize SQL queries, data pipelines, and ETL processes.
- Collaboration Stakeholder Engagement: Work with data analysts, data scientists, and business teams to understand requirements. Provide guidance on data best practices, architecture, and governance. Strong expertise in PySpark, Hadoop ecosystem (HDFS, YARN, Sqoop, Hive, MapReduce).
- Graduate / Postgraduate degree in Computer Science / Engineering from a reputed university
- Strong expertise in PySpark, Hadoop ecosystem (HDFS, YARN, Sqoop, Hive, MapReduce).
- Proficiency in SQL, Teradata, and other relational databases.
- Hands-on experience with Python scripting for ETL, automation, and data processing.
- Experience with big data performance tuning, optimization, and debugging.
- Familiarity with job scheduling tools like Airflow, Oozie, or Control-M.
- Strong knowledge of data warehousing concepts, data modeling, and ETL frameworks.
- Ability to work in Agile/Scrum environments and collaborate with cross-functional teams.
No Referrers Available
There are currently no referrers available for this job. You can still apply, will let you know once there is any referrer available.
