Hadoop Administrator

Bharti Airtel Ltd.

Job Title: Lead - Big Data Architect & Analytics
Location: Gurgaon,
Department: Network

Key Responsibilities:

• Must be capable of handling existing or new Apache HDFS cluster having name node, data node & edge node commissioning & decommissioning.
• Work closely with data architects and analysts to design technical solutions.
• Integrate and ingest data from multiple source systems into big data environments.
• Develop end-to-end data transformations and workflows, ensuring logging and recovery mechanisms.
• Must able to troubleshoot spark job failures.
• Design and implement batch, real-time, and near-real-time data pipelines.
• Optimize Big Data transformations using Apache Spark, Hive, and Tez
• Work with Data Science teams to enhance actionable insights.
• Ensure seamless data integration and transformation across multiple systems.
• 4+ years of mandatory experience with big data.
• 4+ years mandatory experience in Apache Spark.
• Proficiency in Apache Spark, Hive on Tez, and Hadoop ecosystem components.
• Strong coding skills in Python & Pyspark.
• Experience building reusable components or frameworks using Spark

Expertise data ingestion from multiple sources using APIs, HDFS, and NiFi.

• Solid experience working with structured, unstructured, and semi-structured data formats (Text, JSON, Avro, Parquet, ORC, etc.).
• Experience with UNIX Bash scripting and databases like Postgres, MySQL and Oracle.
• Ability to design, develop, and evolve fault-tolerant distributed systems.
• Strong SQL skills, with expertise in Hive, Impala, Mongo and NoSQL databases.
• Hands-on with Git and CI/CD tools
• Experience with streaming data technologies (Kafka, Spark Streaming, Apache Flink, etc.).
• Proficient with HDFS, or similar data lake technologies
• Excellent problem-solving skills — you will be evaluated through coding rounds