Educational Requirements

Bachelor of Engineering,BSc,BTech,BCA,MTech,MCA,MSc

Service Line

Data Analytics Unit

Responsibilities

Big Data Spark Development
Develop and maintain data processing pipelines using Apache Spark (PySpark Scala)
Work with Spark DataFrames, RDDs, and Spark SQL
Implement transformations, joins, aggregations, and optimizations
Tune Spark jobs for performance, scalability, and reliability
Python Scala Programming
Write clean, efficient, and scalable code in Python and Scala
Develop modular and reusable components
Integrate data pipelines with various applications and APIs
ETL Data Engineering
Design and build ETL workflows for structured and unstructured data
Extract data from multiple sources (databases, APIs, flat
Perform data cleansing, transformation, and validation
Ensure data accuracy, consistency, and completeness
Data Platforms Integration
Work with Hadoop ecosystem (HDFS, Hive, Spark)
Handle large datasets in data lakes and warehouses
Process data in formats like Parquet, ORC, JSON, CSV
Collaboration Support
Work with data engineers, analysts, and business stakeholders
Troubleshoot pipeline issues and provide production support
Participate in Agile/Scrum processes
Maintain technical documentation

Additional Responsibilities:

Technical and Professional Requirements:

Primary skills:Technology->Big Data - Data Processing->Spark,Technology->Java->Apache,Technology->Machine Learning->Python

Preferred Skills:

Python+Spark Scala

Job Description