Experience
5 - 9 yrs
Salary (CTC)
₹11.4L - ₹17.1L
Job Location
Hyderabad, India
Vacancy
1
Designation
Pyspark Developer
Job Type
Not specified
Job Description
Educational Requirements
- Bachelor of Engineering
- BTech
- BCA
- BSc
- MTech
- MSc
- MCA
Data Analytics Unit
Responsibilities - We are looking for an experienced Python PySpark Developer to design, develop, and optimize large-scale data processing systems.
- The ideal candidate will work on big data platforms, build scalable ETL pipelines, and process high-volume datasets using Spark and Python.
- Develop and maintain data pipelines using Python and PySpark.
- Process and transform large datasets in distributed environments.
- Build scalable ETL/ELT workflows.
- Work with Apache Spark (PySpark) for batch and real-time processing.
- Optimize Spark jobs for performance and efficiency.
- Handle structured and unstructured data.
- Ingest data from multiple sources: Databases (SQL/NoSQL), APIs, Files (CSV, JSON, Parquet).
- Integrate with data platforms like Hadoop (HDFS) and Cloud (AWS, Azure, GCP).
- Tune Spark jobs (partitioning, caching, parallelism).
- Optimize SQL queries and transformations.
- Improve data processing efficiency and cost.
- Collaborate with data engineers, data scientists, and analysts.
- Translate business requirements into technical solutions.
- Participate in code reviews and agile development practices.
- Debug and resolve issues in data pipelines.
- Monitor job execution and data quality.
- Ensure reliability and availability of data workflows.
- Primary skills: Python, PySpark
- OpenSystem
- Python
- PySpark
- Big Data
- Data Processing
No Referrers Available
There are currently no referrers available for this job. You can still apply, will let you know once there is any referrer available.