Python Pyspark Developer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data processing pipelines using PySpark on AWS EMR.
• Collaborate with cross-functional teams to gather requirements and deliver high-quality solutions that meet business needs.
• Develop scalable and efficient ETL processes to extract insights from various sources of structured and unstructured data.
• Troubleshoot issues related to data quality, performance optimization, and system reliability.

Job Requirements :

• 5-10 years of experience in developing big data applications using Python/PySpark on AWS platform.
• Strong understanding of Django framework for building web applications.
• Experience with designing complex ETL processes for large-scale datasets.
• Proficiency in working with Spark SQL for querying massive datasets.