Role Purpose

The purpose of this role is to design, test and maintain software programs for operating systems or applications which needs to be deployed at a client end and ensure its meet 100% quality assurance parameters

- Strong hands-on experience in PySpark and Scala development.
- Solid understanding of Apache Spark architecture, execution flow, and cluster Manager
- Experience with Hadoop ecosystem: HDFS, Hive, Spark SQL.
- Strong SQL skills for data analysis and transformation.
- Experience in handling large volumes of data (structured/semi-structured).
Good to Have
- Experience with cloud platforms like AWS or Azure (S3, EMR, Glue, Databricks, Data Factory).
- Knowledge of workflow schedulers such as Airflow or Oozie.
- Exposure to Kafka / streaming frameworks.
- Experience with CI/CD, Git, and DevOps practices.

2. Perform coding and ensure optimal software/ module development

Determine operational feasibility by evaluating analysis, problem definition, requirements, software development and proposed software
Develop and automate processes for software validation by setting up and designing test cases/scenarios/usage cases, and executing these cases
Modifying software to fix errors, adapt it to new hardware, improve its performance, or upgrade interfaces.
Analyzing information to recommend and plan the installation of new systems or modifications of an existing system
Ensuring that code is error free or has no bugs and test failure
Preparing reports on programming project specifications, activities and status
Ensure all the codes are raised as per the norm defined for project / program / account with clear description and replication patterns
Compile timely, comprehensive and accurate documentation and reports as requested
Coordinating with the team on daily project status and progress and documenting it
Providing feedback on usability and serviceability, trace the result to quality risk and report it to concerned stakeholders

3. Status Reporting and Customer Focus on an ongoing basis with respect to project and its execution

Capturing all the requirements and clarifications from the client for better quality work
Taking feedback on the regular basis to ensure smooth and on time delivery
Participating in continuing education and training to remain current on best practices, learn new programming languages, and better assist other team members.
Consulting with engineering staff to evaluate software-hardware interfaces and develop specifications and performance requirements
Document and demonstrate solutions by developing documentation, flowcharts, layouts, diagrams, charts, code comments and clear code
Documenting very necessary details and reports in a formal way for proper understanding of software from client proposal to implementation
Ensure good quality of interaction with customer w.r.t. e-mail content, fault report tracking, voice calls, business etiquette etc
Timely Response to customer requests and no instances of complaints either internally or externally

Mandatory Skills: Data Integration Pyspark .

Experience: 5-8 Years .

Pyspark Lead Developer

Job Description

No Referrers Available