Lead Data Engineer

Key Responsibilities:

• Execute a comprehensive data processing monitoring strategy which aligns with the organization's Data Processing Standards and business objectives.
• Provide guidance and mentorship to junior data processing engineers, fostering their growth and development.
• Develop a strong understanding of Dun & Bradstreets inventory data.
• Perform baseline data processing monitoring to proactively identify data issues.
• Employ advanced data analysis and profiling techniques.
• Automate data processing monitoring solutions and internal processes.
• Create or update data models to ensure that data is stored in an organized structure.
• Utilize Power BI and/or Looker to design, create, connect and administer dashboards which derive insights from data processing monitoring results.
• Design and implement a robust data validation and processing framework with automated testing processes.
• Communicate with the globally distributed stakeholders using JIRA and Confluence.
• Capture requirements accurately and seek strong understanding of use cases.
• Recommend improvements to data processing team’s internal processes.
• Generate regular reports on data processing metrics.
• Review data to identify patterns or trends that may indicate errors in processing.
• Develop comprehensive documentation of data processing processes, procedures, and findings, and ensure junior members document their work.
• Comply with and enforce data governance policies and procedures.
• Remain an expert in industry best practices and technologies related to data processing.

Required Traits Include:

• Bachelor's degree in Business Analytics, Computer Science, Information Technology, or a related field.
• 5+ years of experience and demonstrated in-depth knowledge of data analysis, querying languages, data modelling, and the software development life cycle.
• Expertise in SQL (preferably BigQuery).
• Agile mindset and deep understanding of agile project management (Scrum/Kanban).
• Experience in Database design, modelling, and best practices.
• Experience with cloud computing technologies (preferably GCP).
• Experience with Power BI, Looker, or similar data visualisation tool.
• Ability to mentor & provide guidance to less experienced members of the team.
• Analytical, process improvement and problem-solving skills.
• Strong communication and the ability to articulate data issues and solutions.
• Commitment to meet deadlines and uphold the release schedule and guide junior team members’ efforts to ensure the same.
• Experience collaborating across time zones as part of a global team.

Valuable Traits Include:

• Skill in Python and/or Scala for data wrangling and data analysis.
• Experience with DevOps best practices including CI/CD, automation, monitoring, observability, agile project management, version control, and continuous feedback.
• Experience with data observability tools such as Acceldata or Informatica DQ.
• Experience with XML and JSON data structures.
• Understanding of ETL processes and their impact on data processing.
• Knowledge of Machine Learning, specifically anomaly detection.