AI Analyst

Job Description: AI Analyst

AI Requirements Gathering & Analysis

• Partner with business stakeholders across merchandising, supply chain, marketing, store operations, and e-commerce to identify AI opportunities and gather detailed requirements
• Conduct workshops, interviews, and collaborative sessions to understand business pain points, processes, and objectives that can be addressed through AI
• Translate complex business needs into clear, actionable AI/ML requirements and user stories for data scientists, AI-ML engineers
• Define success metrics, acceptance criteria, and KPIs for AI projects to ensure alignment with business goals
• Create detailed business requirement documents (BRDs), functional specifications, and process flow diagrams
• Prioritize AI use cases based on business impact, feasibility, and strategic alignment
• Serve as the liaison between business users and technical teams, ensuring clear communication and mutual understanding.

Data Analysis & Insights

• Perform exploratory data analysis (EDA) to understand data patterns, identify anomalies, and assess data quality for AI model development
• Analyze large datasets from multiple sources including POS systems, e-commerce platforms, inventory systems, and customer databases
• Create data visualizations and dashboards to communicate insights to stakeholders and support data-driven decision-making
• Conduct statistical analysis to validate hypotheses and identify trends that inform AI model design
• Profile and assess data sources to determine their suitability for AI/ML applications
• Identify data gaps, quality issues, and opportunities for data enrichment to improve model performance
• Work with data engineers to define data requirements and ensure proper data collection and storage
• Generate regular reports on AI system performance, business impact, and ROI metrics

AI Testing & Quality Assurance

• Design and execute comprehensive test plans for AI/ML models including functional, integration, performance, and user acceptance testing (UAT)
• Validate AI model outputs against business requirements and acceptance criteria to ensure accuracy and reliability
• Perform model performance testing using appropriate metrics (accuracy, precision, recall, F1 score, AUC-ROC, RMSE, etc.)
• Conduct bias detection and fairness testing to ensure AI models don't discriminate or produce unintended outcomes
• Test AI systems across different scenarios, edge cases, and data distributions to identify potential failures
• Document test cases, test results, and defects in a clear, organized manner using Jira.
• Work closely with data scientists and AI/ML engineers to reproduce, troubleshoot, and resolve model issues
• Validate data quality, feature engineering, and model training processes to ensure robust AI solutions
• Conduct A/B testing and champion/challenger tests to compare model performance and validate improvements
• Monitor deployed AI models for drift, degradation, and anomalies, raising alerts when performance deviates from expectations

Required Qualifications

• Bachelor's degree in Business Analytics, Data Science, Computer Science, Information Systems, Statistics, or related field
• 5+ years of experience as a Business Analyst, Data Analyst, QA Analyst, or similar role, preferably with exposure to AI/ML projects
• Preferred Retail experience.
• Strong analytical skills with experience in data analysis, statistical analysis, and data visualization
• Proficiency in SQL for querying and analyzing data from multiple data sources. Preferable experience with Databricks.
• Experience with data visualization tools (Tableau, Power BI, Looker, or similar)
• Hands-on experience with Python or R for data analysis and scripting
• Knowledge of AI/ML fundamentals including supervised/unsupervised learning, classification, regression, and model evaluation metrics
• Experience with testing methodologies and creating test plans, test cases, and test scripts
• Proficiency with collaboration and documentation tools (Confluence, Jira, Microsoft Office, Google Workspace)