Senior AI scientist

What You Will Work On

• Design and train large scale transformer and hybrid foundation models
• Diagnose and resolve training instabilities at scale
• Navigate scaling tradeoffs across data, compute, and architecture
• Define the technical direction for next generation models

Core Responsibilities

• Apply strong fundamentals in deep learning and representation learning
• Design and modify transformer architectures, including:
• Attention variants
• RoPE
• ALiBi
• Grouped Query Attention (GQA)
• Mixture of Experts (MoE)
• Build models from first principles, not just adapt pre-existing codebases
• Optimization Dynamics & Training Stability
• Own optimizer and scheduler choices, including AdamW
• Lion
• Adafactor
• Learning rate and warmup schedulers
• Understand and debug optimizer instability and gradient pathologies
• Scaling Laws & Compute Tradeoffs
• Apply and validate scaling laws
• Chinchilla style compute vs data tradeoffs
• Loss Functions & Alignment
• Design and experiment with loss functions including Next token prediction

Disclaimer : This job posting has been aggregated from external source. Role details, content, and availability are subject to change. Applicants are advised to confirm the latest information directly on the company website before applying.