CyberSecurity Benchmark Engineer

Role Overview We are building a benchmark to evaluate the cybersecurity capabilities of frontier AI systems. We are looking for cybersecurity professionals who are comfortable working with LLMs to help design and validate challenging security tasks that current AI systems struggle to solve. If you have a solid security background and regularly use AI tools in your work, you're likely a good fit. Key Responsibilities • Create cybersecurity tasks and challenges that are difficult for frontier LLMs and AI agents to solve • Research real-world security issues and translate them into well-defined AI evaluation tasks • Use LLM agents to test your tasks and document where and why they fail • Write clear, concise descriptions of security scenarios suitable as prompts for AI systems • Validate that tasks have a correct, reproducible solution that can be verified • Work with the research team to refine tasks and improve benchmark quality. Required Skills Cybersecurity • 3+ years of experience in any security domain application security, network security, penetration testing, vulnerability research, CTF, or similar • Ability to understand and explain how a security vulnerability or attack works • Comfortable reading code and identifying security issues across common languages Working with AI / LLMs • Regular hands-on experience using LLMs (ChatGPT, Claude, Gemini, etc.) for technical work • Ability to evaluate whether an AI system has solved a problem correctly or not • Basic comfort with LLM APIs or AI-assisted workflows