Senior Product Manager, Multilingual AI and Evals
Own quality assurance and evaluation infrastructure for multilingual AI agents, including translation quality gates, localization testing, datasets, annotation tools, evaluation metrics, dashboards, and feedback loops.
Responsibilities
- Enhance the evaluation agent that determines whether translations are ready for publication
- Balance automation coverage and risk across content tiers
- Diagnose systematic quality-gate failures and drive fixes into the agent
- Build an AI-driven tool that detects localization defects
- Ship an MVP that scans mobile and web applications for localization issues
- Integrate localization tests into internal product testing infrastructure
- Build golden datasets, annotation tools, automated evaluations, metrics dashboards, and feedback loops
- Integrate standardized real-time annotation workflows for linguists
Requirements
- Bachelor's degree or higher
- 3+ years in product management, with hands-on AI agent experience potentially substituting for part of the requirement
- Experience building content platforms, testing platforms, or SaaS products
- Hands-on experience with AI agents, agent harnesses, and evaluations
- Ability to design evaluation harnesses and golden datasets
- Experience shipping products across multiple markets or languages
- Cross-functional leadership across engineering, AI teams, linguists, design, and external partners
Benefits
- Education subsidy
- Team building programs
- Company events
- Wellness allowance
- Meal allowance
- Comprehensive healthcare schemes for employees and dependants