1. Job Description — HAI Quality Evaluation EPM

Summary

Key Responsibilities

• Drive end-to-end program execution for HAI Quality Evaluation, including roadmap development, prioritization, planning, milestone tracking, dependency management, and risk mitigation.
• Partner with researchers, engineers, and evaluation teams to translate research and evaluation objectives into clear program roadmaps, priorities, milestones, and execution plans.
• Lead prioritization across competing evaluation requirements, platform needs, geo-expansion initiatives, and engineering dependencies.
• Manage complex platform and tooling dependencies required to execute quality evaluation programs at scale.
• Serve as the primary program interface between Quality Evaluation and supporting platform and tooling teams, ensuring requirements are clearly defined, prioritized, and incorporated into partner roadmaps.
• Proactively identify platform gaps and blockers, drive cross-functional discussions to resolution, and establish clear ownership and timelines for critical dependencies.
• Partner with platform teams on requirements, integration plans, launch readiness, and longer-term capabilities needed to scale evaluation workflows.
• Drive geo-expansion programs supporting Market Intelligence and Content Enrichment, coordinating readiness across evaluation, engineering, research, platform, and regional stakeholders.
• Develop scalable mechanisms for evaluating GenAI experiences across multiple markets, languages, and locales, accounting for market-specific requirements and dependencies.
• Partner closely with researchers to understand evaluation methodologies, metrics, datasets, and quality criteria and translate them into executable cross-functional programs.
• Establish clear mechanisms for tracking evaluation readiness, platform readiness, geo-expansion readiness, risks, dependencies, and overall program health.
• Drive alignment and decision-making across teams with different priorities, ensuring critical issues and tradeoffs are surfaced early and resolved.
• Communicate program status, risks, technical dependencies, priorities, and decisions clearly to both technical teams and leadership.
• Create scalable processes that improve visibility, accountability, prioritization, and execution across Quality Evaluation programs.

Minimum Qualifications

• Bachelor’s degree or equivalent experience in Computer Science, Engineering, Data Science, or a related technical field.
• 5+ years of experience in engineering program management, technical program management, product management, or a related role supporting complex technical organizations.
• Experience working with AI/ML, Generative AI, research, evaluation, or data-focused engineering teams.
• Proven experience developing and managing technical roadmaps and prioritization frameworks across multiple teams and competing requirements.
• Experience managing programs with significant platform, infrastructure, or tooling dependencies.
• Demonstrated ability to work with platform and engineering teams to define requirements, align priorities, manage dependencies, and drive blockers to resolution.
• Experience partnering closely with researchers and engineers, with sufficient technical fluency to understand evaluation approaches, system dependencies, technical constraints, and engineering tradeoffs.
• Strong cross-functional leadership skills with demonstrated ability to drive alignment across teams without direct authority.
• Proven ability to operate effectively in ambiguous environments and translate complex problems into clear priorities and execution plans.
• Excellent written and verbal communication skills, with the ability to communicate effectively across researchers, engineers, platform teams, product teams, and senior stakeholders.
• Strong analytical and problem-solving skills with the ability to use data and evaluation results to inform program priorities and decisions.
• Ability to work effectively with globally distributed teams and stakeholders.

Good to Have

• Experience supporting GenAI or Large Language Model (LLM) evaluation programs, including human and/or automated evaluation.
• Understanding of GenAI evaluation concepts such as quality metrics, human evaluation, automated evaluation, model comparison, evaluation datasets, and launch readiness criteria.
• Experience managing AI/ML platforms, evaluation platforms, developer tooling, or internal engineering platforms.
• Experience working with platform teams where your program is a consumer of shared infrastructure or tooling, requiring roadmap alignment and prioritization across organizational boundaries.
• Experience with internationalization, localization, or geo-expansion of AI/ML or consumer-facing products.
• Experience coordinating launches across multiple languages, locales, and markets, particularly in APAC.
• Experience working with Market Intelligence, content understanding, content enrichment, or similar data and AI programs.
• Familiarity with the ML lifecycle, including data collection, model development, evaluation, experimentation, launch, and production monitoring.
• Experience defining and managing quality gates, evaluation thresholds, readiness criteria, and go/no-go decisions for AI/ML features.
• Experience working across research and production engineering organizations to transition capabilities from research and experimentation into scalable production systems.