Senior Technical Product Manager - AI Agents, Evals & Reliability
New
J
JobgetherArtificial Intelligence
United StatesFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- Machine LearningProduct ManagementAlgorithmsData Structures
Requirements
- Strong foundation in computer science fundamentals including algorithms, data structures, and system design.
- Solid understanding of machine learning fundamentals and modern AI system behavior in production.
- Significant experience owning complex, technically sophisticated products from strategy through execution.
- Hands-on experience with AI-powered products, specifically LLM-based systems, model evaluation, and prompt/pipeline iteration.
- Strong intuition for AI limitations and failure modes such as hallucinations, bias, model drift, and non-deterministic behavior.
- Ability to read, review, and discuss technical design documents and collaborate with senior engineers.
- Demonstrated ability to make sound product and technical decisions in ambiguous, rapidly changing environments.
- Strong judgment and prioritization skills to balance ambitious goals against technical, operational, and commercial realities.
- Excellent communication and collaboration skills for working across highly technical, cross-functional teams.
- Independent working style with the ability to bring structure to complex problems.
- Experience working in high-talent-density or small, fast-moving teams.
Responsibilities
- Define end-to-end requirements for AI-powered systems, connecting model capabilities and technical constraints to user needs.
- Translate model performance, data limitations, evaluation results, and user feedback into clear product and system decisions.
- Partner closely with ML, backend, mobile, and other engineering teams on architecture, system design, evaluation, and delivery.
- Establish and continuously improve evaluation frameworks spanning offline metrics, online experiments, human feedback, and real-world user outcomes.
- Make informed trade-offs across product quality, reliability, latency, cost, and user experience.
- Drive execution through clear product specifications, disciplined prioritization, and strong technical judgment.
- Own product quality end-to-end, with focus on correctness, predictability, reliability, and failure handling.
- Identify weaknesses in AI behavior, such as hallucinations, bias, and model drift, and drive solutions.
- Establish feedback loops to learn from production behavior and improve AI-powered workflows.
View Full Description & ApplyYou'll be redirected to the employer's site
