- Set the vision, strategy, and roadmap for AI skills.
- Interview customers, observe their workflows, test AI skills, and use findings to inform the roadmap and evaluation sets.
- Prioritize the backlog based on broker value, technical feasibility, and quality risk, and manage work from concept through launch and iteration.
- Build and maintain evaluation sets and quality standards for AI skills, and review regressions before releases.
- Choose between deterministic logic and LLMs for individual skills.
- Own prompt, context, and retrieval design and improve it using measured evaluations.
- Set guardrails for grounding, permissions, confirmation before actions, hallucination controls, and rollback plans.
- Translate broker workflows into buildable skill specifications for engineering.
- Measure product outcomes and model quality, while monitoring latency and cost against quality.
Product ManagementPrompt Engineering