AI Product Lead, AI Experience & Quality
New
F
FlodeskEmail Marketing
RemoteFull-TimeLead
Salary$150,000 - $200,000 base salary, depending on your location and experience. Tier 1 city residents: $165,000 - $200,000; All other locations: $150,000 - $185,000
Apply NowOpens the employer's application page
Job Details
- Required Skills
- Product ManagementJSONPrompt EngineeringLLM
Requirements
- Proven experience shaping or meaningfully improving real LLM or agentic products.
- Practical knowledge of prompts, context, tools, agent behavior, and evaluation processes.
- Hands-on experience with prompt iteration, evaluation tools, and trace review.
- Comfortable working with JSON, schemas, and structured outputs.
- Strong product judgment to turn ambiguous needs into clear behavior and testable hypotheses.
- Experimental rigor with the ability to design test sets and failure taxonomies.
- Systems-thinking ability to create reusable patterns for AI surfaces.
- Clear communication across technical and non-technical disciplines.
- Ability to balance tradeoffs between member value, quality, latency, reliability, and cost.
Responsibilities
- Manage member-facing AI behavior across prompt-to-create, agentic editing, and future AI-driven jobs.
- Create, test, version, and document prompts and agent instructions.
- Define AI context, member data, tools, and product state requirements.
- Own AI evaluation practice, including representative datasets, behavioral scenarios, and scoring rubrics.
- Run recurring evaluation and quality-monitoring processes independently.
- Diagnose interaction failures to identify issues with prompts, context, models, or tools.
- Turn member research, support feedback, and analytics into product priorities and regression cases.
- Partner with engineering and product teams to encode a coherent brand point of view into the AI system.
View Full Description & ApplyYou'll be redirected to the employer's site
