Staff AI Product Analyst
New
A
AngiHome Services
Remote - United StatesFull-TimeStaff
Salary165,000 - 225,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 10+ years of experience across product analytics, data science, or technical product management
- Required Skills
- PythonProduct AnalyticsData scienceLLM
Requirements
- 10+ years of experience across product analytics, data science, or technical product management
- 1+ years of direct, hands-on experience working with AI/LLM systems
- Deep technical proficiency with Python
- Experience with ML frameworks and data visualization/analytics platforms
- Proven track record designing AI evaluation frameworks
- Experience with production monitoring
- Strong communication skills for translating technical insights for non-technical stakeholders
- Systems-oriented mindset
Responsibilities
- Partner with AI product, engineering, and domain experts to define metrics for flagship AI journeys
- Align on test design, interpretation, and causal framing for new AI products
- Map agent data interactions to ensure structured and testable requirements
- Design and maintain evaluation sets, golden datasets, and regression checks
- Categorize failure modes and feed insights back into prompts, RAG, and data pipelines
- Monitor live usage of Agentic AI workflows and manage engineering improvement backlogs
- Own internal playbooks regarding AI agent usage and human checkpoints
- Deliver recurring measurement artifacts to inform leadership roadmaps
View Full Description & ApplyYou'll be redirected to the employer's site
