AI Product Analyst - Voice AI Agents
New
Armenia, CET-overlapping hoursFull-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- Fluent Russian; English level B2 or higher
- Experience
- 3+ years
- Required Skills
- PythonSQLProduct AnalyticsData analyticsLLM
Requirements
- 3+ years of experience in Product Analytics, Data Analytics, Operational Analytics, or analytics roles within complex production systems.
- Strong SQL skills.
- Python for analytical investigations and working with large-scale text datasets.
- Understanding of LLM limitations, hallucinations, evaluation challenges, prompt sensitivity, and AI failure modes.
- Experience building monitoring systems, analytical frameworks, or operational dashboards.
- Strong communication skills and ability to translate analysis into product decisions.
- Ability to work effectively in ambiguous and fast-changing environments.
- Fluent Russian.
- English level B2 or higher.
- Ability to work CET-overlapping hours.
Responsibilities
- Analyze production behavior of Voice AI Agents and conversational AI workflows.
- Investigate hallucinations, inconsistencies, silent failures, and quality degradation in AI-driven conversations.
- Work directly with call transcripts, logs, generated outputs, and operational datasets to uncover patterns and actionable insights.
- Design evaluation frameworks for measuring AI quality, reliability, and business impact.
- Build monitoring systems and dashboards for AI performance visibility.
- Measure the impact of AI systems on operational efficiency, business outcomes, and patient experience.
- Detect quality shifts, anomalies, and emerging failure modes in production environments.
- Partner closely with Product, Engineering, QA, and Operations teams on AI initiatives.
- Support rapid decision-making in fast-changing and ambiguous environments.
View Full Description & ApplyYou'll be redirected to the employer's site
