AI Product Analyst

New

Serbia, CET-overlapping hoursFull-TimeMiddle

Salary not disclosed

Apply NowOpens the employer's application page

Job Details

3+ years of experience in analytical roles related to AI systems, ML products, automation platforms, or complex production systems
Strong Python skills for deep analytical investigations and working with large-scale noisy text data
Advanced SQL skills
Understanding of LLM limitations, hallucinations, evaluation challenges, prompt sensitivity, model trade-offs, and AI failure modes
Experience building monitoring systems, analytical frameworks, or operational dashboards for production systems
Ability to work with incomplete, ambiguous, and rapidly changing data environments
Strong ownership mindset and ability to independently drive investigations and analytical conclusions
Ability to connect AI system quality with business and operational impact
Fluent Russian; English level B2 or higher
Ability to work CET-overlapping hours

Analyze production behavior of AI agents, automation systems, and LLM-powered workflows
Investigate hallucinations, inconsistencies, failure modes, and quality degradation across automated user journeys
Work directly with logs, conversations, raw system outputs, and operational datasets to identify hidden patterns and actionable insights
Design evaluation approaches and analytical frameworks for measuring AI system quality and reliability
Build monitoring systems and dashboards for AI performance visibility
Identify operational and product risks introduced by automation
Measure the impact of AI systems on business metrics, operational efficiency, and user experience
Detect data quality issues, shifts, and anomalies affecting AI system performance
Partner closely with Product, Engineering, QA, and Operations teams on AI-related initiatives
Support rapid decision-making in fast-changing and ambiguous environments

View Full Description & ApplyYou'll be redirected to the employer's site