AI Voice Evaluation Specialist
New
I
InnodataAI evaluation
Remote - United StatesContract
Salary20 - 25 USD per hour
Apply NowOpens the employer's application page
Job Details
- Required Skills
- Analytical SkillsWritten communicationCritical thinking
Requirements
- Bachelor’s degree.
- Strong attention to detail and ability to notice subtle differences in conversational quality.
- Excellent listening and comprehension skills.
- Strong written communication skills, including explaining observations clearly and objectively.
- Ability to consistently follow detailed evaluation guidelines.
- Comfort speaking naturally and role-playing different conversational scenarios.
- Ability to compare two interactions fairly without personal preferences influencing the evaluation.
- Strong critical-thinking and analytical skills.
- Reliability and consistency when completing structured evaluation tasks.
- Familiarity with AI assistants, voice-based AI, or conversational systems.
Responsibilities
- Review assigned scenarios and roleplay natural conversations with two AI models.
- Keep the scenario, conversational approach, and overall interaction as consistent as possible across both models.
- Maintain approximately the same number of conversational turns with each model.
- Record your voice during each interaction and assess the quality and behavior of the model responses.
- Evaluate each model across five defined evaluation dimensions.
- Identify and categorize relevant error clusters in the audio or conversation.
- Compare Model A and Model B based on the evaluation criteria and your observations.
- Select the model with stronger overall performance under the evaluation framework.
- Write a detailed rationale for your preference, citing examples and observations from both conversations.
- Apply evaluation guidelines consistently across scenarios and model interactions.
View Full Description & ApplyYou'll be redirected to the employer's site
