Lead QA Engineer (AI Voice)
New
L
LingaroVoice AI
Listing location: Poland; Workplace type: Remote. You can choose to work remotely or in the office.Full-TimeLead
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English at least B2.
- Experience
- At least 6 years of experience in software QA; at least 2 years of experience in testing voice agents or conversational AI in live delivery.
- Required Skills
- PythonJavascriptTypeScriptAPI testingCI/CDPostman
Requirements
- At least 6 years of experience in software QA.
- At least 2 years of experience testing voice agents or conversational AI in live delivery.
- Experience leading E2E and UAT, and working as the lone or lead QA with limited guidance.
- Experience in international or distributed teams.
- Experience testing speech recognition and speech output, latency, interruptions, accents, and noise.
- Knowledge of voice quality metrics such as word error rate, intent accuracy, and task completion.
- Practical understanding of how LLMs work and fail; deep LLM engineering is not required.
- Test automation experience in Python or JavaScript/TypeScript, with CI/CD and API testing such as Postman.
- Experience using cloud speech services such as Azure, Google, or AWS to generate test calls.
- Practical experience with Jira or Azure DevOps, a test management tool, and troubleshooting from logs, transcripts, and call recordings.
- English at least B2.
- Voice testing platforms, LLM evaluation tools, telephony basics, ISTQB certification, Responsible-AI testing, and customer journey validation across multiple markets are listed as pluses.
Responsibilities
- Design the voice AI test strategy and process from speech input through speech recognition, the LLM, tools and integrations, to speech output.
- Define a scalable automation approach, including multilingual coverage of around 100 languages.
- Test predictable behaviour such as intents, routing, and integrations, as well as non-deterministic AI behaviour.
- Evaluate prompts and LLM responses using test datasets and scoring criteria.
- Lead UAT planning, scenarios, test data, business tester coordination, and sign-off recommendations.
- Gather business acceptance criteria and turn them into measurable targets.
- Own defect and risk management, and provide release go/no-go recommendations.
- Replan and communicate options when development is delayed.
- Challenge stakeholders on readiness and quality, and mentor the Regular engineer.
View Full Description & ApplyYou'll be redirected to the employer's site
