SDET / QA Engineer (AI Evaluation)
New
C
CoduranceAI software testing
Working from anywhere in SpainFull-TimeMiddle
Salary38,000 - 43,000 EUR per year
Apply NowOpens the employer's application page
Job Details
- Languages
- Fluent in Spanish and English (speaking and writing)
- Experience
- At least 4 years of experience in a similar role
- Required Skills
- PythonJenkinsSeleniumCI/CDPlaywright
Requirements
- At least 4 years of experience in a similar role.
- Fluent Spanish and English, speaking and writing.
- Consultancy experience and an open mind around adapting quality practices to clients.
- Hands-on experience with Playwright, SonarQube, and testing tools such as Cucumber, Selenium, Appium, and Jenkins.
- Knowledge of CI/CD tools and processes.
- Strong programming skills with Python; other stacks are a plus.
- Practical experience in AI evaluation, including LLM evals, agent testing, multi-turn chat scoring, and golden dataset management.
- QA strategy knowledge, including holistic testing and the Agile Testing Manifesto, is a bonus.
- Anthropic certification is a bonus.
- Performance/load testing for AI agents or call center agents, and multi-channel integration experience, are bonuses.
- Familiarity with BDD and TDD and an ISTQB certificate are bonuses.
Responsibilities
- Take ownership of a Python LLM evaluation tool, diagnostic suite, and golden datasets to raise pass rates on production AI agents.
- Refine scoring criteria and design multi-turn conversational evaluations for AI agents.
- Set up automated regression testing for core product flows using Playwright, covering integrated workflows and third-party channels.
- Implement SonarQube quality gate monitoring and daily deviation reporting.
- Partner with senior stakeholders on QA strategy, standards, and continuous improvement.
- Estimate, prioritize, plan, and coordinate testing activities.
- Stay up to date with new testing tools and test strategies.
View Full Description & ApplyYou'll be redirected to the employer's site
