QA Automation Engineer
New
Warszawa, Rondo Daszyńskiego, Belgrad, Prince Michael 1, Lisbona, R. Saraiva de Carvalho 1C, Sarajewo, Maglajska 1, Ciechanów, st. Hincesti highway, 53, Warszawa, Country code: PLFull-TimeMiddle
Salary2,500 - 3,500 USD per month
Apply NowOpens the employer's application page
Job Details
- Languages
- English B2
- Experience
- 3+ years
- Required Skills
- CypressQA AutomationTypeScriptAPI testingSeleniumCI/CDPlaywright
Requirements
- 3+ years experience in QA Automation for AI/LLM-powered SaaS platforms and agentic systems.
- Hands-on experience with Playwright, Cypress, Selenium, or similar.
- Solid understanding of autonomous/multi-agent workflow validation.
- Experience testing non-deterministic AI behavior and probabilistic outputs.
- Ability to design LLM evaluation frameworks for reasoning, hallucination, and safety.
- Strong API testing and event-driven architecture experience.
- Experience validating long-running workflows and failure recovery.
- Strong debugging skills across stack layers.
- Experience with CI/CD, regression testing, and observability.
- Intermediate+ English level.
Responsibilities
- Design, develop, and maintain automated testing frameworks for AI/LLM-powered SaaS platforms.
- Validate end-to-end autonomous workflows, including prompt orchestration, tool calling, memory/context handling, and RAG pipelines.
- Build automated evaluation pipelines for reasoning quality, hallucinations, accuracy, and safety.
- Create regression suites for long-running workflows, state persistence, and failure recovery.
- Perform API, async, webhook, and event-driven system testing across distributed architectures.
- Debug issues across frontend, backend, infrastructure, and LLM layers.
- Collaborate with AI and backend engineers to improve reliability and scalability.
- Contribute to CI/CD automation, tracing, monitoring, and synthetic dataset generation.
View Full Description & ApplyYou'll be redirected to the employer's site
