AI Tester / AI Evaluation Engineer
New
H
HiberusAI testing
Workplace type: remote; Locations: Wrocław, Plac Teatralny, Wrocław, Country code: plFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English proficiency sufficient for analysing technical documentation.
- Experience
- Minimum 3 years of experience in software testing, analytical systems or data-driven solutions
- Required Skills
- SQLMachine LearningAPI testingRegression testingPostman
Requirements
- Have a minimum of 3 years of experience in software testing, analytical systems or data-driven solutions.
- Have practical knowledge of creating test cases, test scenarios and defect reports.
- Understand the basics of ML or LLM models and the specifics of testing non-deterministic systems.
- Be able to design test suites for evaluating model output quality, classification and information extraction.
- Know SQL, Postman and API testing tools.
- Be able to evaluate model outputs for accuracy, completeness, source consistency, security and usability.
- Experience testing RAG systems, chatbots, AI assistants or predictive models is a plus.
- Knowledge of automated LLM evaluation tools and prompt regression testing is a plus.
- Experience in GenAI security testing, including prompt injection and jailbreak testing, is a plus.
- Knowledge of accessibility, UX or plain-language requirements is a plus.
- English proficiency sufficient for analysing technical documentation is a plus.
Responsibilities
- Prepare test scenarios for AI, ML and GenAI solutions.
- Design question sets, edge cases and reference data for model evaluation.
- Test the quality of model outputs and the stability of AI system behaviour.
- Perform regression testing after changes to models, prompts, source data or configuration.
- Report defects, anomalies and quality-related risks.
- Collaborate with AI Engineers, analysts, business teams and security teams during solution acceptance.
- Document test results and prepare recommendations regarding AI system quality.
View Full Description & ApplyYou'll be redirected to the employer's site
