Senior Software Engineer - AI Model Evaluation
New
M
MindriftAI Testing
PolandContractSenior
SalaryUp to $40/hr equivalent
Apply NowOpens the employer's application page
Job Details
- Languages
- English (B2+)
- Experience
- 5+ years
- Required Skills
- DockerPostgreSQLPythonJavascriptKafkaTypeScriptFastAPIRedisReact
Requirements
- 5+ years of experience in software development.
- Proficiency in Python (FastAPI), JavaScript/TypeScript (React), Docker, Postgres, Kafka, and Redis.
- Experience writing functional and integration tests.
- B2+ English proficiency.
- Ability to submit tasks by project deadlines meeting specific acceptance criteria.
- Strong understanding of model failures and scenario-based evaluation.
Responsibilities
- Build realistic virtual developer environments with codebases, infrastructure, and project context.
- Design tasks from intermediate environmental states, crafting prompts and defining solvable success criteria for AI agents.
- Write robust tests to verify agent solutions, ensuring they accept valid approaches and reject incorrect ones.
- Iterate on tasks and tests based on QA feedback by reviewing agent outputs and analyzing failures.
- Refine evaluations to ensure they remain fair, robust, and challenging for frontier AI models.
View Full Description & ApplyYou'll be redirected to the employer's site
