Senior Software Engineer (Python) - Agent Evaluation
New
M
MindriftAI Systems Evaluation
Open to candidates in the North America, South America, Asia and Europe.ContractSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- English proficiency at B2 level or higher.
- Experience
- 5+ years of professional experience with Python.
- Required Skills
- DockerPostgreSQLPythonTypeScriptFastAPIReactCI/CD
Requirements
- 5+ years of professional experience with Python.
- Strong experience with FastAPI, pytest, and async/await.
- Hands-on experience with Docker, PostgreSQL, and CI/CD pipelines.
- Proven experience writing and maintaining automated tests.
- English proficiency at B2 level or higher.
- Availability to work 30+ hours per week.
- Ability to complete a technical assessment.
- Willingness to complete identity verification.
- Willingness to join and participate in Discord for project communication.
- Full-stack experience with React and TypeScript is a plus.
Responsibilities
- Build realistic virtual developer environments with codebase, infrastructure, and context.
- Design challenging tasks from intermediate environmental states.
- Define clear success criteria for AI agent performance.
- Write automated tests that verify agent solutions while allowing for diverse correct approaches.
- Iterate on tasks and tests based on QA feedback.
- Analyze agent failures and refine evaluations for robustness.
View Full Description & ApplyYou'll be redirected to the employer's site
