Software Engineer - AI Agent Evaluation
New
M
MindriftAI Testing
New York, New York, United States. United StatesPart-TimeSenior
SalaryUp to $50/hr equivalent
Apply NowOpens the employer's application page
Job Details
- Languages
- B2+ English
- Experience
- 5+ years
- Required Skills
- DockerPostgreSQLPythonJavascriptKafkaTypeScriptFastAPIRedisReact
Requirements
- 5+ years of experience in software development.
- Proficiency with Python and FastAPI.
- Proficiency with JavaScript/TypeScript and React.
- Experience with Docker.
- Experience with Postgres.
- Experience with Kafka and Redis.
- Experience writing functional and integration tests.
- B2+ level English proficiency.
Responsibilities
- Build realistic developer environments including codebase, infrastructure, and project context.
- Design complex tasks for AI coding agents from intermediate project states.
- Define clear success criteria and 'solved' states for AI coding tasks.
- Write verification tests that accept valid solutions and reject incorrect ones.
- Analyze agent failures and iterate on task designs based on QA feedback.
View Full Description & ApplyYou'll be redirected to the employer's site
