Software Engineer – AI Evaluation
New
P
Patronus AIAI evaluation
Remote (India)ContractJunior
Salary50 - 100 USD per hour
Apply NowOpens the employer's application page
Job Details
- Experience
- 2+ years of professional experience in software engineering. (internships excluded)
- Required Skills
- PythonMachine LearningSoftware Engineering
Requirements
- Have 2+ years of professional software engineering experience; internships do not count.
- Have experience with system design, data pipelines, ML, or LLM-based applications.
- Have prior experience in code review, technical writing, AI training, or model evaluation.
- Have strong coding skills in at least one major language, such as Python, Java, C++, JavaScript, TypeScript, or Go.
- Have experience building, reviewing, and debugging production-grade code.
- Be able to follow detailed technical guidelines, work independently, respond to feedback, and meet deadlines.
- Have excellent written communication and close attention to detail.
- Have a bachelor’s degree in computer science, engineering, or a related field, or equivalent experience.
- Author original tasks and solutions; do not use LLM-generated code, content, or text.
- Do not copy published or easily searchable material, or use employer, client, studio, or NDA-covered code or project files.
Responsibilities
- Design realistic, multi-step coding and technical tasks that test AI models’ reasoning, coding, and ability to follow precise instructions.
- Build tasks around codebases, technical specifications, architecture documents, and datasets.
- Evaluate AI-generated code and technical content for correctness, efficiency, clarity, and alignment with engineering practice.
- Grade model outputs against rubrics and ground-truth answers, and write structured feedback.
- Help improve AI understanding of programming, system design, data workflows, and engineering best practices.
- Test and refine content to reflect real-world complexity.
- Review other contributors’ work for accuracy, clarity, and practical relevance.
- Work independently and asynchronously, meet deadlines, collaborate with the Patronus team, and incorporate feedback promptly.
View Full Description & ApplyYou'll be redirected to the employer's site
