Research Engineer, Agentic AI Evals
H
HUDAI Agents
Fully remote-friendly. We already have several fulltime, 100% remote hires. But if you’re in the San Francisco Bay Area or Singapore, we do have an office you can work together in. We do prefer applicants who can show up to meetings in Pacific Time (UTC-7:00/8:00) or China/Singapore Time (UTC +8:00)., Pacific Time (UTC-7:00/8:00), China/Singapore Time (UTC +8:00)Full-TimeMiddle
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- DockerPythonReactLinux
Requirements
- Proficiency in Python
- Proficiency in Docker
- Proficiency in Linux environments
- React experience for frontend development
- Production-level software development experience preferred
- Strong technical aptitude and demonstrated problem-solving ability
- Startup experience in early-stage technology companies
- Strong communication skills for remote collaboration across time zones
- Familiarity with current AI tools and LLM capabilities
- Understanding of safety and alignment considerations in AI systems
- Evidence of rapid learning and adaptability in technical environments (e.g. programming competitions)
- Hands-on experience with or contributed to LLM evaluation frameworks (EleutherAI, Inspect, or similar)
- Built custom evaluation pipelines or datasets
- Worked with agentic or multimodal AI evaluation systems
Responsibilities
- Build out environments for HUD's CUA evaluation datasets, including evals for safety redteaming, general business tasks, long-horizon agentic tasks etc.
- Deliver custom CUA datasets and evaluation pipelines requested by clients
- Contribute to improving the HUD evaluation harness, depending on your interests, skills, and current organizational priorities.
View Full Description & ApplyYou'll be redirected to the employer's site
