Senior AI Engineer - Agentic Evaluation & V&V
New
S
Slingshot AerospaceSpace Operations
Remote, USFull-TimeSenior
Salary150,000 - 250,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 6+ years
- Required Skills
- Python
Requirements
- 6+ years of experience in software engineering, machine learning engineering, applied AI, or equivalent experience
- Strong Python engineering skills with experience building SDKs, libraries, or evaluation tooling
- Experience designing evaluation frameworks, benchmarks, metrics, or test harnesses for AI/ML systems
- Ability to analyze system behavior, identify failure modes, and evaluate performance in complex autonomous or semi-autonomous systems
- Familiarity with modern agent frameworks, orchestration patterns, or protocol-based integrations
- Experience working in cross-functional, multidisciplinary teams
- Strong written and verbal communication skills
- Bachelor’s degree in a relevant science or engineering field, or equivalent experience
Responsibilities
- Extend and maintain Slingshot’s V&V SDK and evaluation framework for simulation-backed validation of agentic AI systems
- Design and implement agent-level and end-to-end evaluations, including benchmark scenarios, scoring logic, and experiment harnesses
- Build benchmark scenarios and tooling that measure planning, reasoning, and operational performance for autonomous mission planning systems
- Translate astrodynamics and mission-domain concepts into executable evaluation scenarios and simulation configurations
- Develop reusable SDK interfaces, adapters, and evaluation utilities that connect V&V systems, TALOS benchmarks, and agent workflows
- Define and apply metrics for capability evaluation, failure analysis, regression detection, and comparative benchmarking
- Partner with cross-functional teams to identify evaluation needs and contribute to improving coverage of critical capabilities
- Contribute to best practices for evaluating complex, autonomous AI systems
- Uphold strong engineering standards through testing, documentation, reproducibility, and maintainable system design
View Full Description & ApplyYou'll be redirected to the employer's site
