Senior AI Engineer - Model Selection and Orchestration
New
Z
ZipFinancial technology
Remote-first opportunity for US-based employees with the option to work in-person out of our Manhattan office.Full-TimeSenior
Salary$162,000 - $205,000
Apply NowOpens the employer's application page
Job Details
- Experience
- 10+ years of professional experience building, testing, deploying, and operating production software; or 6 years and a Master’s degree; or a PhD with 3 years experience.
- Required Skills
- PythonCloud ComputingMachine LearningTypeScriptCI/CDLLMDistributed Systems
Requirements
- 10+ years of professional experience building, testing, deploying, and operating production software.
- Practical experience with AI/ML-enabled applications beyond prototypes.
- Strong experience with Python or TypeScript.
- Modern software engineering practices: API design, automated testing, distributed services, and debugging.
- Practical experience using LLM tool calling, structured outputs, RAG, and agent/workflow orchestration.
- Ability to evaluate classification quality, uncertainty, and fallback strategies using measurable outcomes.
- Experience with cloud deployment, CI/CD, observability, identity/access controls, and secrets management.
- Strong judgment on when to use deterministic software vs. models or retrieval.
- Ability to independently deliver complex technical work within shared architecture standards.
- Hands-on experience using AI-assisted development tools.
Responsibilities
- Design and implement the decision layer that determines agent response approaches including deterministic logic, retrieval, models, and human escalation.
- Develop intent classification and clarification capabilities to distinguish information requests from actions and identify ambiguous requests.
- Benchmark models and retrieval approaches against outcomes like task success, latency, reliability, and inference cost.
- Build model orchestration including adapters, routing logic, structured outputs, retries, and fallback strategies.
- Connect agents to approved APIs and tools with focus on authentication, idempotency, and recovery scenarios.
- Implement input/output protections, prompt-injection defenses, and controlled access in partnership with Security and Risk.
- Develop reusable orchestration patterns and model adapters for domain engineering teams.
- Monitor system behavior and use production evidence to continuously improve AI model selection.
View Full Description & ApplyYou'll be redirected to the employer's site
