Senior Software Engineer, Agent Infrastructure
New
C
CohereAI infrastructure
Location: Toronto; Secondary Locations: Canada, United States; Workplace: RemoteFull-TimeSenior
Salary$160K - $260K; $160K – $260K • Offers Equity • Multiple Ranges; USA - California, New York and Washington: USA - California, New York and Washington $180K – $260K • Offers Equity; USA - All Other States: USA - All Other States $160K – $235K • Offers Equity; Canada: Canada CA$230K – CA$330K • Offers Equity
Apply NowOpens the employer's application page
Job Details
- Required Skills
- KubernetesDistributed Systems
Requirements
- Have experience building production ML infrastructure with strong systems fundamentals.
- Have hands-on experience with agentic systems, multi-agent workflows, or agent development frameworks.
- Be familiar with model routing and LLM provider frameworks across different model types and environments.
- Have experience with scalable, fault-tolerant distributed systems and Kubernetes.
- Have a track record of moving quickly on prototypes and making sound productionization decisions.
- Experience across on-prem, private cloud, and public cloud environments is a bonus.
- Familiarity with storage systems, embedded databases, or filesystem abstractions is a bonus.
- Experience with code execution sandboxes such as gVisor, Firecracker, Kata, or WASM runtimes is a bonus.
- Experience with identity, workload authentication, or capability-based security systems is a bonus.
Responsibilities
- Build secure execution environments for agent-generated code.
- Develop identity, authentication, and trust boundaries for agents.
- Build model routing and orchestration across different model types and environments.
- Implement rate limiting, quotas, and resource management for agent workflows.
- Develop state management, memory, and filesystem abstractions for agents.
- Turn emerging ML research ideas into production-ready infrastructure.
- Build core platform capabilities for execution, storage, and state management.
- Prototype and evaluate technologies, and help decide what should move into production.
- Partner with research teams to shape infrastructure for future agent systems.
View Full Description & ApplyYou'll be redirected to the employer's site
