Senior Software Engineer (AI Inference & Runtime Platform)
New
A
AZXArtificial Intelligence
Fully remote culture with a cluster of teammates in SeattleFull-TimeSenior
Salary$140K - $225K
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years
- Required Skills
- KubernetesC++FastAPIGoRustTerraform
Requirements
- 5+ years of experience shipping production systems in a systems language (Rust, Go, C/C++, or Zig).
- Expertise in operating Kubernetes workloads including controllers, operators, scheduling, and autoscaling.
- Strong understanding of threat-modeling isolation boundaries (namespaces, cgroups, hypervisors) for untrusted code execution.
- Hands-on experience deploying or operating open-weight LLM serving infrastructure like vLLM or SGLang.
- Deep familiarity with performance optimization and debugging in distributed systems.
- Experience with isolation technologies such as Firecracker, Kata, or gVisor.
- Proficiency with infrastructure-as-code tools (Terraform/OpenTofu, Bicep) and observability platforms (OpenTelemetry).
- Experience managing stateful services like vector stores (pgvector/Qdrant) or graph stores (Neo4j).
- Ability to travel 2x per year for company summits.
- Bachelor's degree in a relevant field; Master's is a plus.
Responsibilities
- Manage the serving tier for open-weight models including engine configuration, cold-start strategies, and performance SLOs.
- Administer Kubernetes infrastructure for inference workloads, focusing on operators, autoscaling (KEDA/Karpenter), and GPU scheduling.
- Operate the stateful data plane including vector and graph stores with robust backup, recovery, and scaling strategies.
- Oversee the sandbox runtime control plane for agent-generated code, covering lifecycle management, security, and resource metering.
- Develop FastAPI control-plane services and maintain infrastructure as code using Terraform/OpenTofu or Bicep.
- Collaborate with backend services teams to debug services and maintain observability dashboards.
- Maintain open-source standards with high-quality documentation, reproducible builds, and code review practices.
View Full Description & ApplyYou'll be redirected to the employer's site
