Software Engineer III/Senior, AI Gateway
New
N
ngrok Inc.Cloud Networking
This is a remote position for candidates outside of the Bay Area and a hybrid role for candidates within commuting distance to San Francisco.Full-TimeSenior
Salary165,600 - 247,500 USD per year
Apply NowOpens the employer's application page
Job Details
- Required Skills
- AWSJavaKubernetesC++GoPostgresRust
Requirements
- Statically typed, compiled language experience (Go, Rust, C++, Java).
- Experience working with AI/LLMs.
- Experience with API gateways and product-level opinions.
- Experience with usage limits, quotas, or billing-adjacent systems.
- Experience with customer-facing observability tools.
- Deep understanding of system design and edge-case handling.
- Commitment to developer experience and thoughtful abstractions.
Responsibilities
- Build and evolve AI-aware gateway components for real-time traffic handling.
- Design and implement AI Gateway Traffic Policy Objects including rate limits, usage caps, and access rules.
- Maintain systems for accurately measuring AI usage, including requests and token consumption.
- Develop observability tools to make AI behavior transparent and explainable.
- Design AI-specific gateway abstractions that hide provider complexity.
- Collaborate with Gateway, Customer Data, and Platform teams to ensure production reliability.
View Full Description & ApplyYou'll be redirected to the employer's site
