Full-stack Software Engineer III/Senior, AI Gateway
New
N
ngrok Inc.Cloud Networking
This is a remote position for candidates outside of the Bay Area and a hybrid role for candidates within commuting distance to San Francisco.Full-TimeSenior
SalarySenior Software Engineer Tier 1 (SF, LA, Seattle, NYC): $202,500 – $247,500; Tier 2 (rest of US): $186,300 – $227,700. Software Engineer III Tier 1: $180,000 – $220,000; Tier 2: $165,600 – $202,400.
Apply NowOpens the employer's application page
Job Details
- Required Skills
- AWSHTMLCSSJavaKubernetesTypeScriptC++GoReactRust
Requirements
- Proficiency in a statically typed, compiled language such as Go, Rust, C++, or Java (Go preferred).
- Expertise in frontend engineering, specifically TypeScript, React, HTML, and CSS.
- Experience working with AI/LLMs and their associated edge cases.
- Commitment to developer experience and thoughtful system abstractions.
- Understanding of systems engineering concepts like retries, limits, and cost management.
- Strong preference for experience with AI platforms or inference infrastructure.
- Preferred background in API gateways with product-level opinions.
- Preferred experience with usage limits, quotas, or billing-adjacent systems.
- Preferred experience with customer-facing observability tools.
Responsibilities
- Build and evolve AI-aware gateway components that handle traffic in real time.
- Design and implement AI Gateway Traffic Policy Objects including rate limits, usage caps, and access rules.
- Maintain systems for accurately measuring AI usage, including requests, tokens, and related metadata.
- Expose clear signals regarding AI traffic to ensure observability and explainability.
- Collaborate with product and design teams to build AI-specific gateway primitives.
- Work with Gateway, Customer Data, and Platform teams to ensure alignment of usage data, policy enforcement, and billing signals.
View Full Description & ApplyYou'll be redirected to the employer's site
