Senior/Staff Platform Engineer, Data & AI Infrastructure
K
Keebler HealthHealthcare technology
This is a US-based remote position.Full-TimeSenior
Salary$190K - $260K; $190K – $260K • Offers Equity
Apply NowOpens the employer's application page
Job Details
- Required Skills
- AWSPostgreSQLPythonKubernetesTypeScript
Requirements
- Experience with large-scale search or retrieval systems, such as vector databases, Elasticsearch/Lucene, recommender retrieval, or tuning latency on large indexes.
- Understand ANN index tradeoffs involving memory footprint, recall, and cost.
- Experience building high-throughput systems on quota-limited dependencies; familiarity with token buckets, priority queues, idempotent retries, and graceful degradation.
- Deep experience with Postgres query planning, partitioning, bulk ingestion, connection management, and replica strategy.
- Experience separating OLTP from analytical workloads; Aurora experience is a plus.
- Comfort with AWS networking, VPC design, subnet planning, and EKS networking, or equivalent depth on another major cloud.
- Experience with capacity planning and cost modeling, including projecting system behavior at increased scale.
- Experience with metrics, tracing, and profiling across application and database layers.
- Production experience serving GPU models using tools such as vLLM, SGLang, TGI, TensorRT-LLM, Triton, or TorchServe.
- Understand inference throughput and cost levers, including continuous batching, KV cache memory, quantization, tensor parallelism, and GPU instance selection.
- GPU workload experience on Kubernetes is especially relevant, including node pools, the NVIDIA device plugin, and Karpenter or a similar autoscaler.
- Relevant additional experience may include regulated environments, LLM application infrastructure, startup environments, Python or TypeScript, and infrastructure-as-code tools such as Terraform, CDK, or Pulumi.
Responsibilities
- Scale the retrieval layer as the embedding store grows, defining semantic-search index strategy, latency and recall targets, and cost model.
- Design a shared model gateway with queueing, backpressure, batching, caching, prioritization, and routing across providers and models.
- Evolve AWS networking and compute foundations for data and AI workloads, partnering with the DevOps engineer to anticipate capacity limits and address bottlenecks.
- Define how bulk ingestion, transactional traffic, and analytics should be served, potentially using staging pipelines, replicas, CDC, a warehouse, or a lakehouse.
- Build a self-hosted inference platform for OCR and open-weight LLMs, including benchmarking, model selection, GPU capacity planning, autoscaling, and throughput tuning.
- Create a framework for choosing between self-hosted models and commercial APIs.
- Instrument systems and monitor capacity, latency, throughput, and cost across application and database layers.
View Full Description & ApplyYou'll be redirected to the employer's site