Senior/Staff Platform Engineer, Data & AI Infrastructure

K
Keebler HealthHealthcare technology
This is a US-based remote position.Full-TimeSenior
Salary$190K - $260K; $190K – $260K • Offers Equity
Apply NowOpens the employer's application page

Job Details

Required Skills
AWSPostgreSQLPythonKubernetesTypeScript

Requirements

  • Experience with large-scale search or retrieval systems, such as vector databases, Elasticsearch/Lucene, recommender retrieval, or tuning latency on large indexes.
  • Understand ANN index tradeoffs involving memory footprint, recall, and cost.
  • Experience building high-throughput systems on quota-limited dependencies; familiarity with token buckets, priority queues, idempotent retries, and graceful degradation.
  • Deep experience with Postgres query planning, partitioning, bulk ingestion, connection management, and replica strategy.
  • Experience separating OLTP from analytical workloads; Aurora experience is a plus.
  • Comfort with AWS networking, VPC design, subnet planning, and EKS networking, or equivalent depth on another major cloud.
  • Experience with capacity planning and cost modeling, including projecting system behavior at increased scale.
  • Experience with metrics, tracing, and profiling across application and database layers.
  • Production experience serving GPU models using tools such as vLLM, SGLang, TGI, TensorRT-LLM, Triton, or TorchServe.
  • Understand inference throughput and cost levers, including continuous batching, KV cache memory, quantization, tensor parallelism, and GPU instance selection.
  • GPU workload experience on Kubernetes is especially relevant, including node pools, the NVIDIA device plugin, and Karpenter or a similar autoscaler.
  • Relevant additional experience may include regulated environments, LLM application infrastructure, startup environments, Python or TypeScript, and infrastructure-as-code tools such as Terraform, CDK, or Pulumi.

Responsibilities

  • Scale the retrieval layer as the embedding store grows, defining semantic-search index strategy, latency and recall targets, and cost model.
  • Design a shared model gateway with queueing, backpressure, batching, caching, prioritization, and routing across providers and models.
  • Evolve AWS networking and compute foundations for data and AI workloads, partnering with the DevOps engineer to anticipate capacity limits and address bottlenecks.
  • Define how bulk ingestion, transactional traffic, and analytics should be served, potentially using staging pipelines, replicas, CDC, a warehouse, or a lakehouse.
  • Build a self-hosted inference platform for OCR and open-weight LLMs, including benchmarking, model selection, GPU capacity planning, autoscaling, and throughput tuning.
  • Create a framework for choosing between self-hosted models and commercial APIs.
  • Instrument systems and monitor capacity, latency, throughput, and cost across application and database layers.
View Full Description & ApplyYou'll be redirected to the employer's site
$190K - $260K; $190K – $260K • Offers Equity
Apply Now