Performance Modeling Engineer, AI Inference Deployments (Senior/Principal)

New
J
JobgetherAI inference infrastructure
Remote-first working environment, offering flexibility to work from anywhere in the United States.Full-TimeSenior
SalarySenior: $220,000–$270,000. Principal: $270,000–$350,000.
Apply NowOpens the employer's application page

Job Details

Experience
At least 5 years of professional experience in performance modeling, capacity planning, systems architecture, or a related field involving large-scale AI inference, high-performance computing (HPC), or data-center infrastructure.
Required Skills
PythonFGPA ArchitectureC++Rust

Requirements

  • At least 5 years of professional experience in performance modeling, capacity planning, systems architecture, or a related field involving large-scale AI inference, HPC, or data-center infrastructure.
  • Experience developing analytical or queueing-based performance models and calibrating predictions against real-world system measurements.
  • Systems-level understanding of LLM inference, including prefill and decode, KV cache management, parallelism strategies, MoE routing, batching, TTFT, and tokens-per-second metrics.
  • Knowledge of large-scale deployment architectures and factors affecting inference throughput, latency, hardware utilization, and scalability.
  • Understanding of data-center networking, including interconnect latency, bandwidth, network topology, and distributed-system performance.
  • Knowledge of GPU architecture and performance characteristics, including the ability to compare accelerator platforms.
  • Experience building analytical models, conducting performance experiments, interpreting benchmarks, and translating findings into deployment recommendations.
  • Ability to establish and document assumptions and communicate uncertainty and error margins when technical information is incomplete.
  • Strong Rust programming skills, or expert-level C/C++ proficiency combined with willingness to work extensively in Rust.
  • Proficiency in Python for performance analysis, experimentation, data processing, and visualization.
  • Strong analytical and problem-solving abilities and understanding of hardware, workload, and system-level performance relationships.
  • Written and verbal communication skills for explaining sizing recommendations, performance drivers, and confidence levels to technical teams and customers.
  • Hands-on experience sizing, benchmarking, or optimizing production inference deployments across multiple accelerators is highly desirable.
  • Experience developing customer-facing performance-estimation, infrastructure-sizing, or TCO tools is an advantage.

Responsibilities

  • Translate customer and product-team deployment requirements into structured performance studies, assumptions, scenarios, metrics, and recommendations.
  • Compare GPUs and other AI accelerator platforms using objective, apples-to-apples performance studies.
  • Design and deliver a standalone performance-estimation tool for customer AI inference workloads and deployment configurations.
  • Build and maintain deployment-level performance models for large-scale inference systems and parallelism strategies.
  • Evaluate batch sizes, context lengths, workload characteristics, and mixed accelerator configurations.
  • Model network and system performance, including interconnect latency, bandwidth, topology, and redundancy.
  • Extend analytical and simulation-based deployment models using existing chip performance simulators.
  • Analyze configurations to identify bottlenecks, scalability limitations, and deployment-efficiency opportunities.
  • Validate and calibrate performance projections against hardware measurements and benchmark results.
  • Explain estimates, assumptions, performance drivers, and confidence levels to customers and internal stakeholders.
View Full Description & ApplyYou'll be redirected to the employer's site
Senior: $220,000–$270,000. Principal: $270,000–$350,000.
Apply Now