Performance Modeling Engineer, AI Inference Deployments (Senior/Principal)
New
J
JobgetherAI inference infrastructure
Remote-first working environment, offering flexibility to work from anywhere in the United States.Full-TimeSenior
SalarySenior: $220,000–$270,000. Principal: $270,000–$350,000.
Apply NowOpens the employer's application page
Job Details
- Experience
- At least 5 years of professional experience in performance modeling, capacity planning, systems architecture, or a related field involving large-scale AI inference, high-performance computing (HPC), or data-center infrastructure.
- Required Skills
- PythonFGPA ArchitectureC++Rust
Requirements
- At least 5 years of professional experience in performance modeling, capacity planning, systems architecture, or a related field involving large-scale AI inference, HPC, or data-center infrastructure.
- Experience developing analytical or queueing-based performance models and calibrating predictions against real-world system measurements.
- Systems-level understanding of LLM inference, including prefill and decode, KV cache management, parallelism strategies, MoE routing, batching, TTFT, and tokens-per-second metrics.
- Knowledge of large-scale deployment architectures and factors affecting inference throughput, latency, hardware utilization, and scalability.
- Understanding of data-center networking, including interconnect latency, bandwidth, network topology, and distributed-system performance.
- Knowledge of GPU architecture and performance characteristics, including the ability to compare accelerator platforms.
- Experience building analytical models, conducting performance experiments, interpreting benchmarks, and translating findings into deployment recommendations.
- Ability to establish and document assumptions and communicate uncertainty and error margins when technical information is incomplete.
- Strong Rust programming skills, or expert-level C/C++ proficiency combined with willingness to work extensively in Rust.
- Proficiency in Python for performance analysis, experimentation, data processing, and visualization.
- Strong analytical and problem-solving abilities and understanding of hardware, workload, and system-level performance relationships.
- Written and verbal communication skills for explaining sizing recommendations, performance drivers, and confidence levels to technical teams and customers.
- Hands-on experience sizing, benchmarking, or optimizing production inference deployments across multiple accelerators is highly desirable.
- Experience developing customer-facing performance-estimation, infrastructure-sizing, or TCO tools is an advantage.
Responsibilities
- Translate customer and product-team deployment requirements into structured performance studies, assumptions, scenarios, metrics, and recommendations.
- Compare GPUs and other AI accelerator platforms using objective, apples-to-apples performance studies.
- Design and deliver a standalone performance-estimation tool for customer AI inference workloads and deployment configurations.
- Build and maintain deployment-level performance models for large-scale inference systems and parallelism strategies.
- Evaluate batch sizes, context lengths, workload characteristics, and mixed accelerator configurations.
- Model network and system performance, including interconnect latency, bandwidth, topology, and redundancy.
- Extend analytical and simulation-based deployment models using existing chip performance simulators.
- Analyze configurations to identify bottlenecks, scalability limitations, and deployment-efficiency opportunities.
- Validate and calibrate performance projections against hardware measurements and benchmark results.
- Explain estimates, assumptions, performance drivers, and confidence levels to customers and internal stakeholders.
View Full Description & ApplyYou'll be redirected to the employer's site