Principal Engineer, Model Performance
New
R
RunwareAI inference
United States, We keep core hours for collaboration and calls.Full-TimePrincipal
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 10+ years in GPU, HPC or deep learning systems engineering
- Required Skills
- PythonFGPA ArchitecturePyTorchC++
Requirements
- Have 10+ years of experience in GPU, HPC, or deep learning systems engineering.
- Have several years optimizing inference performance for production models at a company running them at scale.
- Demonstrate a track record of shipping performance improvements on real hardware.
- Bring hands-on CUDA and GPU architecture expertise, including kernel design, memory hierarchy, occupancy, Tensor Cores, fused kernels, CUDA Graphs, and NCCL.
- Use Nsight Systems and Nsight Compute profilers.
- Have production experience with at least one major inference engine, such as TensorRT-LLM, vLLM, or SGLang.
- Know modern serving techniques including continuous batching, paged KV cache, speculative decoding, prefill/decode disaggregation, quantization, and tensor, pipeline, and expert parallelism.
- Be fluent in Python and C++, and comfortable with PyTorch internals, torch.compile, and Triton.
- Have experience building or leading a performance engineering team, hiring, and setting technical direction while remaining hands-on.
- Be comfortable owning outcomes and making decisions in a fast-moving, ambiguous environment.
Responsibilities
- Own inference performance across the fleet’s image, video, audio, 3D, and LLM workloads.
- Set the model performance roadmap based on traffic, cost, hardware, and upcoming workloads.
- Write and tune CUDA kernels, fuse attention and MLP paths, and implement quantization formats while maintaining model quality.
- Develop parallelism and batching strategies for multi-GPU nodes.
- Build the pipeline for optimizing and deploying newly released open models in production.
- Choose and shape the serving stack, including TensorRT-LLM, vLLM, SGLang, custom pipelines, and the Sonic Inference Engine runtime.
- Own reproducible, hardware-aware benchmarking and performance regression testing in CI.
- Partner with infrastructure and hardware teams on GPU memory, storage, and network bottlenecks.
- Hire, mentor, and lead the Model Performance team.
- Use AI-based agents for kernel search and tuning, benchmark generation, profiling triage, and documentation.
View Full Description & ApplyYou'll be redirected to the employer's site
