Senior Machine Learning Engineer, Inference
New
J
JobgetherMachine learning inference
Remote opportunities available in the United States, Canada, the United Kingdom, Germany, Switzerland, and Serbia.Full-TimeSenior
Salary$250,000–$300,000 annual base salary, plus competitive equity.
Apply NowOpens the employer's application page
Job Details
- Languages
- Excellent English communication skills.
- Experience
- At least 5 years of experience in ML inference or systems engineering, at a senior level equivalent to Google L5 or Meta E5.
- Required Skills
- PythonKubernetesC++Rust
Requirements
- Have at least 5 years of experience in ML inference or systems engineering, at a senior level equivalent to Google L5 or Meta E5.
- Have practical experience optimizing inference engines such as vLLM or TRT-LLM at the engine level, beyond basic model deployment.
- Bring a proven background developing backend and machine learning systems and serving models in production.
- Have systems programming and GPU optimization skills using C++, CUDA, Rust, or optimized Python.
- Have hands-on knowledge of quantization, distillation, speculative decoding, caching, continuous batching, and paged attention.
- Have experience with distributed systems, Kubernetes, Ray, load balancing, multi-GPU and multi-node inference, and high-concurrency environments.
- Have excellent English communication skills.
- Additional advantages include open-source inference engine contributions or technical publications, rapid career progression, and a PhD in Computer Science, Physics, or Mathematics, or a degree from a leading computer science program.
Responsibilities
- Own the optimization process from research models to production deployment.
- Improve latency and throughput for real-time voice and LLM inference at consumer scale.
- Optimize inference engines such as vLLM, TensorRT-LLM (TRT-LLM), and SGLang.
- Implement model acceleration techniques, including quantization, distillation, speculative decoding, caching, continuous batching, and paged attention.
- Develop high-performance GPU workloads using C++, CUDA, Rust, or optimized Python.
- Build and scale distributed multi-GPU and multi-node inference systems using Kubernetes, Ray, and custom load-balancing solutions.
- Collaborate with research and product teams to bring new models into production.
View Full Description & ApplyYou'll be redirected to the employer's site
