Senior Inference Engineer
New
V
vCluster LabsCloud Infrastructure
United States: $190K – $230K; Germany: €190K – €230KFull-TimeSenior
Salary$190K - $230K; United States: $190K – $230K; Germany: €190K – €230K
Apply NowOpens the employer's application page
Job Details
- Required Skills
- DockerPythonKubernetesGoLLM
Requirements
- Production LLM serving experience using vLLM, SGLang, or TensorRT-LLM.
- Hands-on experience with inference optimization techniques: quantization, batching, caching, and routing.
- Strong proficiency in Python or Golang with real production code experience.
- Ability to clearly explain technical concepts to engineering and non-technical stakeholders.
- Familiarity with containerized environments like Docker and Kubernetes (bonus).
- Generative AI experience with frameworks such as PyTorch or Transformers (bonus).
- Understanding of the GPU stack including CUDA, NCCL, drivers, and libraries (bonus).
- Knowledge of model architectures and fine-tuning approaches (bonus).
- Experience with NVIDIA Dynamo (bonus).
Responsibilities
- Deploy LLMs to production on GPU infrastructure.
- Build and operate inference serving infrastructure using vLLM, SGLang, or TensorRT-LLM.
- Optimize inference pipelines using quantization, batching, caching, and routing to manage latency and cost.
- Develop production infrastructure using Python or Golang.
- Own the inference roadmap and partner with Product to drive engineering direction.
View Full Description & ApplyYou'll be redirected to the employer's site
