AI Research Engineer (Kernel & Inference Optimization)
New
T
TetherAI inference
100% Remote WorldwideFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Languages
- Excellent English communication skills
Requirements
- Hold a degree in Computer Science or a related field; a PhD in NLP, Machine Learning, or a related field is preferred.
- Have a track record in AI research and development, with publications in A* conferences.
- Know Metal Shading Language (MSL) and be comfortable writing custom compute shaders from scratch.
- Have experience in low-level kernel optimization and inference optimization on mobile devices.
- Demonstrate measurable improvements in inference latency, throughput, and memory footprint for domain-specific applications.
- Understand modern model-serving architectures and inference optimization techniques, including memory management in resource-constrained deployments.
- Have expertise writing GPU kernels for smartphones and familiarity with model-serving frameworks and engines.
- Have practical experience developing and deploying end-to-end inference pipelines on resource-constrained devices.
- Be able to design evaluation frameworks and apply empirical research to latency, computational bottleneck, and memory-constraint challenges.
- Know distributed inference techniques including tensor, pipeline, and expert parallelism.
- Understand the mathematics and structure of diffusion models and Vision Transformers.
- Understand pruning, quantization, Flash Attention, KV cache, and speculative decoding (Eagle).
Responsibilities
- Design and deploy model-serving architectures optimized for throughput, latency, and memory use across diverse environments, including resource-constrained devices and edge platforms.
- Build, run, and monitor inference tests in simulated and live production environments, tracking latency, throughput, memory consumption, and error rates.
- Identify and prepare test datasets and simulation scenarios for real-world deployment challenges on low-resource devices.
- Analyze computational efficiency and diagnose serving-pipeline bottlenecks, including batch processing, network delays, and high memory usage.
- Integrate optimized serving and inference frameworks into production pipelines for edge and on-device applications in collaboration with cross-functional teams.
- Define performance metrics and monitor results to support continuous evaluation and refinement of inference systems.
View Full Description & ApplyYou'll be redirected to the employer's site
