Applied Research Scientist, LLM Evaluation & Post-Training
New
I
Innodata Inc.Data Engineering AI
Remote - United StatesFull-TimeSenior
Salary175,000 - 225,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years
- Required Skills
- PythonMachine LearningPyTorchTensorflowLLM
Requirements
- MS/PhD in Computer Science, Machine Learning, Statistics, Applied Mathematics, AI, or a related quantitative scientific field
- 5+ years of relevant experience in applied research / research science in ML/AI
- Substantial work in LLMs or foundation models
- Demonstrated experience with LLM evaluation, benchmarking, alignment, post-training, or model quality research
- Strong foundation in experimental design, statistical analysis, and scientific reasoning for ML systems
- Strong coding skills in Python for research experimentation and analysis
- Experience working with modern ML tooling/frameworks (e.g., PyTorch, Hugging Face, JAX/TensorFlow)
- Ability to evaluate and compare human and automated evaluation methods
- Experience designing evaluation studies and protocols that are reproducible
- Strong communication skills and ability to present nuanced technical conclusions
Responsibilities
- Define and execute a research agenda focused on LLM evaluation and post-training, especially evaluation-driven model improvement
- Design rigorous experiments to study how evaluation methodologies impact fine-tuning and post-training outcomes
- Develop and validate evaluation frameworks for LLM and multimodal systems
- Lead research on advanced evaluation domains, including long-context, cross-modal, and dynamic multi-turn evaluations
- Analyze model behavior and failure patterns; generate actionable recommendations
- Collaborate with AI/ML Research Engineers to translate research methods into scalable pipelines
- Collaborate with Language Data Scientists to integrate human-in-the-loop and synthetic data/evaluation strategies
- Engage with customer technical stakeholders to review methodologies and provide expert recommendations
- Produce high-quality technical documentation and research reports
View Full Description & ApplyYou'll be redirected to the employer's site
