Senior Evaluation Algorithm Engineer
New
B
BinanceBlockchain AI
Hong Kong / Taiwan, Taipei / Australia, SydneyFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Required Skills
- PythonArtificial IntelligenceData scienceLLM
Requirements
- Master's degree or above in Computer Science, Artificial Intelligence, Mathematics, Statistics, or related fields.
- Solid algorithmic foundation and understanding of LLM principles, training, and fine-tuning.
- Hands-on LLM evaluation experience at a large tech company with commercial deployment evaluation.
- Familiarity with full evaluation pipeline from data preparation and rubric design to R&D implementation.
- Proficiency with mainstream evaluation methods (human evaluation, LLM-as-a-judge, metric computation).
- Experience ensuring evaluation data representativeness, annotation consistency, and result reliability.
- Proficient in Python for data processing, evaluation script writing, and platform development.
- Strong business understanding and communication skills to drive cross-team collaboration.
Responsibilities
- Design end-to-end LLM evaluation plans for business scenarios such as dialogue and financial trading.
- Build evaluation metric systems and rubrics to quantify model performance.
- Lead the design and construction of evaluation datasets and quality control processes.
- Analyze model capability boundaries and failure modes to produce actionable improvement recommendations.
- Drive the automation and scaling of evaluation workflows through platform development.
- Collaborate with algorithm, product, and data teams to translate business objectives into evaluation standards.
View Full Description & ApplyYou'll be redirected to the employer's site
