Evaluations Team Lead

New

JobgetherEnterprise AI

Based in GermanyFull-TimeLead

SalaryCompetitive compensation package combining salary and equity.

Apply NowOpens the employer's application page

Job Details

Have proven experience owning evaluation of tabular machine learning systems used in production or high-impact customer decision-making.
Bring strong statistical expertise, including metric and validation-scheme selection, uncertainty estimation, model comparisons across datasets, and accounting for repeated experimentation.
Have practical experience identifying and preventing data leakage involving preprocessing, feature engineering, joins, temporal dependencies, and related entities across data splits.
Be proficient in Python and SQL, and familiar with scikit-learn, gradient-boosted decision trees, and reliable ML tools or platforms used by multiple teams.
Have experience designing fair, reproducible model comparisons, including hyperparameter optimization, resource allocation, and end-to-end inference latency measurement.
Have people management experience, including hiring, technical mentoring, team development, and performance management, and remain actively involved in technical work.
Communicate effectively with researchers, software engineers, and customer data scientists.
Be able to challenge unsupported conclusions, communicate uncertainty, and ground technical decisions in reliable evidence.
Experience evaluating tabular foundation models or AutoML systems is an advantage.
Familiarity with relational or multi-table datasets and data platforms such as Snowflake or Databricks is a plus.
Knowledge of automated or agent-driven machine learning workflows is beneficial.

Build and maintain a shared evaluation platform for research, engineering, and Applied AI teams.
Develop benchmark suites, curated datasets, and model comparisons for business use cases.
Establish evaluation standards for statistical metrics, validation, data splits, leakage prevention, calibration, uncertainty, and benchmark contamination.
Design reproducible evaluation pipelines with version-controlled inputs, datasets, and artifacts, and integrate automated regression checks.
Benchmark models using fair protocols for hyperparameter tuning, data access, computational resources, and end-to-end latency.
Evaluate predictive performance, inference latency, and cost efficiency across deployment configurations, tasks, and datasets.
Support customer proof-of-concept evaluations with tooling, methodological guidance, and data-driven analysis.
Translate evaluation findings into research priorities, release recommendations, and improvement plans.
Recruit, lead, and develop evaluation specialists through priority-setting, technical coaching, and performance feedback.
Contribute hands-on Python development, experimental design, statistical analysis, and evaluation methodology.

View Full Description & ApplyYou'll be redirected to the employer's site