Senior Platform Engineer I, AI Evaluation (24 months fixed-term)

Bachelor's or Master's degree in Computer Science, Data Engineering, or equivalent professional experience. 5 years of Software Engineering including significant time working on the evaluation of generative AI systems or other evaluations of ML model quality. Strong programming skills in Go, Python, SQL, and at least one data pipeline framework (e.g., Airflow, Dagster, Prefect). Familiarity with the architecture of large language models and their industry-standard APIs.

Be fluent in offline and online evaluation strategies. Specify eval pipelines using declarative syntax. Understand stratified datasets and ground truth labeling. Appreciate various eval scoring schemes.