Human Data Quality Engineer
New
P
ProlificArtificial Intelligence
Location: MexicoFull-TimeSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years
- Required Skills
- PythonSQLData AnalysisMachine LearningLLM
Requirements
- 5+ years of experience in building quality, evaluation or annotation systems within AI, machine learning, LLMs or human data environments.
- Strong Python and SQL skills, with a passion for using data to solve complex quality problems.
- A solid understanding of machine learning pipelines and how human data impacts model performance.
- Strong analytical and statistical thinking, with experience designing scalable quality frameworks.
- The confidence and credibility to interact with stakeholders at frontier labs and act as a partner.
- The ability to leverage your experience and expertise to influence and guide stakeholders at every level, both client side and internally.
- The ability to turn your own data analysis and quality methodology into requirements that product and engineering can build into systems.
- The ability to explain your data analysis and findings clearly to non-technical stakeholders.
- A proactive, builder's mindset, navigating ambiguity.
Responsibilities
- Design the quality frameworks that underpin complex human data programmes, from evaluation rubrics through to launch readiness.
- Work with clients as a strategic thought partner, challenging annotation schemas and data requirements when they won't produce the signal the model needs.
- Advise on project design and how the choice of schema can impact data quality.
- Build quality upstream across the operational workflow, from recruitment, screening, and training through to writing guidelines and running calibration sessions.
- Build scalable quality systems, measurement frameworks and automated checks using Python and SQL.
- Partner with product and engineering to build the quality infrastructure that delivers high-quality human data at scale.
- Investigate data quality and integrity issues, identifying root causes and turning insights into scalable improvements.
- Architect and build dashboards, monitoring and reporting that provide clear visibility into quality and operational performance.
- Raise the quality capability across the company, upskilling operations and acting as a thought mentor to junior analysts.
- Help define how Prolific approaches quality across new AI domains, shaping best practice as the team grows.
View Full Description & ApplyYou'll be redirected to the employer's site
