Python Engineer - AI Data & Evaluation
New
G
Gramian ConsultancyIT Professional Services
Brazil. Argentina. Colombia. Chile. Uruguay. Mexico, 4 hours of overlap with PST working hoursContractSenior
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- At least 5 years of professional experience
- Required Skills
- PythonMachine LearningTypeScript
Requirements
- At least 5 years of professional experience with Python or TypeScript.
- Hands-on experience building or maintaining pipelines and infrastructure using Python, or contributing to large TypeScript codebases.
- Practical experience with preference data collection, review, or rubric creation.
- Experience designing or applying evaluation criteria for AI-generated content or technical outputs.
- Strong written and verbal communication skills.
- Ability to explain and justify quality judgments using explicit evaluation criteria.
- Experience working in Tier 1 companies.
- Experience navigating or contributing to large codebases (nice-to-have).
- Prior experience working on AI, machine learning, data quality, or evaluation projects (nice-to-have).
Responsibilities
- Design and refine evaluation rubrics and criteria for preference data.
- Review and label data while identifying subtle quality issues and inconsistencies.
- Build and maintain Python or TypeScript pipelines and infrastructure supporting data generation, collection, and evaluation.
- Develop processes that translate qualitative assessments into scalable evaluation workflows.
- Synthesize findings from data reviews into clear technical write-ups, updates, and recommendations.
- Collaborate with researchers and engineers to improve evaluation methodologies and data quality.
- Iterate on evaluation processes based on observed issues and project requirements.
- Communicate technical findings in a structured, concise, and actionable format.
View Full Description & ApplyYou'll be redirected to the employer's site
