Applied Data Scientist, Health AI Evaluation & Datasets
New
I
Innodata Inc.Health AI
Remote - United StatesFull-TimeMiddle
Salary150,000 - 175,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years
- Required Skills
- PythonSQLMachine LearningPandasscikit-learn
Requirements
- 5+ years of data science experience.
- 2+ years of experience with healthcare, clinical, biomedical, payer, provider, pharma, life sciences, or comparable regulated health data.
- Working knowledge of healthcare data and standards: EHR structure, clinical documentation conventions, ICD-10, CPT, SNOMED CT, LOINC, and RxNorm.
- Hands-on experience designing ML datasets, including writing annotation guidelines and setting quality thresholds.
- Familiarity with LLM-based health AI workflows, including prompt design, RAG, and LLM-as-judge methods.
- Strong Python and SQL programming skills.
- Experience with data science libraries: pandas, scikit-learn, statsmodels.
- Familiarity with modern LLM tooling such as Hugging Face and evaluation frameworks.
- Statistical literacy across sampling design, bias/fairness analysis, and significance testing.
- Solid grasp of healthcare privacy and governance (HIPAA, de-identification standards).
- Relevant degree such as biostatistics, epidemiology, computational biology, health informatics, or computer science.
Responsibilities
- Translate customer goals into detailed dataset specifications, taxonomies, rubrics, and sampling plans.
- Design training and evaluation datasets across multimodal health data including clinical text, images, EHR records, and medical literature.
- Develop evaluation strategies for RAG and source-grounded health AI, focusing on faithfulness and guideline adherence.
- Define sampling strategies, label schemas, and quality thresholds in partnership with SMEs and technical teams.
- Build statistical and ML checks to ensure data trustworthiness, including bias analysis and leakage detection.
- Instrument datasets into evaluation and post-training pipelines such as LLM-as-judge prompts and regression suites.
- Evaluate health AI behavior regarding calibration, hallucination, and equity in clinical workflows.
- Manage data quality from intake to delivery while maintaining compliance with HIPAA and de-identification standards.
View Full Description & ApplyYou'll be redirected to the employer's site
