Applied Data Scientist, Finance AI Evaluation & Datasets
New
I
Innodata Inc.Data Engineering, AI
Remote - United StatesFull-TimeSenior
Salary150,000 - 175,000 USD per year
Apply NowOpens the employer's application page
Job Details
- Experience
- 5+ years of data science experience, with at least 2+ years in financial services, fintech, banking, or a comparable regulated data environment.
- Required Skills
- PythonSQLMachine LearningPyTorchData sciencePandasscikit-learn
Requirements
- 5+ years of data science experience, with at least 2+ years in financial services, fintech, banking, or a regulated data environment.
- Real working knowledge of financial data and workflows including financial statements, SEC filings, and transaction data.
- Hands-on experience with unstructured and multimodal financial data such as PDFs, scanned documents, spreadsheets, charts, and call transcripts.
- Hands-on experience designing datasets for ML, including writing annotation guidelines and setting quality thresholds.
- Strong Python and SQL skills with comfort in libraries like pandas and scikit-learn.
- Working familiarity with Hugging Face, PyTorch, or model APIs.
- Statistical literacy including sampling design, inter-annotator agreement metrics, and confidence intervals.
- Solid grasp of financial services privacy, compliance, and governance (e.g., PII handling, GLBA, MNPI).
- Familiarity with LLM-based and multimodal financial AI workflows such as prompt design, RAG, and LLM-as-judge methods.
- Degree in statistics, data science, economics, finance, or a related quantitative field, or equivalent experience.
Responsibilities
- Translate customer goals into concrete dataset specifications, taxonomies, rubrics, and acceptance criteria.
- Design training and evaluation datasets across financial workflows including financial QA, filings analysis, credit, underwriting, and fraud/AML investigation.
- Develop evaluation methodologies for financial AI systems focusing on numerical consistency, hallucination rates, and robustness.
- Define sampling strategies, label schemas, and adjudication workflows while writing clear annotation guidelines.
- Build statistical and ML tooling for dataset quality, including bias analysis, leakage detection, and distribution shift checks.
- Create reproducible evaluation and dataset-quality evidence to support model risk management and governance.
- Partner with technical teams to instrument datasets into training and monitoring pipelines.
View Full Description & ApplyYou'll be redirected to the employer's site
