Testing Lead-QA
New
IndiaFull-TimeLead
Salary not disclosed
Apply NowOpens the employer's application page
Job Details
- Experience
- 3–4 years
- Required Skills
- PythonMachine LearningMLFlowCI/CDLLMMLOpsGenerative AILangChain
Requirements
- 3–4 years of experience in software testing, with significant ownership or leadership experience in AI, Deep Learning, LLM, or Generative AI testing environments.
- Strong hands-on experience testing non-deterministic AI systems, machine learning models, and advanced language models.
- Excellent Python programming skills with experience in test automation, data validation, and quality engineering frameworks.
- Strong understanding of transformer architectures, deep learning workflows, and model evaluation methodologies.
- Proven ability to create clear, structured, and maintainable technical documentation.
- Experience developing testing strategies for prompt engineering, conversational AI systems, and retrieval-augmented generation (RAG) architectures.
- Familiarity with CI/CD pipelines, MLOps practices, and production monitoring for AI systems.
- Strong analytical and problem-solving skills with the ability to work effectively in fast-paced and ambiguous startup environments.
- Excellent communication and stakeholder management skills.
- Experience with Vision-Language Models, multimodal AI systems, or computer vision technologies is highly desirable.
- Familiarity with tools such as LangChain, LlamaIndex, MLflow, vector databases, and embedding technologies is considered a strong advantage.
Responsibilities
- Define and lead end-to-end testing strategies for Deep Learning, LLM, and VLM products and pipelines.
- Establish testing frameworks covering model evaluation, acceptance criteria, release readiness, and risk assessment.
- Create and maintain comprehensive documentation related to testing methodologies, model assumptions, known limitations, and quality sign-offs.
- Design and execute testing strategies for prompt engineering, RAG pipelines, hallucination control, multi-turn conversations, and long-context model behavior.
- Develop and manage golden datasets, regression testing suites, and benchmarking processes.
- Evaluate multimodal and vision-language systems, including image-text alignment, OCR, captioning, and reasoning capabilities.
- Build Python-based automation frameworks for model evaluation, validation, and regression testing.
- Integrate testing processes into CI/CD and MLOps pipelines to support continuous delivery and model monitoring.
- Generate quality reports, dashboards, and actionable insights for engineering and leadership teams.
- Monitor production performance, identify model drift or degradation, and document behavioral changes across model versions.
View Full Description & ApplyYou'll be redirected to the employer's site
