AI Evaluation Specialist
Evaluate highly nuanced AI system outputs across different modalities: text, image, video, and multimodal interactions.
* Evaluate accuracy, appropriateness, quality, clarity, and cultural alignment of model outputs against complex guidelines.
* Assess quality against project-specific criteria such as correctness, coherence, completeness, style, cultural appropriateness, and safety.
* Identify subtle errors, hallucinations, or biases in AI responses.
Main Responsibilities:
* Evaluate LLMs across multiple modalities (text, image captions, video descriptions, and multimodal prompts).
* Assess quality against project-specific criteria.
* Provide detailed written feedback, tagging, and scoring of outputs to ensure consistency.
* Escalate unclear cases and contribute to refining evaluation guidelines.
* Collaborate with Project Managers and Quality Leads to meet accuracy, reliability, and turnaround benchmarks.
Required Skills & Qualifications:
* Bachelor's degree/ diploma or equivalent educational qualification.
* 1+ years of experience in data annotation, LLM evaluation, content moderation, or related AI/ML domains.
* Demonstrated experience working with data annotation tools and software platforms.