Language Evaluation Specialist Role
iMerit seeks detail-oriented and analytically minded experts to evaluate the accuracy, quality, and cultural alignment of AI system outputs against complex guidelines. These evaluations will directly inform the development and fine-tuning of advanced language models.
Evaluators will assess the appropriateness of model outputs across multiple modalities (text, image captions, video descriptions, and multimodal prompts) against project-specific criteria such as correctness, coherence, completeness, style, cultural appropriateness, and safety.
* Evaluate outputs generated by large language models (LLMs) for clarity and relevance.
* Assess quality against project-specific criteria such as correctness, coherence, completeness, style, cultural appropriateness, and safety.
* Identify subtle errors, hallucinations, or biases in AI responses.
* Apply domain expertise and logical reasoning to resolve ambiguous or unclear outputs.
* Provide detailed written feedback, tagging, and scoring of outputs to ensure consistency across the evaluation team.
Key Skills & Competencies:
* Strong critical reading, observational, and evaluative skills across different modalities.
* Ability to articulate nuanced judgments with precision and clarity.
* Excellent English comprehension (CEFR B2 or above); additional languages a plus.
* Familiarity with LLMs, generative AI, and multimodal systems.
* Strong attention to detail and ability to apply guidelines consistently.
* Awareness of cultural and linguistic nuances, including potential bias and harm in AI outputs.
* Comfort with evolving workflows, rapid feedback cycles, and complex quality frameworks.
Benefits of Working with Us:
* Ongoing training and professional development opportunities.
* A collaborative and dynamic work environment.
* The chance to contribute to high-impact projects and make a meaningful difference.