OverviewAI Agent Evaluation Analyst - AI Trainer at Mindrift. We connect domain experts with AI projects to ethically shape the future of GenAI. This is a flexible, project-based opportunity that is remote and part-time.ResponsibilitiesReviewing evaluation tasks and scenarios for logic, completeness, and realismIdentifying inconsistencies, missing assumptions, or unclear decision pointsHelping define clear expected behaviors (gold standards) for AI agentsAnnotating cause-effect relationships, reasoning paths, and plausible alternativesThinking through complex systems and policies to ensure agents are tested properlyWorking closely with QA, writers, or developers to suggest refinements or edge case coverageRequirementsExcellent analytical thinking: can reason about complex systems, scenarios, and logical implicationsStrong attention to detail: can spot contradictions, ambiguities, and vague requirementsFamiliarity with structured data formats: can read, not necessarily write JSON/YAMLCan assess scenarios holistically: what’s missing, what’s unrealistic, what might break?Good communication and clear writing (in English) to document findingsBonus qualificationsExperience with policy evaluation, logic puzzles, case studies, or structured scenario designBackground in consulting, academia, olympiads (logic/math/informatics), or researchExposure to LLMs, prompt engineering, or AI-generated contentFamiliarity with QA or test-case thinking (edge cases, failure modes, what could go wrong)Understanding of scoring or evaluation in agent testing (precision, coverage)BenefitsGet paid for your expertise, with rates that can go up to $15/hour depending on your skills and project needsFlexible, remote, freelance project that fits around your commitmentsParticipate in an advanced AI project and build relevant experienceInfluence how future AI models understand and communicate in your fieldSeniority levelInternshipEmployment typePart-timeJob functionOtherIndustriesIT Services and IT Consulting
#J-18808-Ljbffr