We're seeking meticulous professionals to rigorously scrutinize and evaluate AI-generated content, identifying vulnerabilities and ensuring adherence to safety and quality standards.
Key Responsibilities:
* Conduct thorough red teaming exercises to identify potential threats from large language models.
* Evaluate and stress-test AI prompts across various domains to ensure accuracy and reliability.
* Develop and apply comprehensive test cases to assess the integrity of AI-generated responses, including bias, toxicity, hallucinations, and misuse potential.
* Collaborate with data scientists and safety researchers to report risks and suggest mitigation strategies.
Requirements:
* Proven experience in AI red teaming or LLM safety testing, with a strong background in Quality Assurance or test case development for AI/ML systems.
* Familiarity with prompt engineering and ethical considerations in generative AI, with a focus on responsible innovation.
* A robust understanding of the importance of accuracy, fairness, and transparency in AI decision-making processes.
Deliverables:
* A detailed report outlining identified vulnerabilities and recommended mitigations.
* A set of optimized test cases for evaluating AI-generated content.