We are seeking skilled professionals to test and evaluate AI-generated content for vulnerabilities, risks, and compliance with safety, ethical, and quality standards.
Key Responsibilities:
1. Conduct thorough security assessments of large language models (LLMs) to identify potential weaknesses.
2. Develop and execute test cases to evaluate the accuracy, bias, toxicity, hallucinations, and misuse potential in AI-generated responses.
3. Collaborate with data scientists and researchers to analyze results, report risks, and suggest mitigations.
4. Perform manual quality assurance and content validation across model versions to ensure factual consistency and guideline adherence.
5. Design evaluation frameworks and scoring rubrics for prompt performance and safety compliance.
6. Document findings, edge cases, and vulnerability reports clearly and concisely.
Requirements: Proven experience in AI security testing, LLM risk assessment, or adversarial prompt design. Familiarity with prompt engineering, NLP tasks, and ethical considerations in generative AI is essential.
Preferred Qualifications: Prior work with LLM safety initiatives, risk management, security testing, or AI governance is highly valued. Background in linguistics, psychology, or computational ethics is a plus.