AI Red Teamer & LLM Quality Assurance Specialist
We are seeking highly analytical and detail-oriented professionals with hands-on experience in AI/LLM Quality Assurance, to help us rigorously test and evaluate AI-generated content.
* Conduct in-depth evaluations of large language models (LLMs) to identify potential vulnerabilities, assess risks, and ensure compliance with safety and quality standards.
* Evaluate and stress-test AI prompts across multiple domains to uncover potential failure modes.
* Develop and apply test cases to assess accuracy, bias, toxicity, hallucinations, and misuse potential in AI-generated responses.
* Collaborate with data scientists, safety researchers, and prompt engineers to report risks and suggest mitigations.
* Perform manual QA and content validation across model versions, ensuring factual consistency, coherence, and guideline adherence.
* Create evaluation frameworks and scoring rubrics for prompt performance and safety compliance.
* Document findings, edge cases, and vulnerability reports with high clarity and structure.
Requirements:
* Proven experience in AI red teaming, LLM safety testing, or adversarial prompt design.
* Familiarity with prompt engineering, NLP tasks, and ethical considerations in generative AI.
* Strong background in Quality Assurance, content review, or test case development for AI/ML systems.
* Understanding of LLM behaviors, failure modes, and model evaluation metrics.
* Excellent critical thinking, pattern recognition, and analytical writing skills.
* Ability to work independently, follow detailed evaluation protocols, and meet tight deadlines.
-----------------------------------
About the Job:
This is an exciting opportunity to join our organization as a key member of our AI/ML quality assurance team. You will be working closely with our data scientists, safety researchers, and prompt engineers to develop and implement comprehensive evaluation frameworks for our large language models.
The ideal candidate will have a strong background in AI/ML quality assurance, excellent analytical and writing skills, and the ability to work independently in a fast-paced environment.
We offer a dynamic and collaborative work environment, opportunities for growth and professional development, and a competitive compensation package.