Evaluating Large Language Model Safety

SOLUTION

Innodata led a comprehensive safety evaluation of the LLM. Our team leveraged expertise in similar projects to address the key challenges: 

• Tailored Red Teaming Prompts: Innodata’s red team developed thousands of prompts to bypass the LLM’s safety filters and content guards. These prompts covered a comprehensive taxonomy of inappropriate content, including violence, sexual content, and profanity. 
• Age-Specific Voice: The prompts were crafted to mimic the natural language of children across different age groups, from early elementary to high school. This ensured the testing process realistically reflected how children might interact with the LLM. 
• Vulnerability Analysis: Our experts documented and analyzed the “jailbreaking” methods used to bypass the LLM’s safeguards. This analysis pinpointed the areas where the model was most susceptible to manipulation, allowing for targeted improvement efforts. 
• Enhanced Guidelines & Taxonomy: Innodata’s experience in similar projects proved valuable in collaborating with the client to refine the safety guidelines and content taxonomy. This collaborative approach ensured a more comprehensive and effective safety framework for the LLM. 

IMPACT

Through Innodata’s rigorous testing and analysis, the technology company gained a clear picture of the LLM’s weaknesses. This information allowed them to address vulnerabilities and strengthen the safety guardrails before deployment in classrooms. Ultimately, Innodata’s work helped ensure a safer and more appropriate learning environment for children using the LLM.