Quick Concepts: Jailbreaking

What is Jailbreaking in Generative AI?

Jailbreaking is a form of hacking that aims to bypass an AI model’s ethical safeguards and elicit prohibited information. It uses creative prompts in plain language to trick generative AI systems into releasing information that would otherwise be blocked by their content filters.

So far, the most popular methods of jailbreaking have been to ask the AI to assume a different identity such as a fictional character or another chatbot with fewer restrictions. The subsequent prompts may include elaborate storylines or games (sometimes involving language translation, fragments of code, etc.) in which the AI is gradually coaxed into chatting about illegal acts, hateful content, or misinformation.

Purposes and Consequences of Jailbreaking

Presently, although AI model testers do conduct jailbreaks to identify and patch vulnerabilities (this is known as red-teaming), the majority of jailbreaks are conducted by users as a pastime or personal challenge. Companies such as OpenAI attempt to fight jailbreaks as they occur, but due to the volume and creativity of jailbreak efforts, this is an uphill battle.

Jailbreaking can create serious, even disastrous problems for generative AI users as the technology is adopted more widely and extensively. If everyday users are able to exploit vulnerabilities and extract safeguarded data or change a model’s behavior, cybercriminals could as easily hack into generative AI systems to access sensitive data, spread viruses, or run misinformation campaigns.

How Can Jailbreaking be Prevented or Reduced?

Most companies currently use red-teaming as a first line of prevention and defense against jailbreaking. In red-teaming, a team of data scientists is tasked with attempting to hack into generative AI systems to find and fix vulnerabilities during model development and pre-release. After the model is released, additional teams monitor and address issues as they emerge.

However, these measures are not enough. Jailbreaking attempts by users have only increased in frequency and efficacy. Since LLMs are incompletely understood, their ethical safeguards merely alter their surface behaviors, leaving intact the base models, which have access to most of the harmful content on the internet. Therefore, users are able to get past safeguards merely by changing their prompts and pushing into deeper layers of the model. Until developers are able to tune base models at a deeper level or establish more intelligent safeguards, jailbreaking is likely to proliferate.

Bring Intelligence to Your Enterprise Processes with Generative AI

Whether you have existing generative AI models or want to integrate them into your operations, we offer a comprehensive suite of services to unlock their full potential.