Security Test: LLM JailBreak¶

Description¶

Default Severity: high

Basically, jailbreaking is when someone figures out a way to trick a large language model into doing things it shouldn't do. The vulnerability comes from crafty inputs that bypass built-in restrictions, allowing the model to generate harmful or unintended content. This is dangerous because it can lead to misuse—like spreading misinformation or even aiding in cyberattacks—and it undermines the safety measures you rely on when deploying these models. Developers should watch out for ineffective input validations and overly trusting model safeguards, ensuring they thoroughly test against manipulative inputs.

Reference:

• https://genai.owasp.org/llmrisk/llm01-prompt-injection/
  • https://owasp.org/www-project-top-10-for-large-language-model-applications/

Configuration¶

Identifier: injection/llm_jail_break

Examples¶

All configuration available:

checks:
  injection/llm_jail_break:
    skip: false # default

Compliance and Standards¶