JailbreakEval: An Integrated Toolkit for Evaluating Jailbreak Attempts Against Large Language Models

Abstract. We present JailbreakEval, an integrated toolkit designed for evaluating jailbreak attempts against large language models (LLMs). As LLMs become increasingly deployed, understanding their vulnerability to jailbreak attacks that bypass safety alignment is crucial. However, existing evaluation methodologies are often fragmented and lack standardization, making it difficult to compare results across different studies. JailbreakEval provides a unified framework for evaluating jailbreak attacks, including a comprehensive set of evaluation metrics, a library of jailbreak prompts, automated evaluation pipelines, and visualization tools. The toolkit supports various evaluation dimensions, including attack success rate, response harmfulness, and bypass effectiveness. We demonstrate the utility of JailbreakEval through extensive experiments on multiple LLMs, providing insights into the robustness of different safety alignment strategies.