PsySafe: A Comprehensive Framework for Psychological-based Attack, Defense, and Evaluation of Multi-agent System Safety

Zaibin Zhang, Yongting Zhang, Lijun Li, Hongzhi Gao, Lijun Wang, Huchuan Lu, Feng Zhao, Yu Qiao, Jing Shao

About this paper

Multi-agent systems, when enhanced with Large Language Models (LLMs), exhibit profound capabilities in collective intelligence. However, the potential misuse of this intelligence for malicious purposes presents significant risks. To date, comprehensive research on the safety issues associated with multi-agent systems remains limited. In this paper, we explore these concerns through the innovative lens of agent psychology, revealing that the dark psychological states of agents constitute a significant threat to safety. To tackle these concerns, we propose a comprehensive framework (PsySafe) grounded in agent psychology, focusing on three key areas: firstly, identifying how dark personality traits in agents can lead to risky behaviors; secondly, evaluating the safety of multi-agent systems from the psychological and behavioral perspectives, and thirdly, devising effective strategies to mitigate these risks. Our experiments reveal several intriguing phenomena, such as the collective dangerous behaviors among agents, agents' self-reflection when engaging in dangerous behavior, and the correlation between agents' psychological assessments and dangerous behaviors. We anticipate that our framework and observations will provide valuable insights for further research into the safety of multi-agent systems. We will make our data and code publicly accessible at https://github.com/AI4Good24/PsySafe.

Figure 1: Examples of Agents’ Interactions after Psychological-based Attack. After being attacked, the multi-agent system, whether for safe daily tasks or dangerous jailbreak tasks, provides dangerous answers. Agents collaborate with each other to generate dangerous content. Responses identified as dangerous are highlighted in red, whereas safe responses are indicated in green. From Zaibin Zhang et al. (2024), arXiv license.

Figure 2: Overview of PsySafe. ‘Psychology’ denotes the six moral dimensions we adopt. ‘Attack’ refers to our attack methodology, including the construction of attack prompts and the exploration of various angles in attacking multi-agent systems. ‘Agent System’ refers to the prevalent frameworks among current multi-agent systems, comprising hierarchical, joint, and hybrid structures. ‘Defense’ signifies the defensive strategies we propose, encompassing input, doctor, and police defense mechanisms. ‘Evaluation’ represents our evaluation techniques, encompassing psychological evaluation and the identification of joint and process danger conditions. From Zaibin Zhang et al. (2024), arXiv license.

Not packaged. A REPS package would add charts and verified data.How to package an experiment