MultiAgent Collaboration Attack: Investigating Adversarial Attacks in Large Language Model Collaborations via Debate

Alfonso Amayuelas, Xianjun Yang, Antonis Antoniades, Wenyue Hua, Liangming Pan, William Wang

About this paper

Large Language Models (LLMs) have shown exceptional results on current benchmarks when working individually. The advancement in their capabilities, along with a reduction in parameter size and inference times, has facilitated the use of these models as agents, enabling interactions among multiple models to execute complex tasks. Such collaborations offer several advantages, including the use of specialized models (e.g. coding), improved confidence through multiple computations, and enhanced divergent thinking, leading to more diverse outputs. Thus, the collaborative use of language models is expected to grow significantly in the coming years. In this work, we evaluate the behavior of a network of models collaborating through debate under the influence of an adversary. We introduce pertinent metrics to assess the adversary's effectiveness, focusing on system accuracy and model agreement. Our findings highlight the importance of a model's persuasive ability in influencing others. Additionally, we explore inference-time methods to generate more compelling arguments and evaluate the potential of prompt-based mitigation as a defensive strategy.

Figure 1: Agent collaboration can be vulnerable to adversarial attacks. Agents, controlled by different authorities and built using various models, interact through diverse collaboration methods, such as collaborative debate. However, these collaborative scenarios can be threatened by malicious agents that may exploit superior knowledge, larger model sizes, or greater persuasion power to gain an unfair advantage. From Alfonso Amayuelas et al. (2024), CC BY 4.0.

Figure 2: Sample Debate (from MMLU). The models’ goal is to select the correct one through an iterative debate. Debate: Initially, each model independently answers the question. In every round, models review each other’s answers and can update their own. Adversary: The adversary is given a wrong answer and attempts to convince the other models it is correct, succeeding in this example. A detailed version of this example is provided in Appendix A. From Alfonso Amayuelas et al. (2024), CC BY 4.0.

Not packaged. A REPS package would add charts and verified data.How to package an experiment