# govsim: replication runs from the Multi-Agent Databank

Five LLM agents share a common-pool resource and must agree on limits to avoid collapse. A replication of Piatti et al. (NeurIPS 2024) across three scenarios.

- Page: https://databank.antimemetics.institute/papers/govsim
- Data tab: https://databank.antimemetics.institute/papers/govsim/data
- Original paper: Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents (Giorgio Piatti, Zhijing Jin, Max Kleiman-Weiner, Bernhard Schölkopf, Mrinmaya Sachan, Rada Mihalcea, 2024): https://arxiv.org/abs/2404.16698
- Source: https://github.com/giorgiopiatti/GovSim
- Index built: 2026-10-19T13:41:40.501101Z

## Files

The databank hosts the index and the files below. Each run's own files live in its deposit's public store, at the addresses in runs.csv.

- https://databank.antimemetics.institute/data/govsim/runs.csv: one row per published run, newest first: run, condition, package, state, timing, seed, store, the addresses of its run.json and events.jsonl.zst, the packaging revision it was built from, then every parameter and every result as its own column.
- https://databank.antimemetics.institute/data/govsim/urls.txt: every run object in the store, one address per line (run.json and events.jsonl.zst for each run), in condition order.
- https://databank.antimemetics.institute/index/experiments/govsim/index.jsonl: the run index the site reads: one JSON row per run with its store and card (identity, parameters, lifecycle, results, usage).
- https://databank.antimemetics.institute/index/catalog.json: the experiment manifest, with the parameter and result definitions below.

## Runs

785 runs across 10 models in 153 conditions (a condition is one full parameter set; its id names the run's folder in the store). States: completed.

Runs by experiment and model:

| experiment | azure/DeepSeek-V4-Flash | azure/DeepSeek-V4-Pro | azure/gpt-5.6-sol | azure/gpt-5.6-terra | azure/gpt-6-astra | azure/grok-4.6 | azure/mistral-medium-3-5 | vllm/Qwen/Qwen1.5-110B-Chat-GPTQ-Int4 | vllm/Qwen/Qwen1.5-72B-Chat-GPTQ-Int4 | vllm/mistralai/Mistral-7B-Instruct-v0.2 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| fish_baseline_concurrent | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| fish_baseline_concurrent_universalization | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| fish_perturbation_no_language | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| fish_perturbation_outsider | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| fish_perturbation_outsider_universalization | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| pollution_baseline_concurrent | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| pollution_baseline_concurrent_universalization | 10 | 10 | 10 | 10 | 10 | 10 | 10 | 5 | 5 | 5 |
| pollution_perturbation_no_language | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| pollution_perturbation_outsider | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| pollution_perturbation_outsider_universalization | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| sheep_baseline_concurrent | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| sheep_baseline_concurrent_universalization | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| sheep_perturbation_no_language | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| sheep_perturbation_outsider | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |
| sheep_perturbation_outsider_universalization | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 |

## Run objects

Each run lives at `<store>/runs/<condition>-govsim/<run>/`:

- `run.json`: the run's card as published: identity, inputs (parameters, seed), lifecycle, provenance, derived results and usage.
- `events.jsonl.zst`: the full event stream, zstd-compressed JSONL, one event per line in sequence order.

Fetch everything, keeping the store's layout:

```sh
curl -sO https://databank.antimemetics.institute/data/govsim/urls.txt
wget --quiet --force-directories --input-file urls.txt
```

One condition only:

```sh
grep '/runs/<condition>-govsim/' urls.txt | wget --quiet --force-directories --input-file -
```

Read a stream:

```sh
zstd -dc <run>/events.jsonl.zst | head
```

## Run it

One command is one run, with the REPS runner (https://github.com/antimemes/REPS; its README covers installing Nix and storing provider credentials). Every parameter is bound on the command line. The values below are the manifest's initial values, not a published run's.

```sh
$(nix-build --no-out-link --tarball-ttl 0 https://github.com/antimemes/REPS/archive/main.tar.gz \
  -A exec.govsim) \
  --set experiment=fish_baseline_concurrent \
  --set max_rounds=0 \
  --set model=mock/model \
  --set embedder=hash \
  --set temperature=0 \
  --set top_p=1 \
  --set max_tokens=null \
  --set reasoning_effort=low \
  --set threads=2
```

Check without executing: add `--dry-run` to the experiment arguments; it prints the condition id and seed. Print the manifest with `--describe`. Credentials: `reps-runner credentials --help`.

To reproduce a published run, take its row in runs.csv: bind each parameter column with `--set name=value`, pass `--seed` with its seed, and build from the revision in its source_ref column in place of main.tar.gz; a reference `github:<owner>/<repo>/<commit>` is the archive `https://github.com/<owner>/<repo>/archive/<commit>.tar.gz`. The model's replies can still vary, so a reproduction shares the published run's condition, not its transcript.

Revisions the published runs were built from:

- github:antimemetics-institute/agentdatabank/8d7a313808a170638f68844a3561086b4fb0b56d (285 runs): https://github.com/antimemetics-institute/agentdatabank/archive/8d7a313808a170638f68844a3561086b4fb0b56d.tar.gz
- github:antimemes/REPS/65980356082491b178083e00e9831fc61c05b3bf (225 runs): https://github.com/antimemes/REPS/archive/65980356082491b178083e00e9831fc61c05b3bf.tar.gz
- github:antimemetics-institute/agentdatabank/3c1af4f280188a503ca482c594008b4609035295 (206 runs): https://github.com/antimemetics-institute/agentdatabank/archive/3c1af4f280188a503ca482c594008b4609035295.tar.gz
- github:antimemetics-institute/agentdatabank/af0fcb2f8c16189835de25accbaf31f240d707d1 (65 runs): https://github.com/antimemetics-institute/agentdatabank/archive/af0fcb2f8c16189835de25accbaf31f240d707d1.tar.gz
- github:antimemetics-institute/agentdatabank/76a74cd063c59ad2c7c8a3b4f294471356ed9e84 (4 runs): https://github.com/antimemetics-institute/agentdatabank/archive/76a74cd063c59ad2c7c8a3b4f294471356ed9e84.tar.gz

## Parameters

- `embedder` (enum): Memory embeddings: hash gives artificial similarities for offline testing; mxbai uses the paper’s embedding model at the revision pinned in the adapter and may download about 1.3 GB.
- `experiment` (enum): Scenario and treatment to run.
- `max_rounds` (int): Round (month) cap; 0 = the upstream per-experiment value (12 baseline, 15 outsider).
- `max_tokens` (int): Per-call completion budget sent as a cap on every request; null sends none, so the server applies its own limit (upstream asked for 8000, which never bound in practice).
- `model` (llm): Model used by every agent. Use mock/model for an offline test.
- `reasoning_effort` (enum): Reasoning effort for compatible models; null leaves the provider default. For Astra use low and set temperature/top_p to null.
- `temperature` (float): Sampling temperature; null omits it from requests (upstream default 0.0).
- `threads` (int): CPU threads per process for PyTorch and numeric libraries.
- `top_p` (float): Nucleus sampling top_p; null omits it from requests (upstream default 1.0).

## Results

- `rounds` (Rounds simulated, months): Months reached by the simulation. Last recorded harvest round plus one, since round numbering starts at zero. Zero if no harvesting was recorded. Actual progress may be shorter than the configured run length.
- `collapsed` (Resource collapsed): Whether the shared resource was recorded as depleted. Collapse means a round's recorded post-harvest resource pool fell below 5 or was missing (NaN). The calculation uses the last non-missing pool value in each round, or NaN if all are missing. No also covers runs with no recorded harvesting. Collapse is a simulation outcome, not an execution failure.
- `survival_months` (Survival duration, months): Month of collapse, or the configured run length if none was recorded. Uses the first collapse round plus one, or the configured run length if no collapse was recorded, following the paper's formula. This can exceed actual recorded progress. Zero if no harvesting was recorded.
- `total_harvest` (Total resource collected, resource units): Resource collected by all agents over the whole run. Sum of resource_collected across all logged harvest actions, agents and rounds. Resource meaning depends on the fish, sheep or pollution scenario.
- `gain_per_agent` (Mean gain per agent, resource units per agent): Average resource collected per harvesting agent over the whole run. Average total resource collected per agent over the whole run, not per month. Includes only agents with recorded harvesting activity. This is the paper's total gain metric.
- `final_resource` (Final resource pool, resource units): Resource left after the last recorded harvest. Uses the last non-missing post-harvest pool value in the final recorded harvest round, before any subsequent regeneration. Zero if no harvest actions were logged.
- `equality` (Harvest equality): How evenly agents shared the collected resource; higher is more equal. One minus the Gini coefficient of total resource collected per logged harvester; higher values mean more equal gains. All-zero gains score 1, so equality alone does not establish productive cooperation. No harvest actions yields 0.
- `over_usage` (Requests above sustainable share): Share of harvest requests above the estimated sustainable allowance. Fraction of harvest actions requesting more than (pre-harvest pool // 2) // distinct harvesters in that round; // is integer division. This adapter reconstruction counts requests, not allocations, and can differ from upstream's fixed agent count and reset-time threshold, especially when outsiders join. No harvest actions yields 0.

## Deposits

Who published the runs and under what terms. Excluded runs are left out of the index, runs.csv and urls.txt; their records remain in the store.

### Raphael Francis Ltd

- Runs: 785
- License: CC BY 4.0 (https://creativecommons.org/licenses/by/4.0/)
- Page: https://huggingface.co/buckets/raphaelfrancis/govsim
- Objects resolve under: https://huggingface.co/buckets/raphaelfrancis/govsim/resolve
- Excluded from the index:
  - 4 runs (20260918t011923z-3bcb1a4e712e, 20260917t220911z-f6342fea4f0a, 20260917t204313z-30d2f1fcc315, 20260917t182820z-6e4a0ea46603): Azure grok-4.6 returned HTTP 200 responses with empty choices during the 2026-09-18 outages (03:04:49-03:07:30Z and 00:17:58-00:19:27Z) and scattered failures. Upstream GovSim replaced these responses with default values, including harvest decisions defaulted to 0 tons. All four affected published runs are excluded. Original records remain intact in the bucket; replacements were published on 2026-10-01.
  - 35 runs (20260918t112712z-2c04a6d5ab0c, 20260918t121614z-a3a35a6eb7fb, 20260918t131134z-de3be6d7f65c, 20260918t140733z-593aea66d8fb, 20260918t151830z-0be04ba8e6ba, 20260918t114214z-68d5307b8f61, 20260918t123350z-e307fdfee435, 20260918t132524z-b391c6cd7492, 20260918t142620z-13ec496def38, 20260918t153741z-1f8ce5430544, 20260918t151621z-bf8e9ed770fa, 20260918t162151z-b20e69bcc0b0, 20260918t172411z-2e9fa92dad64, 20260918t172855z-706c7881b630, 20260918t173256z-6657f6b78258, 20260918t152222z-ef53d6ba6c1d, 20260918t162705z-9a1506bcc11a, 20260918t172605z-8b47f5a73618, 20260918t182327z-20f369d0b49c, 20260918t192015z-3ae092633a59, 20260918t201648z-2b5e99832b2c, 20260918t211110z-108c4f8fa1b1, 20260918t220748z-1916b5443ae7, 20260918t225108z-8207ef24e722, 20260918t232832z-f320e0ba8f64, 20260918t125225z-422202abde3d, 20260918t130217z-8fd061dcf956, 20260918t133538z-858c269d3c52, 20260918t143654z-63235f4366f5, 20260918t151455z-17624a4d5f0f, 20260918t130708z-0a2c245984b9, 20260918t140531z-67b1167e978c, 20260918t152017z-f457af4c8960, 20260918t162414z-247253f2c59e, 20260918t172141z-c886e108d6c6): Superseded original tier-1 azure/mistral-medium-3-5 runs, rerun with the same seeds and parameters after the assistant-prefill fix added message-level prefix: true. The 35 replacement runs were published and verified on 2026-09-25. Original records remain intact in the bucket; the site uses the replacements.

## Citation

Cite the replication data and the original paper.

```bibtex
@misc{raphael-francis-ltd-2026-govsim-replication-dataset,
  author = {{Raphael Francis Ltd}},
  title = {govsim: replication dataset},
  year = {2026},
  howpublished = {Multi-Agent Databank},
  url = {https://databank.antimemetics.institute/papers/govsim/data},
  note = {Index built 2026-10-19}
}
```

```bibtex
@inproceedings{NEURIPS2024_ca9567d8,
  author = {Piatti, Giorgio and Jin, Zhijing and Kleiman-Weiner, Max and Sch\"{o}lkopf, Bernhard and Sachan, Mrinmaya and Mihalcea, Rada},
  booktitle = {Advances in Neural Information Processing Systems},
  doi = {10.52202/079017-3548},
  editor = {A. Globerson and L. Mackey and D. Belgrave and A. Fan and U. Paquet and J. Tomczak and C. Zhang},
  pages = {111715--111759},
  publisher = {Curran Associates, Inc.},
  title = {Cooperate or Collapse: Emergence of Sustainable Cooperation in a Society of LLM Agents},
  url = {https://proceedings.neurips.cc/paper_files/paper/2024/file/ca9567d8ef6b2ea2da0d7eed57b933ee-Paper-Conference.pdf},
  volume = {37},
  year = {2024}
}
```

## About the experiment

Can AI agents share a resource without exhausting it?

Five agents benefit from taking a shared resource, but overuse threatens
everyone’s future. GovSim tests whether they can agree on limits and follow them.

Based on [Cooperate or Collapse](https://arxiv.org/abs/2404.16698)
(Piatti et al., NeurIPS 2024). [Original code](https://github.com/giorgiopiatti/GovSim).

## How it works

Each round represents a month: agents harvest, observe each other’s actions,
discuss limits, and reflect. The resource can recover between rounds if enough
remains. Memories of earlier actions and conversations inform later decisions.

## Scenarios and conditions

The **scenario** changes the agents’ roles, decisions, and shared resource:

| Scenario | What each agent decides each month | Shared resource |
| --- | --- | --- |
| Fishing (fishery) | A fisher chooses how many tons of fish to catch. | A lake holding up to 100 tons of fish; each ton caught removes one ton. |
| Sheep herding (pasture) | A shepherd chooses how many sheep to graze. | Up to 100 hectares of grass; each sheep consumes one hectare. |
| Pollution | A factory owner chooses how many pallets of widgets to produce. | A river with up to 100% clean water; each pallet pollutes one percentage point. |

All three use the **same resource dynamics**: after each month, the remaining
fish, grass, or clean water doubles, up to its starting capacity. At full capacity,
the group can use 50 units per month without reducing next month’s supply—10 each
for five agents. The scenarios test whether the same cooperation problem elicits
different behavior when framed as extraction, grazing, or pollution
([paper, §§2.2–2.3](https://arxiv.org/html/2404.16698v4#S2.SS2)).

The **treatment** changes the agents’ instructions, information, or group membership:

| Treatment | What changes from baseline | What it tests |
| --- | --- | --- |
| Baseline | Five agents choose their own harvests, see others’ harvests, and discuss limits. | Can cooperation emerge? |
| Universalization | Adds a reminder that the resource shrinks if everyone exceeds the sustainable share. | Does guidance about collective consequences improve cooperation? |
| No discussion | Removes group conversation and observations of others’ harvests. | Can cooperation persist with less social information? |
| Outsider | Starts with four community-minded agents; a selfish newcomer joins after three months. | Can the group withstand a newcomer who disregards its norms? |

Each **recorded condition** also fixes the model and run settings.

### Additional variants in the upstream code

**Paraphrase 1 and 2** change the wording of the fishing baseline’s instructions,
while retaining its rules. They are optional upstream variants, not listed in
the authors’ [table of paper experiments](https://github.com/giorgiopiatti/GovSim#table-of-experiments).
Their configurations select the
[alternative fishing prompts](https://github.com/giorgiopiatti/GovSim/blob/1d11adf047b24fa2ba0d44a1d4931015ea2e5210/simulation/scenarios/fishing/agents/persona_v3/cognition/utils.py).
The code also supports combining an outsider with universalization.

## Run settings

- **Quick demonstration:** one round with `mock/model` and the `hash` embedder.
  These use scripted replies and artificial memory similarities.
- **Model behavior:** use a real model and `mxbai` for memory retrieval.
- **Full scenario:** set `max_rounds = 0`; normally 12 months, or 15 with outsiders.

**Resource collapse is an outcome, not an execution failure.** A short run
without collapse does not demonstrate lasting cooperation. Resource units differ
across scenarios; cumulative harvest also depends on run length.

### Reproducibility and recorded data

These runs are not automatically reproductions of the paper. Inspect each
recorded condition for its model, generation settings, embedder, and seeds.
The `mxbai` model revision is pinned in the adapter. A seed patch makes resource
allocation repeatable for the same seed and actions; hosted-model replies may
still vary.

`over_usage` measures requests above an agent’s sustainable share. Its calculation
uses the current number of harvesters, which differs from the upstream
simulation’s fixed count when an outsider joins.

Model calls and resource updates appear during execution; conversations and
memories are imported afterward. Imported timestamps record ingestion time,
not when agents acted. The Mayor’s messages are templated, with no model call.

The runner saves parameters, seeds, source references, and build information in
`run.json` and `run.start`. GovSim records its resolved configuration in
`govsim.config`, resource history in `govsim.state.resource`, and simulation rows
as `govsim.harvest`, `govsim.utterance`, `govsim.summary`, or
`govsim.resource_limit`. Other actions remain `govsim.record`.

Each upstream log has a `govsim.upstream_log` marker with its path, byte count,
SHA-256 hash, and ingested record count. Memory nodes become `govsim.memory`;
embeddings are not imported. Original fields and interaction HTML are retained
on disk; the viewer displays plain text. Available checkpoints are retained on
failure, with unreadable or missing logs reported as `govsim.unparsed_log`.