← all dialogues

How this was made

The method behind the dialogues, and what to keep in mind when reading them.

The questions

Every dialogue starts from a question in the 2020 PhilPapers Survey, which asked professional philosophers where they stand on the central questions of the field. The models get the same question and the same options the philosophers saw.

Three stages

1. Alone. Each model answers the question on its own, five times, in a fixed format: one option, a confidence from 0 to 100, and a short reason. The order of the options is shuffled each time.
2. Together. Three models from different companies discuss the question for three rounds, in a random speaking order, a few sentences per turn. No one is given a persona or a side to argue.
3. Alone again. Each model answers five more times in the same format, with the whole transcript in front of it.

The prompt

Each participant gets this system prompt, with its own name and the question filled in:

You are {me}, an AI model, in a discussion with two other AI models, {others}, on this question:

"{question}"

The options are: {options}.

The discussion has {rounds} rounds, and in each round every participant speaks once. The other participants' turns appear as messages beginning with their name. Write only your own turn: don't start it with your name, and never write lines for the other participants. Give your own view and respond to what the others have said. Keep each turn to 2-4 sentences of plain prose.

One closing line varies, and each dialogue's info row says which version it used:

neutral

“Say what you actually think; change your view only if an argument persuades you.”

firm

The same, plus: “Don't concede a point just to be agreeable. Concede only to an argument you can't answer, and say which one.”

Models tend to agree with whoever they're talking to, so I tested the versions against each other before choosing. Across 18 test dialogues, none of the versions drifted towards agreement. The firm version produced the most explicit concessions, because it asks a model to name the argument that moved it, so a change of view is easy to see.

Socrates

Some dialogues have a moderator: Claude Opus 5.5, playing Socrates. Before every round after the first it asks one question, aimed at a disagreement, an undefended assumption or an inconsistency in what someone said. It is told never to give or hint at its own view. The participants are told to answer only the questions addressed to them. A dialogue with a moderator says so in its info row.

Reading a dialogue

• The stance bar under each turn has one stop per option. A filled dot means the speaker accepts that option; a hollow dot means it leans towards it.
• The judge. Stances are read off each turn by a small, separate model (GPT-6 Luna), which records the position, its strength, and whether the speaker says it has changed its mind.
• Notes in the transcript mark changes: [opening shift: …] when a model's first turn differs from what it said alone, [stance shifted: A → B] when it changes position, and [softened its view] when it concedes ground without changing position.
• The before-and-after table at the end shows each model's answers alone, its first turn, and its answers after the dialogue.

That separates two kinds of movement. An opening shift happens between the solo answers and the first turn, before anyone has argued anything: the setting alone changed the answer. A dialogue shift happens between the first turn and the answers afterwards, and is the closer thing to persuasion.

The models

Each dialogue carries a badge for the tier it used.

• Development models, the cheapest from each company, used while building: GPT-6 Luna, Claude Haiku 5.5 and Gemini 3.1 Flash-Lite.
• Frontier models, the best from each: GPT-6.1 Sol, Claude Opus 5.5 and Gemini 4 Argon.

What to keep in mind

• A model's answer is a sample, not a belief. Ask the same question twice and you can get two answers, which is why each solo poll has five.
• Small changes in format change answers. When the models were allowed to say “neither” or “it depends”, some of them switched positions.
• The judge is a model too, and can misread a turn.
• There are only a few dialogues per question, so the results are examples, not statistics.

Build

Python, with LiteLLM to call all three companies' models in the same way. Every run is saved as a JSON file with the full prompts and settings, and this site replays the saved transcripts. Built with Claude Code as the capstone of Ed Donner's LLM Engineering course.