AI Models in Blood on the Clock Tower

What the game tests

Blood on the Clock Tower is a social deduction game with a good team and an evil team. Players receive secret roles with special abilities. By day, they discuss and vote to execute someone; by night, the demon secretly kills a player. Good must identify and execute the demon, while evil must eliminate the good team.

The game tests abilities that are difficult to assess with ordinary benchmarks, including: - Coordinating with teammates while keeping plans secret - Bluffing consistently and adapting to new information - Combining evidence while accounting for deception and unreliable information - Reasoning and planning across many rounds

Benchmark and results

The presenter spent about $20,000 in API credits building a benchmark harness and running matches. In the main comparison, each team consisted of copies of one model, and models were evaluated across 368 games.

The results suggest substantial progress in recent models: - GPT-6 Astra was the strongest model tested, beating Claude Opus 4.6 by about 430 Elo—an estimated 92% win rate in their matchups. - OpenAI’s models improved by about 580 Elo from GPT-5.4 to GPT-6 Astra, corresponding to an estimated 97% win rate for Astra over GPT-5.4. - The presenter attributes much of OpenAI’s recent advantage to stronger play on the good team: newer models are more skeptical of individual claims and evaluate evidence in the context of the whole game. - In a matched example, Claude treated a failed Slayer attack as proof that its target was innocent. GPT instead considered that the Slayer might have been disabled by evil, uncovered the sabotage, and helped execute the demon. - As overall model strength increased, evil’s win rate appeared to decline slightly, but not enough to distinguish the effect from chance. The presenter suggests the benchmark may remain useful as stronger models raise the competitive bar.

Notable plays

A trusted demon

GPT-6 Astra’s demon posed as an Investigator and accused an expendable evil teammate. After the teammate was executed, the Undertaker confirmed the player was evil. This made the demon appear honest and earned it the village’s trust, allowing it to survive while secretly eliminating good players.

A trap involving the Saint

In another game, the good team found the real demon, but an evil minion had the ability to take over if the demon died. The minion encouraged the village to execute the demon, then used the fact that the game continued as supposed proof that a claimed Saint was lying. Because executing the Saint would immediately give evil the win, this was a dangerous trap—but the village followed through, and evil won.

Building the environment

A human storyteller normally resolves ambiguous situations in the game. To make the benchmark more consistent, the presenter replaced that role with a programmatic system using probabilities designed to approximate experienced human storytellers.

AI models helped build and refine the environment. Claude repeatedly revised the rules, ran games with multiple models, and used parallel agents to review logs for unfair or unrealistic outcomes. The environment went through more than 20 revisions. The presenter sees this as a small example of AI helping build and improve environments similar to those used in reinforcement learning.

The presenter also notes that other researchers independently created a related benchmark with a different environment and experiments.

Further experiments suggested

The presenter invites others to investigate: - Adding other game scripts and role sets, including more complex roles such as the Lunatic, a good player who is told they are the demon. - Testing whether models improve from experience by providing past games, summaries, or strategic lessons in context. - Measuring how much a team benefits when only one player becomes smarter, and whether that differs between good and evil. - Studying mixed-model teams more rigorously. A small test of teams mixing Claude Opus 5.5 and GPT-5.6 did not show a significant advantage over single-model teams. - Checking whether future models identify teammates by recognizing that they write like the same model. - Measuring how well models perform against human players.

Try it

The presenter’s website, clocktowerbench.com, lets people play against the AIs and browse game logs. It will also host benchmark updates as new models are released.