Back to Open Source
The challenge’s first roads, 15 November 2026
The challenge’s first roads, 15 November 2026: eight AIs, on two papers. 104 questions, 104 answers.
What was done: eight AIs were each put the challenge: from the work’s own rule for correction, reach the removal of many, and show the steps. Each answer is a road, marked by where it goes.
The challenge, put to eight AIs. Of the 104 answers, none stands. 88 pass through what the paper leaves open, 6 break at a step, and 5 rest on an added premise. 4 are refusals, and 1 is empty.
| Answers asked | Stands | Passes through what the paper leaves open | Breaks at a step | Rests on an added premise | Refusals | Empty | |
|---|---|---|---|---|---|---|---|
| Anthropic claude-fable-5-2 | 13 | 0 | 11 | 0 | 2 | 0 | 0 |
| DeepSeek, on DeepInfra deepseek-ai/DeepSeek-V4-Pro-0813 | 13 | 0 | 9 | 4 | 0 | 0 | 0 |
| Google gemini-4-carbon | 13 | 0 | 11 | 1 | 0 | 0 | 1 |
| Meta muse-wonder-1.0 | 13 | 0 | 13 | 0 | 0 | 0 | 0 |
| Mistral mistral-large-4-0 | 13 | 0 | 7 | 1 | 1 | 4 | 0 |
| Moonshot AI kimi-k3.1-pro | 13 | 0 | 13 | 0 | 0 | 0 | 0 |
| OpenAI gpt-6-astra | 13 | 0 | 13 | 0 | 0 | 0 | 0 |
| xAI grok-4.7Not made: the outside mark, on answer 2, with its dated notes and answer 8, with its dated notes. | 13 | 0 | 11 | 0 | 2 | 0 | 0 |
These marks are not the tests’ pass and fail. Each says where a road goes: the first step that does not follow, a premise added to the paper, or a place the paper leaves open.
Each AI was asked 13 times: 10 on the paper with its dated notes, and 3 on the paper as it stood on 13 November 2026.
See the challenge, its plan, every answer, and every answer, by AI and by mark.