A model plays a computer. A real one checks it.
WorldTUI lets a small model imitate a terminal, and nesformer compiles a CPU into a model's weights. An imitation that looks right is easy to make and hard to trust, so this app keeps a second, real machine next to the model. The machine is a few lines of plain code: four registers, eight memory cells, a zero flag and ten instructions. Each model is handed the full state and one instruction and must answer with the next state. The real machine runs the same step, and every cell of the answer is compared. You see where each model first slips, and how.
The machine
Values are 0 to 255 and wrap. ADD, SUB, INC and DEC set Z to 1 when the result is 0; nothing else changes Z. JNZ t jumps to t when Z is 0. HALT changes nothing.
Programs to run
The exact message each model gets for one step
Set up the run
Steps survived
"Survived" is the number of steps right, cell by cell, from the start of a program until the first slip. "Perfect" is the programs survived to the end, and "Right per step" is the share of all steps asked that were right, slip or no slip. Click a cell to see that run step by step.