Navigating ARC-AGI: From Zero to One

Part I: Foundations

The Philosophy and Design of ARC-AGI

Core Knowledge Priors: keeping the comparison fair

For the human-vs-AI comparison to mean anything, the test has to lean on fluid intelligence — the ability to reason, solve novel problems, and adapt to new situations — and not crystallized intelligence, which relies on accumulated knowledge and skills. If a task needed you to know history or English, it would just favor whoever was pre-trained on the right material, and you'd be measuring prior exposure instead of reasoning.

ARC-AGI gets around this by relying on only a small set of Core Knowledge Priors — cognitive building blocks that are either present at birth or picked up very early in life, with minimal explicit instruction.

Keeping the required knowledge down to these few universal primitives means a good score has to come from actually reasoning, not from what you happened to know going in. The public training set is put together to walk you through every prior you'll need for the evaluation tasks — think of it as the tutorial level.

One Benchmark Becomes Three: ARC-AGI-1 → 2 → 3

Here's something that trips up newcomers: "ARC-AGI" isn't one test anymore. There are now three, and each one was born the moment AI got too good at the last. That pattern—build a test, watch it get solved, build a harder one—is the whole rhythm of this field.

ARC-AGI-1 (2019) was the original. It did its job for years, but it had a weak spot: a big chunk of it could be cracked by brute-force program search, which says more about compute than about reasoning. By the end of 2024, OpenAI's o3 cleared the human bar on it, and by 2026 most frontier models score in the 90s. It's effectively solved now—still useful as a warm-up, but no longer the real challenge.

ARC-AGI-2 (March 2025) is the current battleground and what the prize money is chasing. It keeps the same format and philosophy but is built specifically to defeat brute force and to target the things modern reasoning models are bad at. It was also calibrated against real people—a panel of 400+ testers in San Diego, with every kept task solvable by at least two humans in two tries. The average person scores around 60–66%; the panel together gets 100%.

ARC-AGI-3 (preview 2025, full launch March 2026) threw out the format entirely. Instead of static grids, you get dropped into a tiny turn-based game with no rules, no goal, and no instructions, and you have to figure it all out by playing. It's covered in its own section below. The headline: humans breeze through it, and every frontier AI scores under 1%.

Note the two very different numbers for ARC-AGI-2. That gap—between what a model can do with unlimited compute versus what fits inside the competition's offline budget—is the single most important thing to understand about where the field is right now. More on that just below.

The ARC-AGI Ecosystem: Datasets and Evaluation

To actually compete, you need to know the practical side: which datasets exist, how scoring works, and where the community lives.

Navigating the ARC Datasets

The data comes in a few separate sets, each with its own job. Using them the right way matters — both for building your solver and for trusting the numbers you get back.

Understanding the Rules of the Game

The current live competition is ARC Prize 2026, with $2,000,000 in prizes split across three tracks: a $1M track for cracking ARC-AGI-2 (the 85% Grand Prize, still unclaimed), a milestone-based track for building ARC-AGI-3 agents, and a $450K paper track for new ideas. All three want open-source work.

Part II: Core Methodologies

Program Synthesis and Domain-Specific Languages (DSLs)

Program synthesis is the oldest serious approach to ARC, and it goes straight at the heart of the problem: work out the rule from the examples. The idea (also called inductive programming) is to automatically write a program that satisfies a spec — and here the spec is just the demonstration pairs.

So you're hunting for a program P that turns every training input into its matching output. Find one, assume it captures the rule, and run it on the test input to get your answer.

This is the inductive route: pin down a general rule first, then execute it. The other route, transduction, skips the program and predicts the output directly from the examples. Program synthesis ran the show in ARC's early years — the 2020 Kaggle winner was built this way.

# Simplified representation of the solver for task 5521c0d9 from Hodel's arc-dsl
def solve_5521c0d9(I):
    # 1. Extract all non-background objects from the input grid 'I'.
    objs = dsl.objects(I, univalued=True, diagonal=False, without_bg=True)

    # 2. Merge all extracted objects into a single 'foreground' object.
    foreground = dsl.merge(objs)

    # 3. Create a new grid by removing the foreground, leaving only the background.
    empty_grid = dsl.cover(I, foreground)

    # 4. Create a function 'offset_getter' that calculates an upward shift vector
    #    equal to an object's height. This is done by composing three functions:
    #    height -> invert -> toivec (get height, negate it, convert to vector).
    offset_getter = dsl.chain(dsl.toivec, dsl.invert, dsl.height)

    # 5. Create a function 'shifter' that takes an object and moves it.
    #    The 'fork' primitive applies the 'shift' operation, using the object
    #    itself as the first argument and the result of 'offset_getter(object)'
    #    as the second argument.
    shifter = dsl.fork(dsl.shift, dsl.identity, offset_getter)

    # 6. Apply the 'shifter' function to every object in the 'objs' list
    #    and merge the results into a single object of shifted shapes.
    shifted = dsl.mapply(shifter, objs)

    # 7. Paint the final 'shifted' object onto the 'empty_grid'.
    O = dsl.paint(empty_grid, shifted)

    return O

The Power of Search: From Brute Force to Neurally-Guided

Once you have a DSL, solving a task turns into a search problem: find the sequence of primitives that does the job.

Beyond Brute Force: Smarter Search

Modern solvers use smarter search to get through that space without checking everything:

The big step forward was letting a neural network steer the search — turning it from "try everything in order" into something closer to "learned intuition" about which paths are worth following. It's faster, and a lot closer to how people actually search.

The 2025 twist: evolve the program, and let the searcher teach itself

Two ideas pushed search forward in the last competition. The first is evolutionary search: instead of building a program once, you keep a small population of candidate solutions and have an LLM mutate and recombine the best ones, generation after generation. Jeremy Berman rode this to the top of the boards — and made one counterintuitive discovery along the way. He stopped evolving Python and started evolving plain-English descriptions of the rule. Turns out natural language is easier for an LLM to tweak and refine than code, and it got him to about 29% on ARC-AGI-2 for roughly $8 a task.

The second is self-improvement. The award-winning SOAR method runs evolutionary search, then does something clever with the wreckage: every failed program is secretly a correct program for some other task (whatever it actually computed). Relabel those failures as solved examples, fine-tune the model on them, and search again — a little smarter each round. It climbed to ~52% on ARC-AGI-1 with no hand-written DSL at all. The theme to notice: in both cases, the magic isn't a single brilliant guess, it's the loop.

Test-Time Adaptation: The Modern Paradigm

Program synthesis ruled the early years, but the breakthroughs of 2024 came from Test-Time Adaptation (TTA) — letting the model adjust to each task, at inference time, using that task's own demonstration pairs. By 2024 every top solution had some form of it.

TTA splits into two flavors. Test-Time Scaling (TTS) spends more compute at inference but leaves the weights alone; Test-Time Training (TTT) briefly fine-tunes the model on the spot.

Test-Time Scaling (TTS)

TTS just throws more compute at inference, without touching the weights. That can be as simple as sampling lots of answers and keeping the best, or as involved as chain-of-thought reasoning or Sakana AI's Adaptive Branching Monte Carlo Tree Search (AB-MCTS).

Test-Time Training (TTT)

TTT actually nudges the weights at inference time — a few steps of gradient descent on the handful of demo pairs for the task in front of it, then it throws the update away. MIT's team pioneered it for ARC, and it became the backbone of several top 2024 solutions.

The Duality of Induction and Transduction

There's a split in how you can solve an ARC task, made precise in the prize-winning paper by Li et al. It lines up with System 1 and System 2 thinking — the fast-and-intuitive versus slow-and-deliberate idea from cognitive science. It's worth getting, because the strongest solvers use both at once.

The very best ARC solutions are ensembles — they run both the inductive and the transductive solver, since the two tend to fail on different tasks.

Refinement Loops: the idea that tied 2025 together

If you only remember one thing from the last competition, make it this. When the ARC Prize team looked back at everything that worked in 2025, almost every strong system — tiny models, giant models, program searchers — was doing the same thing underneath. They called it the refinement loop, and summed it up with a line worth sitting with: refinement is intelligence.

The shape is simple, almost embarrassingly so:

What makes ARC perfect for this is that it hands you a free, trustworthy checker: a candidate either reproduces the demonstration pairs exactly or it doesn't. That verifiable signal is why looping works so well here — and, honestly, why it's hard to copy this trick into messier domains where "correct" is fuzzy.

The loop shows up at three levels, and the best teams used all of them at once:

• While making training data — generate synthetic puzzles, throw out the ones that don't pass a spec, keep the rest. The 2025 winner built hundreds of thousands of tasks this way.
• While answering — a model that revises its own output step by step (the tiny recursive models below live entirely here), or a reasoning model talking itself out of a wrong answer.
• While picking a final answer — generate many candidates under different rotations/recolorings and vote.

The Plot Twist of 2025: Tiny Models That Punch Way Above Their Weight

Here's the result nobody saw coming. While everyone assumed you'd need a bigger and bigger model, three of the most talked-about papers of 2025 went the opposite way — and the paper prizes went to all three of them. On ARC, it turns out depth of thinking beats sheer size.

One more thread worth flagging, because it'll matter going forward: a 2025–26 line of work argues we've been framing ARC wrong by feeding grids to language models as text. "ARC Is a Vision Problem!" treats each grid as an image and trains a vanilla vision transformer on it — these are puzzles you look at, after all. Keep an eye on the vision-first crowd.

The Interactive Frontier: ARC-AGI-3

Everything so far has been about static puzzles: here are a few examples, predict the answer. In March 2026 the ARC Prize Foundation changed the game entirely — and if you want to work on something genuinely unsolved, this is where to look.

arc-agi-3 · real footage from the game “ft09” play a real game ↗

No instructions, no stated goal. You just act, watch what changes, and work out for yourself that the boxed grid has to be made to match the pattern. Obvious once you see it — yet every frontier model still scores under 1% on games like this.

▸ what the agent was actually thinking

“If we look at the first three completed grids, each contains a small 3×3 pixel pattern in its center… the positions of the black pixels directly dictate which of the outer 6×6 blocks get colored light blue instead of dark blue. The yellow border acts as a ‘Submit’ button for the quadrant.”

— a real frontier model playing ft09, quoted verbatim from this replay. It reasoned out the rule, took 102 actions, and still cleared just 1 of the game’s 6 levels.

What it's actually testing

Static ARC tests whether you can spot a pattern. Interactive ARC tests the messier, more human skills that come before that:

And the scoring reflects that shift: you're not graded on a final answer but on how efficiently you act compared to a human, averaged over a set of roughly 135 hand-built games. A perfect 100% means you match human efficiency everywhere.

What the early agents did

The preview round was small, and the best agent still only reached about 12.6% efficiency — but the approaches that worked tell you where to start:

• Predict what your actions do. The winning agent (StochasticGoose) trained a small CNN with reinforcement learning to guess which buttons actually change the screen, so it could explore on purpose instead of at random.
• Build a map. The runner-up kept a graph of game states and pruned moves that led nowhere or looped back.
• Plain LLM agents struggled. Dropping a big language model in and asking it to play mostly burned through actions. This is an RL / world-model problem, not a prompt-engineering one.

Part III: Practical Guide

# Clone the ARC-AGI repository for the data
git clone https://github.com/fchollet/ARC-AGI.git

# Install necessary libraries
pip install numpy matplotlib

# Create project structure
mkdir arc_solver
cd arc_solver

arc_solver/
├── ARC-AGI/          # The cloned repository
├── solver.py         # Our main solver script
└── visualize.py      # A utility for plotting grids

# In visualize.py
import matplotlib.pyplot as plt
from matplotlib import colors
import numpy as np

# Define the 10 official ARC colors
ARC_COLORMAP = colors.ListedColormap([
    '#000000', '#0074D9', '#FF4136', '#2ECC40', '#FFDC00',
    '#AAAAAA', '#F012BE', '#FF851B', '#7FDBFF', '#870C25'
])

def plot_grid(ax, grid, title=""):
    """Plots a single ARC grid with the official colormap."""
    norm = colors.Normalize(vmin=0, vmax=9)
    ax.imshow(np.array(grid), cmap=ARC_COLORMAP, norm=norm)
    ax.grid(True, which='both', color='white', linewidth=0.5)
    ax.set_xticks(np.arange(-0.5, len(grid[0]), 1), minor=True)
    ax.set_yticks(np.arange(-0.5, len(grid), 1), minor=True)
    ax.set_xticklabels([])
    ax.set_yticklabels([])
    ax.set_title(title)

def plot_task(task):
    """Plots all training and test pairs for a given ARC task."""
    num_train = len(task['train'])
    num_test = len(task['test'])
    num_total = num_train + num_test
    fig, axs = plt.subplots(2, num_total, figsize=(3 * num_total, 6))
    
    for i, pair in enumerate(task['train']):
        plot_grid(axs[0, i], pair['input'], f"Train {i} Input")
        plot_grid(axs[1, i], pair['output'], f"Train {i} Output")

    for i, pair in enumerate(task['test']):
        plot_grid(axs[0, num_train + i], pair['input'], f"Test {i} Input")
        if 'output' in pair:
            plot_grid(axs[1, num_train + i], pair['output'], f"Test {i} Output")
        else:
            axs[1, num_train + i].axis('off')
            axs[1, num_train + i].set_title(f"Test {i} Output (Predict)")
    
    plt.tight_layout()
    plt.show()

# In solver.py
import json
from visualize import plot_task

def load_task(task_path):
    with open(task_path, 'r') as f:
        return json.load(f)

# Example usage
task_file = 'ARC-AGI/data/training/ed36ccf7.json'
task = load_task(task_file)
# plot_task(task)  # Uncomment to visualize

# In solver.py
import numpy as np

def dsl_rotate_90(grid):
    return np.rot90(grid, 1)

def dsl_flip_horizontal(grid):
    return np.fliplr(grid)

def dsl_flip_vertical(grid):
    return np.flipud(grid)

# Our DSL is a dictionary mapping function names to functions
DSL = {
    'rotate_90': dsl_rotate_90,
    'flip_h': dsl_flip_horizontal,
    'flip_v': dsl_flip_vertical,
}

# In solver.py
from itertools import product

def apply_program(grid, program):
    """Applies a sequence of DSL functions to a grid."""
    current_grid = np.array(grid)
    for func_name in program:
        current_grid = DSL[func_name](current_grid)
    return current_grid.tolist()

def find_program(task, max_depth=3):
    """Searches for a program that solves the task."""
    train_pairs = task['train']
    
    # Generate all possible programs up to max_depth
    for depth in range(1, max_depth + 1):
        for program_tuple in product(DSL.keys(), repeat=depth):
            program = list(program_tuple)
            is_solution = True
            # Verify the program against all training pairs
            for pair in train_pairs:
                input_grid = pair['input']
                expected_output = pair['output']
                predicted_output = apply_program(input_grid, program)
                
                if predicted_output != expected_output:
                    is_solution = False
                    break
            
            if is_solution:
                print(f"Found solution program: {program}")
                return program
    
    print("No solution found.")
    return None

# In solver.py
def solve_task(task):
    """Finds a program and applies it to test inputs."""
    program = find_program(task)
    if program is None:
        return # Return empty predictions if no solution found
    
    predictions = []
    for pair in task['test']:
        test_input = pair['input']
        predicted_output = apply_program(test_input, program)
        predictions.append(predicted_output)
    return predictions

def main():
    task_file = 'ARC-AGI/data/training/ed36ccf7.json' # A genuine 90-degree rotation task
    task_id = task_file.split('/')[-1].replace('.json', '')
    task = load_task(task_file)
    
    predictions = solve_task(task)
    
    # Format for submission (simplified for one task)
    submission = {}
    if predictions:
        # ARC Prize allows multiple attempts, here we just submit one
        submission[task_id] = [{'attempt_1': pred, 'attempt_2': pred} for pred in predictions]

    with open('submission.json', 'w') as f:
        json.dump(submission, f, indent=4)
    print("submission.json created.")

if __name__ == '__main__':
    main()

Building a Robust Validation Pipeline

Before you write any solver code, build a local validation pipeline. It's the step most people skip, and it's the one that keeps you from fooling yourself about how well you're really doing on the hidden test set.

• Mimic Kaggle: Your pipeline should strictly separate the public training and public evaluation datasets. Your algorithm should never see the evaluation tasks during its development phase.
• Avoid Data Leakage: Repeatedly modifying your algorithm based on its score on the evaluation set, or manually inspecting those tasks to guide development, constitutes data leakage. This will lead to an inflated, unreliable local score that will not translate to the private leaderboard.
• One-Shot Execution: To get your best estimate of true performance, run your final, trained solver on the entire public evaluation set in a single execution. If your solution is computationally expensive, you can build confidence by testing on a random sample of tasks, holding out the rest for a final validation run before a full execution.
• Log your cost, too: on ARC-AGI-2, dollars and seconds per task are part of the score — and the contest box is offline with a hard time limit. Track compute from day one, so a solver that "works" but would blow the budget doesn't surprise you at submission time.

Deconstructing the Champions: Analysis of Winning Solutions

The fastest way to get good at this is to read the winners' code, and ARC makes that easy — everyone has to open-source. Below are two cohorts worth studying: the 2024 winners (on ARC-AGI-1), who set the template everyone still builds on, and the 2025 winners (on the much harder ARC-AGI-2), who show where the bar is now.

The 2024 class (ARC-AGI-1)

These solutions defined the modern playbook: a fine-tuned LLM, test-time training, and clever ways of generating and ranking candidates.

The 2025 class (ARC-AGI-2)

Same competition, much harder benchmark, and the scores dropped accordingly. What's interesting is that the winners didn't invent a new paradigm — they took the 2024 playbook and either industrialized it or twisted it. Notice the cost column: on ARC-AGI-2, efficiency is part of the score, so these are pennies-per-task, not dollars.

Charting Your Course: Strategies for ARC Prize 2026

You've got the philosophy, the methods, and the winners. Here's how to actually start competing. The combination that still wins is unglamorous: solid engineering, a strict validation habit, and one or two genuinely new ideas on top.

First, pick your track

The 2026 competition has three doors, and they suit different people. Be honest about which one you are before you sink months into it:

The ARC-AGI-2 recipe that works

If you're going for the static-puzzle track, don't start from scratch — reproduce the NVARC stack first, then improve it. No single approach solves everything, so the state of the art is a hybrid:

The Frontier: Where Do New Ideas Come From?

Reproducing the winners gets you onto the board, not to 85%. ARC-AGI-2 was built specifically to break today's methods, so the real progress is in attacking the three weaknesses it targets: