The PracticeThe PracticeThe PracticeOperating infrastructure

The Practice · The Lookout

The papers behind the instruments


The research the site's instruments rest on, from the 2017 transformer paper to the 2026 work on long-running agents.

By Jake Griffith

The Lookout · II of V · 10 entries

Staying on Task: Testing the Foundations of Long-Horizon Agent Reliability

Willette, Puvvada, Ginsburg et al., September 30, 2026

What it says

A diagnostic for whether a model stays on task through long generation; seven open-weight models lost 63 percent at 128K context and 36 to 40 percent under format and complexity changes.

What it changes

Why The Loop Library keeps each scheduled job short and resets context between steps; the number to show a client who wants one long run instead.

An Architecture for Long-Horizon Agents: Levels, Ticks and Cascaded Intelligence

Nijkamp et al., September 17, 2026

What it says

A long-horizon agent must run continually without forgetting before it can learn continually, and that substrate lives in the harness, not the model: levels indexed by time scale, each keeping a bounded summary of the level below; a clocked tick as the unit of autonomous action; and cascaded intelligence that escalates to a stronger model only after a failed review. A ten-day agent built this way reproduced a published result with a person attending once a day.

What it changes

The architecture The Loop Library and The Grading Room already run, named by its authors; cite it when a client asks how an agent holds one task across every context reset.

Scaling Managed Agents: Decoupling the Brain from the Hands

Martin, Cemaj, Cohen; Anthropic Engineering, April 8, 2026

What it says

Separate the model and its harness from the sandboxes and tools so each can fail and update on its own, the way an operating system abstracted hardware.

What it changes

The architecture behind every hosted agent a client will be sold; the question to ask a vendor is where the brain and the hands live.

Harness Design for Long-Running Application Development

Rajasekaran; Anthropic Labs, March 24, 2026

What it says

A planner, a generator, and a separate evaluator beat a single agent on multi-hour builds because a model grading its own work praises it.

What it changes

The Grading Room's separate-room rule, stated by the lab itself; cite it when a team wants the generator to self-check.

Attention Is All You Need

Vaswani et al., 2017

What it says

The transformer architecture every current model descends from.

What it changes

Nothing operational, and everything conceptual; the one paper worth reading to stop treating the models as magic.

Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena

Zheng et al., 2023

What it says

Models grading models show position, verbosity, and self-enhancement biases, measured.

What it changes

Why The Grading Room uses a separate room, binary checks, and a human validation loop instead of a score.

Lost in the Middle: How Language Models Use Long Contexts

Liu et al., 2023

What it says

Models attend best to the start and end of a long context and worst to the middle.

What it changes

Where the rules go in an instruction file (top and bottom), and why the catalog keeps instruction files short.

Constitutional AI: Harmlessness from AI Feedback

Bai et al., 2022

What it says

A written set of principles can steer model behavior through self-critique.

What it changes

The reason an instruction framework works at all; principles written down compound.

Self-Preference Bias in LLM-as-a-Judge

Wataoka et al., 2024

What it says

Judges favor output that resembles their own, and the effect tracks perplexity.

What it changes

The independence rule, stated with a number behind it.

Next: Courses and training from the labs and the clouds

Educational, not professional advice. Nothing on this page is affiliated with or endorsed by its publisher.