Every team has the same structure. A few engineers ship clean, well-tested, well-scoped work, and the rest ship work that mostly works. The gap between them is not typing speed. It is everything that happens before and after the typing: the steps they remember to run, the checks they know to perform, the conventions they have internalized through years of scar tissue. That knowledge has always been the real asset, and it has never scaled, because it lived inside a small number of heads.

LLMs alone do not fix this. They make everyone faster, which is a different thing from making everyone produce the same quality. Done naively, they widen the variance, because an LLM is a multiplier on the quality of the instructions it receives. The fix is to give every engineer, and every model, the same instructions. That is what a skill is, and it is why skills are the mechanism that finally brings everyone to the same level.

The Variance Problem Has Always Been a Knowledge Problem

#

Ask yourself what actually separates the output of your strongest engineer from your weakest. It is rarely the language syntax. Both of them can write a function.

The difference is that the strong engineer, before touching code, does a long list of invisible things. They look for an existing solution before building a new one. They write down what “done” means before they implement. They check whether the change can be undone. They name things the way the rest of the codebase names things. They write the edge case the junior would have forgotten. They run the linter, and they run it before opening the pull request, not after a reviewer asks.

Each of these is a step, and a step can be written down. For most of the history of software, these steps were transmitted by osmosis. You learned them by pairing with someone better, by getting review comments on your seventh PR, by breaking production once and remembering forever. This is mentorship, and it works, but it is slow, expensive, and uneven. It cannot keep up with a team that is hiring, and it cannot keep up with a codebase that is changing. The senior engineer’s edge was always a process they ran in their head, and a process in a head does not scale.

LLMs Raise the Floor but Not the Ceiling

#

Drop an LLM into this situation and the naive expectation is that it levels the field. It does not, at least not on its own.

What an LLM actually does is amplify whatever it is given. Give it a vague one-line prompt and you get back plausible, generic code that does not match your conventions and forgets your edge cases. Give it a precise specification, the relevant files, the patterns to reuse, and the checks to satisfy, and you get back code that is hard to distinguish from your best engineer’s output. The model is the same in both cases. The difference is entirely in the context, which, as I argued in The Importance of Context When Interacting with LLMs, is the entire mechanism by which a frozen set of weights produces behavior relevant to your situation.

This has an uncomfortable implication. Before skills, the LLM made the senior engineer better and the junior engineer faster, and the gap between them stayed roughly the same. The senior engineer instinctively provides the context the model needs, because they know what good work requires. The junior engineer does not know what they do not know, so they ask for less, and they get less back. The tool that was supposed to democratize quality quietly reproduced the existing hierarchy, because it rewarded the same hidden knowledge that had always separated them.

A Skill Is the Senior Engineer’s Process, Made Executable

#

A skill is a versioned file, usually markdown, that tells an LLM agent exactly how to perform a task. Not a vague hint. The steps, in order, the checks to run before declaring success, the gates the output must pass, and the format the output should take. When the agent loads the skill, it stops improvising and follows the encoded process instead.

This is where the comparison to a human matters. A junior engineer told to “go implement the feature” forgets half the steps, because they never fully learned them. A junior engineer, or even an autonomous agent, told to implement the feature through a skill cannot forget the steps, because the steps are in the prompt the model reads on every run. The skill is not advice the engineer might ignore. It is part of the execution path.

Think of it as the checklist idea from medicine and aviation, with one critical upgrade. A surgical checklist still depends on a tired human choosing to read it and choosing to follow it. A loaded skill does not depend on anyone’s discipline. The model follows it because the skill is the instruction, and following instructions is what the model does.

Why This Collapses the Variance

#

Run the same task through two engineers with two different LLMs, but the same skill, and watch what happens to the output. Both implementations start from the same specification step. Both run the same “have you checked for an existing solution?” step. Both end at the same verification step, with the same gates. The code they produce is not identical, but it converges toward a common standard, because the process that produced it is identical.

The thing that used to vary, the invisible checklist inside each engineer’s head, is now constant. What varies is only the judgment applied at each step, and even that is bounded by the gates the skill enforces. Skills do not make everyone equally brilliant. They make everyone equally unable to skip the steps that matter, and skipping the steps that matter is what produced most of the variance in the first place.

This is the same logic I described, from the other direction, in Developer Trust Profiles. There I argued the ideal end state is one where “every contributor, senior engineer or new hire, funnels their work through agents that enforce the same standards,” so that authorship stops carrying signal and “the output converges into something homogeneous.” The trust profile was the bridge, and the point was to make itself obsolete. Skills are how you actually walk across that bridge. They are the shared, enforced pipeline that makes the variance shrink in the first place.

The Leverage Moves Up, Again

#

If the junior engineer can now produce work that follows the senior engineer’s process, what is left for the senior engineer to do?

The answer is the same one that keeps appearing everywhere AI touches the development pipeline. The bottleneck moves up the decision chain, as in The Shifting Bottleneck, and the leverage moves with it.

The new seniority is not in running the skills. Anyone, and any agent, can do that. The new seniority is in writing them. Deciding what the process should be in the first place. Deciding which gate matters and which is theater. Deciding what “done” means for this kind of task, precisely enough that a model can enforce it. The judgment that used to be applied privately, one pull request at a time, is now applied once, at the skill level, where it benefits every future execution.

This is why a good skill library is one of the most valuable assets a team can hold. It is the institutional memory of how the team does things well, written in a form that executes itself instead of sitting in a wiki nobody reads. When a senior engineer leaves, the skills stay, and the standard stays with them. When a new engineer joins, they do not spend a year absorbing the conventions through code review. They load the skills on day one, and their first pull request already follows the team’s process.

What a Skill Has to Get Right to Actually Level the Field

#

Not every file labeled “skill” collapses the variance. Most do not. Writing a skill that genuinely raises everyone to the same level requires a few specific disciplines.

Encode the steps that are actually forgotten, not the steps everyone already does. A skill that says “write clean code” is worthless, because nobody sets out to write dirty code and the phrase carries no executable instruction. A skill that says “before implementing, run a search for existing solutions and list what you found” changes behavior, because that is exactly the step people skip when they are in a hurry.

Make the gates concrete enough for a model to evaluate. “Make sure it is well tested” is an opinion. “Write tests for the empty input, the maximum value, and the concurrent case, then run the suite and confirm it passes” is a gate. The whole mechanism depends on the model being able to check its own work, and it can only do that against criteria it can test. This is the same reason I argued, in Rethinking Code Review in the Age of LLMs, that quality is a property of your constraints, not of your reviewers. The constraints in a skill are where the quality actually lives.

Keep the skill shorter than the attention it will receive. As the “lost in the middle” work showed, a model’s reliability degrades when a context fills with noise. A skill that tries to encode every possible consideration becomes a skill the model half-follows. Prefer several focused skills over one giant document, and cut anything the model would do correctly without being told.

Version and maintain it like the code it produces. A skill that encodes a convention from two years ago is worse than no skill, because it enforces a stale standard on every run. Treat the skill library as part of the codebase, with owners, review, and the same “does this still earn its keep?” scrutiny you would give any dependency.

The Realistic Limits

#

This only works for the part of the job that is describable, and it is worth being clear about where it stops.

Skills cannot encode taste that the writer cannot articulate. If your best engineer’s advantage is a feel for when an abstraction is about to collapse, and they cannot explain the signals they are reading, that advantage does not survive the translation into a file. It stays in their head, and the skill without it will produce work that is competent but not inspired. For the routine majority of software, which is most of software, the describable process is enough. For the genuinely hard design calls, it is not.

Skills can also lock in mediocrity. A skill that encodes a mediocre process enforces that mediocrity on everyone, consistently, forever. The mechanism is morally neutral. It collapses the variance in whichever direction the skill points, so a careless skill library can quietly lower a strong team to a lower common standard instead of raising the rest. The remedy is that the skills themselves have to be written by the people whose process you actually want to reproduce, and revisited when the process improves.

Finally, this raises the floor for producing code, but producing code was never the only bottleneck. As Software Engineering Teams in the Age of AI argues, deciding whether a feature should exist, and writing a precise specification for it, remain human and valuable. Skills make the execution layer homogeneous. They do not decide what to execute.

What to Do Next

#

You do not need a grand migration to start closing the gap.

Pick the one task where your team’s output varies the most, the one where the senior engineer’s pull request looks nothing like the junior’s. Write down the steps your strongest engineer actually takes when they do it well, the steps they would never admit to because they seem obvious to them. Turn that into a skill, and run the next instance of that task through it, regardless of who is doing it.

Watch the output converge. Then do it again for the next task. The variance does not close all at once. It closes one encoded process at a time, and each skill you add is a piece of seniority that stops being a private habit and starts being a shared standard.

The team that wins in this era is not the one with the most powerful model. It is the one whose best engineer’s process runs, unchanged and unskipped, in every other engineer’s session.

See also

#

• Developer Trust Profiles - names the end state this article describes how to reach, where authorship stops carrying signal because everyone ships the same quality
• The Importance of Context When Interacting with LLMs - why context is the entire mechanism by which a frozen model produces good output, and why a skill is just well-engineered context
• The Shifting Bottleneck - how each layer of automation pushes leverage, and seniority, one step up the decision chain
• Rethinking Code Review in the Age of LLMs - the case that quality is a property of your constraints, not your reviewers, which is exactly what a skill encodes
• Software Engineering Teams in the Age of AI - which processes to keep and which to eliminate once execution is cheap
• When Engineers Disagree on Best Practices - how teams settle the conventions that skills eventually enforce

References

#

• tomzx/agents - a working library of the skills this article describes, including the create-implementation and quick-pr-review skills referenced throughout the blog
• Brown et al., “Language Models are Few-Shot Learners” - established in-context learning, the capability that lets a loaded skill alter a model’s behavior without retraining
• Liu et al., “Lost in the Middle: How Language Models Use Long Contexts” - why a skill must stay focused, since model reliability degrades as context fills with noise
• Wikipedia, “The Checklist Manifesto” - Atul Gawande’s case that disciplined checklists outperform expert memory, the human analogue to the skills mechanism

A new team’s first disagreement about best practices is not a warning sign. It is the moment the team actually starts becoming a team. What separates teams that survive this phase from those that stall is not which practices they adopt, but how they resolve the disagreement.

Why the Forming Stage Produces Disagreement

#

Every engineer arrives carrying a backpack full of practices that worked somewhere else. The testing strategy that saved your last team from weekly regressions. The code review checklist that caught the security bug before it shipped. The branching model that kept releases sane. These are not random preferences. They are scar tissue from real failures, and the engineer who brings them believes, correctly, that they matter.

The problem is that every other engineer on the new team arrives with their own backpack, full of different scar tissue, formed by different failures. When these backpacks collide, you get the pattern Bruce Tuckman described in his 1965 study of small-group development: forming, storming, norming, performing. Teams do not skip from forming straight to performing. They pass through storming, the phase where members test each other, surface conflicting assumptions, and negotiate how work should be done.

The forming stage is uniquely prone to practice disagreements for three reasons.

First, there is no shared history. On an established team, conventions are settled and invisible. The team just does things a certain way because that is how it has always been done. A new team has no “always,” so everything is up for debate at once.

Second, there is no established trust. Disagreeing about code style feels low-stakes on a team where everyone has shipped together for a year, because everyone knows the argument is about the style, not about competence. On a new team, the same disagreement can feel like a referendum on someone’s expertise, because no one has yet proven anything to anyone else.

Third, there is no shared vocabulary for resolving conflict. An established team has accumulated meta-norms: how they make decisions, who decides what, when to escalate, when to let something go. A new team has none of this. So every disagreement is also, simultaneously, a negotiation about how disagreements get resolved.

Most “Best Practice” Debates Are Preference Debates

#

The first thing to recognize is that the phrase “best practice” is doing a lot of misleading work. A best practice is a practice that is demonstrably better than its alternatives across most contexts. Very few software engineering practices clear that bar.

Consider the debates that consume the most oxygen on a new team. Tabs versus spaces. Git Flow versus trunk-based development. Story points versus no estimates. Unit tests versus integration tests as the primary signal. Commit message conventions. These are not best practices. They are conventions, and a convention’s value comes almost entirely from being shared, not from being optimal. The team that uses the second-best testing framework consistently will outperform the team that uses the best one inconsistently.

This does not mean all practices are equal. Some debates involve genuine principles where the choice has lasting consequences: whether to build or buy a critical dependency, how much to invest in observability before shipping, whether a monolith or services is the right starting architecture. These deserve real deliberation. The skill is in telling the convention debate from the principle debate, because they require completely different resolution mechanisms.

Reversible Versus Irreversible Decisions

#

A distinction that Jeff Bezos drew in his 2016 letter to shareholders maps directly onto the convention-versus-principle problem. Some decisions are one-way doors. Once you walk through, you cannot come back. Choosing a primary database, committing to a cloud provider, or picking a framework that will permeate your codebase are one-way door decisions. These deserve heavy deliberation, because the cost of being wrong is high and the cost of reversing is higher.

Most practice debates on a new team are two-way doors. You can adopt trunk-based development, try it for six weeks, and switch back if it is not working. You can pick a linter configuration, discover it fights your codebase, and change it. Two-way door decisions should be made quickly, because the fastest path to knowing is often to try.

A new team that treats every practice debate as a one-way door will spend its first two months in meetings and produce nothing. A team that treats a genuine one-way door as reversible will make a choice it regrets for years. The discipline is in classifying the decision before debating it.

The Experiment Mindset

#

The single most effective frame for resolving practice disagreement during forming is to treat practices as hypotheses, not as identities.

When two engineers disagree about whether to require code review before merge, the unproductive version of that conversation is an argument about whether code review is good. Both people dig in, cite their favorite blog posts, and the discussion becomes about winning.

The productive version converts the disagreement into an experiment. You require code review for the next month, track how many defects it catches in review versus in production, and look at the data together at the end. Now the two engineers are not opponents. They are co-investigators running the same experiment.

The experiment works because it removes the requirement that someone be wrong. An engineer who advocated for mandatory review does not lose face if the experiment shows it adds little value, because they ran the test in good faith. An engineer who opposed it does not lose face if the experiment shows it catches real bugs, because they agreed to look at the evidence.

A few rules make experiments work.

Time-box them. An open-ended experiment is just a delayed argument. Four to six weeks is usually enough to see signal without exhausting the team’s patience.

Define success criteria before you start. If you wait until the experiment is over to decide what would count as success, you will re-litigate the original disagreement with data instead of opinions, which is only marginally better.

Keep the cost of reversal low. If switching back after the experiment is expensive, the experiment is really a one-way door in disguise.

Write down the result. A one-paragraph note recording what you tried, what you observed, and what you decided turns a single resolution into institutional memory that saves the next new team member from re-fighting the same battle.

How You Decide Matters More Than What You Decide

#

Here is the point that is easy to miss when you are in the middle of a heated debate about, say, whether to use feature branches or trunk-based development.

The specific answer matters less than you think. What matters enormously, especially during the forming stage, is that the team builds a repeatable process for reaching decisions together.

A team that resolves its first disagreement well, with a clear process, shared evidence, and a decision everyone can live with, has just created something more valuable than the practice it chose. It has created the template for the next disagreement, and the one after that. Each resolved disagreement makes the next one easier, because the meta-process is now familiar.

A team that resolves its first disagreement badly, by letting the loudest person win, or by avoiding the conflict until it festers, or by escalating to a manager who picks a winner, has also created a template. That template will repeat. The first disagreement a new team faces is really a dress rehearsal for every disagreement that follows.

That template is why the forming stage deserves deliberate attention rather than the hope that things will sort themselves out. The norms a team establishes in its first weeks are sticky. They become “how we do things here,” and once that identity forms, it is far harder to change later than to set well at the start.

A Playbook for the First Disagreement

#

Concretely, when the first real practice disagreement surfaces on a new team, the following sequence works.

Name it. Say out loud that this is a disagreement about practice, that disagreements are expected, and that the goal is to find a resolution the team can commit to, not to identify who is right.

Classify it. Ask whether this is a convention debate, where value comes from consistency, or a principle debate, where the choice has lasting consequences. Ask whether it is a one-way door or a two-way door.

For conventions and two-way doors, decide fast. Default to whoever has to live with the consequence most directly, or to the option that is easiest to change, or to the existing convention if one already exists. Time-box the decision to one conversation, not three.

For principles and one-way doors, slow down. Write a short decision record capturing the options, the trade-offs, and the reasoning. Get the right people in the room, which usually means the people who will own the consequences, not necessarily the people with the strongest opinions.

Either way, record the outcome. A one-page architecture decision record or a paragraph in the team handbook turns a fragile verbal agreement into something a new hire can read six months later.

And then commit. The principle of disagree and commit applies here. Once the team has decided, even those who argued otherwise support the decision fully. A practice executed with full commitment and an imperfect choice beats a better practice executed with resentment and inconsistency.

When Someone Refuses to Commit

#

There is a failure mode worth naming.

Occasionally an engineer treats a practice debate as a hill to die on, refuses to accept the team’s decision, and continues working the old way. This is no longer a disagreement about practice. It is a disagreement about whether the team’s decision-making process has legitimacy.

This must be addressed directly, and early. A team that tolerates a member who overrides collective decisions after they are made has no process at all. It has a veto by the most stubborn person. The conversation is no longer about the practice. It is about participation in a shared system.

In a healthy forming team, this conversation is rare, because the process itself was fair enough that losing a debate does not feel like losing status. When it does happen, resolving it cleanly, with empathy but without accommodation, is one of the most important trust-building acts the team will undertake. Underneath that trust is psychological safety, the shared belief that you can speak up, dissent, and even be wrong without being humiliated, which Google’s Project Aristotle identified as the single strongest predictor of team effectiveness. A team cannot storm productively without it.

The Payoff

#

A team that navigates its forming-stage disagreements well emerges with two assets.

The first is a set of shared conventions, probably imperfect, that let the team move fast without relitigating every choice. The second, and more important, is a shared process for handling the next thing they do not yet agree on.

New disagreements never stop. The codebase grows, the team grows, the technology landscape shifts, and yesterday’s settled practice becomes tomorrow’s debate. The teams that handle this well are not the ones that picked the best practices on day one. They are the ones that learned, during forming, how to disagree productively, decide efficiently, and commit fully.

That skill compounds for the life of the team.

See also

#

• Software Engineering Teams in the Age of AI - team composition and which processes earn their keep under pressure
• Principles – decision-making principles, including separating structural friction from waste

References

#

• Wikipedia, “Tuckman’s stages of group development” - the forming-storming-norming-performing model that explains why early friction is structural, not pathological
• Wikipedia, “Disagree and commit” - documents the one-way/two-way door distinction and the disagree-and-commit principle for group decision-making
• Google re:Work, “Guide: Understand team effectiveness” - Project Aristotle’s finding that psychological safety is the top predictor of team performance

When one person can spawn a dozen LLM agent sessions in parallel, the bottleneck is no longer the agents. It is the human trying to keep track of them all. Your working memory holds a handful of items at best, your context switching cost is real and measurable, and every interruption leaves residue that degrades the next task you pick up. The question is not whether you can run 20 sessions simultaneously. The question is whether you can remain effective while doing so.

The Cognitive Bottleneck

#

Before discussing strategies, it helps to understand what actually limits a human supervisor.

Working memory is tiny. George Miller’s foundational work suggested that humans can hold approximately seven plus or minus two items in short-term memory. Later research by Nelson Cowan revised this estimate down to about four chunks for young adults. Either way, the number is small. If each agent session requires you to remember its goal, its current state, its blockers, and the decision you were about to make, you hit the ceiling at three to five sessions, not twenty.

Context switching has a measurable cost. Research on task switching in cognitive psychology shows that switching between tasks incurs a time penalty, sometimes called “switch cost.” This cost ranges from fractions of a second to several minutes depending on task complexity. When you switch from agent session A to agent session B, you must offload A’s context from your working memory, load B’s context, and reconstruct where you left off. The more complex each session’s context, the more expensive the switch.

Attention residue accumulates. When you switch from one task to another without fully completing the first, part of your attention remains stuck on the previous task. The Zeigarnik effect describes how unfinished tasks persist in memory, consuming cognitive resources even when you are trying to focus on something else. With multiple agent sessions running concurrently, many are perpetually unfinished, creating a constant background hum of attention residue.

Decision fatigue compounds. Each agent session requires you to make decisions: approve this output, redirect this approach, answer this clarifying question. Research on decision fatigue shows that the quality of decisions degrades after a long session of choice-making. Supervising many sessions means making many decisions, and the later decisions in the day will be worse than the earlier ones unless you manage the load.

Working memory, switch cost, attention residue, and decision fatigue together define the ceiling on how many sessions you can manage. Every strategy below works by attacking one or more of them.

Strategy 1: Externalize All Session State

#

The single most powerful principle is to stop relying on your brain to hold session state. If it is not written down, it is consuming working memory.

Maintain a state file per session. Every agent session should produce a machine-readable and human-readable state file that captures its current status, goal, progress, blockers, and next steps. When you return to a session after an hour or a day, you read the file, not your memory. The file is the single source of truth. This is the agent equivalent of the workstack concept applied at scale.

Use structured status summaries. The state file should follow a consistent template across all sessions. A format like the following works well:

Session: fix-auth-bug
Goal: Fix OAuth callback failing on production
Status: blocked
Progress: Identified root cause (clock skew on load balancer)
Blocker: Need production access to verify fix
Next: Deploy fix to staging, verify callback works

When every session uses the same structure, you can scan a dashboard of twenty sessions in under a minute. The consistency eliminates the cognitive parsing overhead that comes from varied formats.

Write decisions to a log. Every decision you make about a session should be written to the session’s log file with a timestamp and rationale. This serves two purposes. First, when you return to the session, you can see not just where it is but why it is there, without reconstructing the reasoning. Second, the log becomes an audit trail that other humans (or agents) can consume.

Checkpoint before switching. Before switching away from any session, write a one-line “resume hint” that tells your future self exactly where to pick up. This takes five seconds and saves minutes of reconstruction time. The hint should be specific enough that you can act on it without re-reading the entire session history.

Strategy 2: Standardize All Interactions

#

Variability is the enemy of scale. Every difference between how sessions operate is a new thing your brain must parse, increasing cognitive load and switch cost.

Use a uniform session protocol. Every session should follow the same lifecycle: initialization, execution, checkpointing, escalation, and completion. The protocol defines what the agent does at each stage, what it reports, and how it signals that it needs human input. When the protocol is uniform, switching between sessions costs less because you always know what to expect.

Standardize output formats. If session A reports progress in a bulleted list and session B writes a paragraph and session C uses a table, your brain must context-switch not just on content but on format. Pick one format and enforce it everywhere. Markdown headers, numbered lists, and consistent section ordering let your eyes scan quickly without parsing structure.

Establish a shared vocabulary. Define the terms agents use to communicate status: “blocked,” “waiting-for-human,” “in-progress,” “complete,” “failed.” Every agent uses the same words to mean the same things. This reduces ambiguity and eliminates the need to translate between sessions.

Create session naming conventions. A session named “auth-fix-3” tells you nothing when you have twenty sessions. A session named “fix-oauth-callback-prod-bug” tells you exactly what it is doing. Descriptive names reduce the time to identify and triage sessions, which reduces the overhead of switching between them.

Strategy 3: Make Interactions Asynchronous and Batched

#

Synchronous interaction with agents is the enemy of parallelism. If you must respond to each agent within seconds, you can only effectively manage one at a time.

Let agents run to a natural stopping point. Instead of watching each agent in real-time and responding to each question as it arises, configure agents to work until they hit a genuine blocker. At that point, they checkpoint their state and wait. You then process the batch of blocked agents at your convenience.

Batch your decisions. When multiple sessions need human input, collect all the requests and process them in a single sitting. This is more efficient than context-switching for each one individually, because you stay in the same cognitive “mode” for the entire batch. Five decisions made in one focused session will be faster and higher quality than the same five decisions made across five interruptions.

Time-box your session reviews. Instead of monitoring sessions continuously, check them at fixed intervals: every 30 minutes, every hour, twice a day. This creates predictable rhythm and eliminates the reactive, interrupt-driven pattern that fragments attention. Between check-ins, agents work autonomously and you focus on other things.

Coalesce interruptions. If you have three sessions that will each need input within the next 15 minutes, do not handle them one at a time. Wait until all three are ready, then handle them together. Interruption coalescing is a well-known technique in operating systems and it applies equally to human attention management.

Strategy 4: Make Agents Resolve Their Own Ambiguities

#

Every question an agent asks you is a context switch. The fewer questions it asks, the more sessions you can manage.

Provide rich default resolution rules. Instead of letting agents escalate every ambiguity, give them standing instructions for common situations. “If the test suite fails, attempt to fix it before escalating.” “If a dependency is missing, install it from the lockfile.” “If a naming convention is unclear, follow the pattern used in the nearest file.” These defaults eliminate a large fraction of questions that would otherwise interrupt you.

Define escalation thresholds explicitly. Agents should escalate to you only when they cross a defined threshold: they have tried multiple approaches and failed, they need access they do not have, or they are about to make an irreversible decision. Everything else should be handled autonomously. The clearer the threshold, the fewer unnecessary escalations.

Allow agents to ask other agents. If you have multiple sessions running, agents can sometimes resolve each other’s questions. An agent that needs to understand how another part of the system works can query the agent working on that part. This peer-to-peer resolution keeps you out of the loop for routine coordination.

Use confidence-based escalation. Agents that can express confidence in their outputs can be configured to escalate only when confidence drops below a threshold. High-confidence work proceeds without interruption. Low-confidence work pauses for review. This naturally filters your attention toward the sessions that need it most.

Strategy 5: Use Hierarchical Orchestration

#

The most effective way to manage many agents is to not manage them all directly.

Introduce orchestrator agents. Instead of you supervising 20 worker agents, have 4 orchestrator agents each supervise 5 workers. You interact only with the orchestrators, who aggregate and summarize their workers’ status. This reduces your direct interaction count by a factor of 5 and lets you operate within your working memory limit. This mirrors the pattern described in Scaling the LLM Agent Company, where the orchestration layer replaces the human management layer.

Fan-out and fan-in. Give an orchestrator a single high-level task. It decomposes the task, assigns subtasks to workers, collects results, and presents you with a synthesized output. You never interact with the individual workers. Your cognitive load is the same whether the orchestrator manages 3 workers or 30.

Assign team leads per domain. If your agents work across multiple domains (frontend, backend, infrastructure, testing), assign one orchestrator per domain. Each domain lead reports to you with a one-paragraph summary. You manage four domain leads instead of twenty individual contributors.

Layer the hierarchy as needed. For very large numbers of agents, add another layer. A chief orchestrator manages domain orchestrators, who manage team leads, who manage workers. Each layer compresses information, so you always deal with a manageable number of direct reports.

Strategy 6: Practice Progressive Disclosure

#

You should never need to understand everything about every session to make a decision. Information should arrive in layers, from summary to detail, on demand.

Lead with the one-line summary. Every session status should begin with a single sentence that tells you whether action is needed: “Session blocked, needs your decision on X” or “Session running normally, 60% complete.” If the summary says no action needed, you move on. You never read the details unless the summary demands it.

Provide drill-down on request. Behind the one-line summary is a paragraph. Behind the paragraph is the full session log. Behind the log are the raw outputs. Each layer is available but not shown by default. This keeps your default view clean and your cognitive load low.

Use severity indicators. Color-code or tag sessions by urgency: red for “needs immediate human input,” yellow for “proceeding but with a risk you should know about,” green for “all clear.” Your eye scans for red first, yellow second, and ignores green. This triage happens in seconds, not minutes.

Summarize at the right granularity. An orchestrator reporting “all five workers are making progress” is useless if one of them is about to make a critical mistake. An orchestrator reporting every token from every worker is noise. The right granularity is: what is done, what is in progress, what is blocked, and what decision you need to make next.

Strategy 7: Isolate Sessions from Each Other

#

Interference between sessions is a major source of cognitive load. When sessions share state, a change in one can invalidate your understanding of another.

Enforce session independence. Each session should operate in its own workspace, with its own file tree, its own dependencies, and its own state. Sessions should not mutate shared resources that other sessions depend on. This prevents the “I changed something in session A that broke session B” problem, which is one of the most expensive debugging scenarios when managing many sessions.

Use separate branches or worktrees. If sessions work on the same repository, each should use its own git worktree or branch. This ensures that the output of one session does not corrupt the working state of another. When you are ready to integrate, you merge branches deliberately, not accidentally.

Scope tools and permissions per session. Each session should have access only to the tools and resources it needs. A session fixing a frontend bug does not need database write access. This reduces the blast radius of errors and eliminates cross-session interference.

Make session boundaries explicit. When you switch from session A to session B, the boundary should be clear. Close session A’s workspace, open session B’s. The physical or virtual separation reinforces the cognitive separation.

Strategy 8: Build a Session Dashboard

#

When managing more than three or four sessions, you need a single view that shows all of them at once. Switching between terminal tabs or IDE windows to check on each session is itself a form of context switching.

Create a single pane of glass. A dashboard that lists all active sessions with their status, progress, and severity indicators lets you assess the entire fleet in one glance. This dashboard can be as simple as a generated text file or as sophisticated as a web UI. The key is that it presents all sessions in a uniform, scannable format.

Surface only actionable information. The dashboard should show you what needs your attention, not everything that is happening. Sessions running normally should be collapsed or summarized. Sessions that need input should be highlighted. The dashboard is a triage tool, not a log viewer.

Automate status collection. The dashboard should be generated automatically from the state files that each session produces. No manual updates. If a session’s state file changes, the dashboard reflects it. This eliminates the overhead of manually polling each session for status.

Add alerting for critical events. Instead of watching the dashboard continuously, configure alerts for events that genuinely require immediate attention: a session failed, a session is about to make an irreversible change, a session has been blocked for more than N minutes. Everything else can wait for your next scheduled review.

Strategy 9: Use AI to Triage AI

#

One of the most effective ways to manage many agent sessions is to use an LLM to help you manage them.

AI-assisted summarization. Before you look at a blocked session, have an LLM read the session log and produce a one-paragraph summary of what happened, what the blocker is, and what decision you need to make. This saves you from reading potentially hundreds of lines of conversation history. The summary gets you to the decision point faster.

AI-assisted triage. When you return from a break and find 12 sessions waiting for input, have an LLM rank them by urgency and group similar requests. “Sessions 3, 7, and 12 all need the same decision about database schema, you can answer them together.” “Session 9 is urgent, it is blocked on a production deploy.” This pre-processing reduces your cognitive load and helps you batch effectively.

AI-assisted decision drafting. For each blocked session, have an LLM propose a decision with rationale. You review the proposal, accept it or modify it, and move on. Reviewing a well-formed proposal is faster and less cognitively demanding than constructing a response from scratch, especially for the tenth session of the day.

AI-assisted state reconstruction. When you return to a session after days away, have an LLM produce a “previously on” summary that reconstructs the relevant context. This is faster and more reliable than reading the full log yourself, and it handles the long-gap resumption problem that pure state files struggle with.

Strategy 10: Manage Your Own Cognitive Resources

#

The human supervisor is a finite resource. Strategies that optimize agent throughput while ignoring human cognitive limits will fail.

Chunk sessions into clusters. Instead of treating 20 sessions as 20 independent items, group them into 4 clusters of 5. Each cluster shares a domain, a goal, or an orchestrator. You manage 4 clusters, not 20 sessions. This leverages chunking, the same memory technique that lets experts recall complex board positions in chess.

Dedicate focus blocks. Reserve uninterrupted blocks of time for session supervision. During a focus block, you process the queue of blocked sessions, make decisions, and unblock agents. Outside of focus blocks, you do other work. This prevents the constant low-level attention drain that comes from monitoring sessions while trying to do other things.

Reserve decision budget for high-stakes sessions. Not all decisions are equal. A decision about whether to deploy to production matters more than a decision about variable naming. Allocate your decision budget accordingly. Make the important decisions when you are fresh. Batch the trivial decisions and make them in a low-energy slot.

Take recovery breaks. Context switching and decision-making are cognitively expensive. After a heavy session of supervising multiple agents, take a deliberate break to let attention residue dissipate. This is not laziness, it is maintenance of the tool that matters most: your judgment.

Strategy 11: Reduce Session Count Through Better Task Definition

#

The easiest session to manage is the one you never needed to start.

Write precise specifications. A well-specified task runs autonomously and produces fewer questions. A vaguely specified task generates constant clarifying questions, each of which is a context switch. Investing in specification quality before launching a session pays dividends throughout the session’s lifetime. This is the same principle described in The Importance of Context When Interacting with LLMs: the quality of the context determines the quality of the output.

Merge related tasks. If two tasks touch the same code or the same domain, combine them into one session instead of running two. One session managing a coherent area is easier to track than two sessions with overlapping scope.

Eliminate unnecessary sessions. Before launching a new session, ask whether the task is worth the supervision overhead. Some tasks are faster to do yourself than to delegate to an agent and then supervise. The shifting bottleneck principle applies: as agent supervision becomes the bottleneck, the optimal strategy shifts from spawning more agents to being more selective about which tasks to delegate.

Pre-compute common answers. If you find yourself answering the same type of question across multiple sessions, encode the answer once in a shared resource that all sessions can access. A shared FAQ, a coding standards document, or a decision tree eliminates a category of future questions.

Strategy 12: Version and Review Session Patterns

#

Managing many sessions is a skill that improves with deliberate practice.

Record what worked. When a session runs smoothly from start to finish, note what made it smooth. Was the specification clear? Were the escalation thresholds well-calibrated? Did the status format make triage easy? These observations compound into better patterns over time.

Post-mortem failed sessions. When a session goes off the rails, review why. Did the agent lack critical context? Did the escalation threshold let it run too long without checking in? Did the state file fail to capture the information you needed to intervene effectively? Each failure is a data point for improving the protocol.

Iterate on the protocol. The session protocol, the status format, the escalation rules, and the dashboard design should all evolve as you learn what works. Treat your agent management system as a product that you are continuously improving. The goal is not to find the perfect system on day one but to get a little better every week.

Share patterns across teams. If multiple people in your organization supervise agent sessions, share what works. The patterns that reduce cognitive load for one person will likely help others. This is the same knowledge-sharing principle that makes software engineering teams effective: institutional knowledge compounds when it is explicit and shared.

The Ceiling: How Many Can You Actually Manage?

#

There is no single number, but the constraints are real.

With no system, no tooling, and synchronous interaction, a human can effectively manage 1 to 2 concurrent agent sessions. The cognitive overhead of tracking each session’s state in working memory and responding to questions in real time is too high to scale further.

With externalized state files, standardized protocols, and asynchronous batching, 4 to 6 concurrent sessions become feasible. You are no longer holding state in your head, and you batch your interactions to reduce switch cost.

With hierarchical orchestration, progressive disclosure, and a dashboard, 10 to 15 concurrent sessions are manageable. The orchestrators compress information, the dashboard provides a single view, and you interact with a manageable number of direct reports.

With AI-assisted triage, tight escalation thresholds, and well-specified tasks, 20+ concurrent sessions are achievable for a human who has invested in the workflow. At this scale, you are no longer managing individual sessions. You are managing a system that manages sessions.

The progression mirrors the task-stack philosophy: you push interruptions onto a stack, handle them in batches, and pop them off when done. The difference is that the stack now contains not just your own tasks but the states of dozens of autonomous workers, and the stack is externalized rather than in your head.

The Meta-Principle

#

Every strategy in this article reduces to one of four operations on the cognitive constraints:

Reduce working memory load by externalizing state, standardizing formats, and chunking sessions into clusters.

Reduce switch cost by standardizing protocols, batching interactions, and isolating sessions.

Reduce attention residue by letting sessions reach natural stopping points, taking recovery breaks, and using focus blocks.

Reduce decision fatigue by having agents resolve their own ambiguities, using decision templates, and reserving decision budget for high-stakes choices.

The agents are not the bottleneck. Your brain is. Every hour you spend engineering the workflow around your own cognitive limits is worth ten hours of trying to power through them.

References

#

• Miller, “The Magical Number Seven, Plus or Minus Two” – established the working memory capacity limit that constrains how many sessions a human can track mentally
• Zeigarnik effect – explains attention residue from unfinished tasks, a core problem when many sessions run concurrently
• Decision fatigue – describes how decision quality degrades over a session, relevant to supervising many agents
• Chunking (psychology) – the memory technique that enables grouping sessions into manageable clusters
• Scaling the LLM Agent Company – established the orchestration layer as the replacement for human management in agent systems
• The Importance of Context When Interacting with LLMs – established that specification quality determines agent autonomy, reducing the need for human intervention
• The Shifting Bottleneck – established the pattern of bottlenecks moving up the decision chain, applicable to the human supervisor as the new bottleneck
• Workstack – the note-taking concept that externalizes task state, the basis for session state files
• Task-stack – the persistent task tracking tool that models the push/pop pattern for interruptions

Note (2026-07-02): slack-cached has been renamed to slackx.

Slack is where your team’s decisions live, but the data doesn’t belong to you. Search is slow, threads scroll out of reach, and the moment you leave a workspace the history is gone. There’s no official CLI, and the web client is the only first-class way to read anything.

I built slack-cached to fix this. It’s a small Python CLI that caches Slack threads, channel messages, users, and channels to a local SQLite database. Once cached, the data is yours: query it with SQL, grep it, feed it to an LLM, or just read it offline.

The problem

#

Slack makes exporting and archival surprisingly hard. Export tools exist for admins, but most members aren’t admins. The search box returns messages, but not in a form you can slice, join, or version. Important decisions get buried in threads that nobody scrolls back to.

The problem gets worse when you want to do anything programmatic. Building a knowledge base, summarizing a channel, or tracking decisions all require raw access to the messages. Hitting the Slack API on demand works, but you pay the latency and rate-limit cost every time, and edits disappear if you only ever fetch live.

How slack-cached works

#

$ uv sync
$ uv run slack-cached --help
usage: slack-cached [-h] {fetch,show,fetch-users,fetch-channels,show-users,show-channels,poll} ...

Cache Slack threads to a local SQLite database.

Cache a thread from a URL (no stdout, just a one-line summary on stderr):

$ slack-cached fetch https://acme.slack.com/archives/C0123ABCDEF/p1700000000123456
cached 3 messages (3 new/updated, full) for C0123ABCDEF/1700000000.123456

Or by explicit channel and timestamp; run it again and it only asks Slack for what changed:

$ slack-cached fetch --channel C0123ABCDEF --ts 1700000000.123456
cached 3 messages (3 new/updated, full) for C0123ABCDEF/1700000000.123456

$ slack-cached fetch --channel C0123ABCDEF --ts 1700000000.123456   # run again
cached 4 messages (1 new/updated, incremental) for C0123ABCDEF/1700000000.123456

Read it back, human-readable by default:

$ slack-cached show https://acme.slack.com/archives/C0123ABCDEF/p1700000000123456
Thread C0123ABCDEF/1700000000.123456
3 message(s)

[2023-11-14T22:13:20+00:00] Alice Smith (alice)
    Has anyone tried the new deploy script?

[2023-11-14T22:15:12+00:00] Bob Lee (bob)
    Yes, but you need to bump the token first.

[2023-11-14T22:16:45+00:00] Alice Smith (alice)
    Thanks!

show auto-fetches if the thread isn’t cached yet, and renders real names like Alice Smith (alice) instead of raw user ids once you’ve cached users. Pass --json to get the raw records for piping into other tools:

$ slack-cached show --json https://acme.slack.com/archives/C0123ABCDEF/p1700000000123456
{
  "channel": "C0123ABCDEF",
  "channel_name": "deploys",
  "thread_ts": "1700000000.123456",
  "message_count": 3,
  "messages": [
    {
      "ts": "1700000000.123456",
      "user": "U1",
      "text": "Has anyone tried the new deploy script?",
      "payload": { "ts": "1700000000.123456", "user": "U1", "type": "message" },
      "user_name": "Alice Smith (alice)"
    },
    {
      "ts": "1700000112.000200",
      "user": "U2",
      "text": "Yes, but you need to bump the token first.",
      "payload": { "ts": "1700000112.000200", "user": "U2", "type": "message" },
      "user_name": "Bob Lee (bob)"
    },
    {
      "ts": "1700000205.500300",
      "user": "U1",
      "text": "Thanks!",
      "payload": { "ts": "1700000205.500300", "user": "U1", "type": "message" },
      "user_name": "Alice Smith (alice)"
    }
  ]
}

Channels, users, and polling

#

Cache every top-level message in a channel:

$ slack-cached fetch --channel C0123ABCDEF
cached 142 messages for C0123ABCDEF (38 fetched)

Add --full-threads to also pull every reply thread, so you get the full conversation tree:

$ slack-cached fetch --channel C0123ABCDEF --full-threads
cached 318 messages for C0123ABCDEF (214 fetched, 47 threads with replies fetched)

For continuous capture, poll watches multiple channels and fetches new messages on a schedule. Each entry in --channels can be a channel id, a bare name (general), or a #-prefixed name (#general); names are resolved against the cached channels, which is why the per-cycle summary below reports ids. Progress goes to stderr, and (because of --json) one compact JSON object per cycle goes to stdout. Ctrl+C stops it gracefully:

$ slack-cached poll --channels #general,#engineering,#deploys --interval 5m --last 5m --full-threads --json
polling 3 channel(s) every 5m (lookback: 5m, full_threads: True, concurrency: 3)
{"cycle": 1, "elapsed_seconds": 1.234, "channels": [{"channel": "C001", "fetched": 3, "total": 312}, {"channel": "C002", "fetched": 4, "total": 188}, {"channel": "C003", "fetched": 0, "total": 27}]}
cycle 1: 7 new message(s) across 3 channel(s) in 1.2s
^C
poll stopped after 1 cycle(s)

That stdout stream is easy to wire into a downstream pipeline or a knowledge-base builder.

Finally, cache the workspace’s users and channels so threads can be rendered with real names:

$ slack-cached fetch-users
processed 184 users (184 added, 184 total in db)
$ slack-cached fetch-channels
processed 37 channels (37 added, 37 total in db)

$ slack-cached show-users
184 user(s)

U1  alice - Alice Smith
U2  bob - Bob Lee
...

$ slack-cached show-channels --json
{
  "channel_count": 37,
  "channels": [
    {
      "id": "C1",
      "name": "general",
      "is_private": false,
      "fetched_at": 1700000260.0,
      "payload": { "id": "C1", "name": "general", "is_channel": true }
    },
    ...
  ]
}

The refresh strategy

#

fetch always reaches out to Slack, but it’s incremental. On a re-fetch, it calls conversations.replies with oldest=<latest_cached_ts>, so the API returns only new replies and any edits at the boundary. Messages are upserted by ts, which means edits replace the old text in place instead of creating duplicates.

Rate limits are handled for you. HTTP 429 / ratelimited responses are retried with exponential backoff, up to five attempts, respecting the Retry-After header. That matters for --full-threads and poll, where you can easily fire hundreds of calls against a busy channel.

Where the cache lives

#

The database defaults to $XDG_CACHE_HOME/slack-cached/threads.db (typically ~/.cache/slack-cached/threads.db). Override it per command with --db /path/to/file.db. Because it’s plain SQLite, you can open it directly and ask it anything Slack’s search box can’t:

$ sqlite3 ~/.cache/slack-cached/threads.db \
    "SELECT (u.real_name || ' (' || u.name || ')') AS who, COUNT(*) AS msgs
     FROM messages m JOIN users u ON m.user = u.id
     WHERE m.channel = 'C0123ABCDEF'
     GROUP BY who ORDER BY msgs DESC LIMIT 5;"
Alice Smith (alice)|142
Bob Lee (bob)|87
Carol Ng (carol)|53

That query is the real point of the tool. The cache is not an opaque blob; it’s a table of messages you can SELECT from, join against users and channels, and export however you like.

Authentication

#

Credentials load from environment variables first, then a config file at $XDG_CONFIG_HOME/slack-cached/config:

SLACK_TOKEN=xoxb-...
SLACK_COOKIE=...
SLACK_API_BASE_URL=https://slack.com/api

SLACK_COOKIE is there for xoxc- web-client tokens, which need the matching cookie to authenticate. Every command also accepts --api-base-url, which is how the built-in fake Slack server plugs in.

A fake Slack server, for free

#

The repo ships slack-fake-server, a deterministic fake Slack API for testing:

$ uv run slack-fake-server --port 8199 --num-threads 50 --rate-limits
2026-06-15T12:00:00Z [info     ] fake_slack_server_starting   host=127.0.0.1 port=8199 seed=42 users=8 channels=5 threads=50 rate_limits=True

It serves conversations.list, conversations.replies, conversations.history, and users.list, and can simulate Slack-tier rate limiting. Point slack-cached at the fake server and you can develop and test against a realistic API without touching your real workspace:

$ slack-cached fetch --api-base-url http://localhost:8199/api --channel C0123ABCDEF --full-threads
cached 318 messages for C0123ABCDEF (214 fetched, 47 threads with replies fetched)

When this is useful

#

• Knowledge bases and channel summaries. Poll a set of channels, then build monthly digests or feed the SQLite cache to an LLM.
• Decision tracking. Important calls often happen in threads. Cache them so they survive workspace churn and account turnover.
• Offline access and archival. Keep a readable copy of the conversations you actually care about, independent of Slack’s retention window.
• Bulk analysis. Once the data is in SQLite, you can answer questions with a query that Slack’s search box can’t express.

What to Do Next

#

$ git clone https://github.com/TomzxCode/slack-cached
$ cd slack-cached
$ uv sync
$ export SLACK_TOKEN=xoxb-...
$ slack-cached fetch-channels
processed 37 channels (37 added, 37 total in db)
$ slack-cached fetch --channel C0123ABCDEF --full-threads
cached 318 messages for C0123ABCDEF (214 fetched, 47 threads with replies fetched)
$ slack-cached show https://acme.slack.com/archives/C0123ABCDEF/p1700000000123456
Thread C0123ABCDEF/1700000000.123456
3 message(s)

[2023-11-14T22:13:20+00:00] Alice Smith (alice)
    Has anyone tried the new deploy script?

[2023-11-14T22:15:12+00:00] Bob Lee (bob)
    Yes, but you need to bump the token first.

[2023-11-14T22:16:45+00:00] Alice Smith (alice)
    Thanks!

Repository and source code, documentation.

References

#

• TomzxCode/slack-cached README – primary source for features, commands, and behavior.
• slacrawl – related prior art referenced by the project.

When you automate code review, the obvious design is to treat every pull request identically. Same checks, same threshold, same outcome, regardless of author. This obvious design is wrong.

A reviewer that applies the same scrutiny to a developer who has shipped two hundred clean PRs and to a stranger on their first contribution is either too strict for the first or too lax for the second. It is a blunt instrument, and bluntness is the enemy of autonomy. The more you rely on automated approval to unblock work, the more you need a mechanism that varies scrutiny by evidence.

I built that mechanism as a pair of skills in my agent library, developer-trust-profile and initialize-developer-trust-profile. This is the idea behind them, and why I believe it is the missing primitive for autonomous code review.

The problem with a reviewer that has no memory

#

Most automated review tools are stateless. They look at a diff, run their checks, and emit a verdict. Next PR, they start over, as if they had never seen the author before.

This is not how any experienced human reviewer works. When a trusted colleague opens a PR, you skim it, confirm the tests pass, and approve. When someone with a history of breaking the build opens one, you read every line. You already know who tends to forget tests, who mixes unrelated refactors into a single commit, who writes the clearest descriptions. That knowledge is not bias to be eliminated; it is signal, accumulated over hundreds of reviews, that tells you where to spend your attention.

A stateless automated reviewer throws all of that signal away. It re-derives, badly and from scratch, what a human reviewer simply remembers. The result is a system that either approves indiscriminately (unsafe) or applies maximum suspicion to everyone (slow, and corrosive to trust).

The trust profile is the answer to a direct question: how does an automated reviewer remember?

What a trust profile stores

#

Each developer gets a single file, ~/.developer-trust/{github_username}.md, and the whole directory is a git repository. The file accumulates observations across every review.

Four trust levels exist, and each one maps to a different behavior, not just a different label:

Beyond the level, the profile keeps a running overview, lists of strengths and weaknesses observed across PRs, recurring PR patterns, and a full review history table with dates, repos, outcomes, and a one-line note per review.

Crucially, it is plain markdown in a git repo. That makes the system’s memory auditable, diffable, and portable. You can see the exact review that tipped an author from neutral to cautious, and the reasoning behind it. For a mechanism that gates code into production, that paper trail is not a nicety; it is a requirement.

How earned trust changes review behavior

#

The levels are not decorative. They modify how the automated reviewer interprets its own checks.

The reviewer I run, quick-pr-review, evaluates a fixed set of gates: significant public interface changes, security-sensitive code, new dependencies, reversibility, passing CI, alignment with the linked issue’s acceptance criteria. For a neutral author, each gate is binary. For a trusted author, a borderline call (is this small new export a “significant” public interface change?) leans toward passing. For a cautious author, that same borderline call is treated as a failure.

This is differential scrutiny, and it is the whole point. The checks are identical; the threshold moves with evidence. A trusted author gets unblocked faster because the reviewer stops re-litigating cases it has effectively already won. A cautious author gets caught earlier, before a recurring weakness reaches production again.

And then there is always_reject, the hard stop. When an author sits at that level, the reviewer does not fetch the diff, does not post a comment, does not approve. It reports that the PR was skipped and that a human needs to look. This is the system saying: I have enough evidence to know I should not be making this decision.

The loop closes

#

The profile is not a static config file that a human maintains by hand. It is both consumed and produced by the review pipeline.

Before a review, quick-pr-review reads the author’s profile to set its thresholds. After the review, it writes back: it appends a row to the history, merges new observations into the strengths and weaknesses, and reconsiders the trust level if the accumulated evidence warrants it. Each profile update is committed to the local git repo with a message like Update alice trust profile (approved: acme/api#42).

This makes the system a slow learner rather than a judge. A single review does not move the trust level, unless it is egregious. Trust degrades through recurring patterns observed across many PRs, and the level only shifts when the weight of evidence demands it. That hysteresis is deliberate. It prevents one bad day from branding a developer, and it prevents one lucky PR from buying unwarranted autonomy.

Bootstrapping without survivorship bias

#

The hardest part of any reputation system is the cold start. A new contributor, or a contributor new to the system, starts at neutral with an empty file. That is safe but unhelpful; you want a profile grounded in reality, not a blank slate that takes months to fill.

The initialize skill bootstraps a profile from history by scanning the author’s last N pull requests across every repository the token can see. The detail that separates this from a naive implementation is that it samples both merged and rejected PRs.

Sampling only merged PRs is survivorship bias in its purest form. It would make every developer look good, because the failures were quietly closed and forgotten. By pulling closed-without-merge PRs as well, and by deriving the outcome from actual review data (was there a changes-requested that was never superseded?) rather than assuming merge equals approval, the bootstrap produces a profile that reflects how a developer actually works, not just their wins.

Processed oldest-first, each historical PR feeds into the same update pipeline as a live review, so a freshly initialized profile looks exactly like one that was built up review by review over time.

Real limitations

#

The design has real trade-offs, and they are worth naming.

Profiles live on the reviewer’s machine and are not shared. This is a feature (no global reputation database, no public scoring of humans) and a cost (each reviewer builds a different picture, and the memory does not transfer). For a single operator running their own agents, that is the right trade. For an organization, you would want a shared, access-controlled store, and the design does not pretend otherwise.

There is a risk that a reputation hardens into something a developer cannot escape. The skills mitigate this by removing observations that recent evidence contradicts and by reconsidering the level on every update. But any system that summarizes a human into a label can lock them in. The fix is transparency (everything is in a diffable file) and a human who can edit the file when the summary is wrong.

There is also a gaming risk. Once authors know a trust system exists, they can optimize for it. On balance this is fine: optimizing for clean, well-tested, well-scoped PRs that reference their issues is exactly the behavior you wanted anyway. The system fails safe, because the worst case is that people produce better PRs.

Why this matters more in the age of LLM-authored code

#

I have written elsewhere about why human review of LLM-generated code is a poor use of attention (see The Future of Code Review). The short version is that the human’s leverage has moved upstream, to specifying the problem, while machines verify compliance.

But that argument has a gap. Even after you accept automated review, you are left with a follow-up question: should the automated reviewer treat every author identically? The answer is no, and for a reason that is sharper now than it was five years ago.

In an LLM-heavy workflow, the “author” of a PR is increasingly a human paired with a model, or an agent operating on a human’s behalf. The trust profile stops measuring just a developer’s coding skill and starts measuring something more valuable: the quality of a human’s oversight of their tools. Two developers can submit PRs that an LLM wrote. One vets the output carefully, keeps PRs atomic, links the spec, and ships reversible changes. The other rubber-stamps whatever the model produced, mixes concerns, and breaks public interfaces. A stateless reviewer cannot tell them apart. A reviewer with a trust profile can, and it calibrates its scrutiny accordingly.

Seen this way, the trust profile is the layer beneath the automated reviewer: the checks define what “good” looks like, and the profile decides how much to trust that a given author is delivering that standard.

But that framing also points at the profile’s eventual obsolescence, which is the goal.

A bridge, not a destination

#

There is a sense in which the trust profile is a mechanism I want to make obsolete.

The ideal end state is not a finely calibrated reputation system that perfectly sorts developers into tiers. The ideal end state is that there is nothing to sort. Every contributor, senior engineer or new hire, funnels their work through agents that enforce the same standards: atomic PRs, tests that pass, interfaces that do not break, changes that are reversible and tied to a spec. The output converges into something homogeneous, and by the time it reaches the code stage it is, for all practical purposes, perfect. Authorship stops carrying signal, because every author is producing the same uniform quality through the same disciplined pipeline.

In that world the reviewer’s checks still run, but they never flag anything, because the problems were engineered out upstream rather than caught downstream. And the trust profile, with nothing left to differentiate, collapses: everyone sits at trusted, the level never moves, and the file stops being worth reading.

That is the accurate framing for the trust profile. It is not the destination; it is the bridge. Today’s reality is heterogeneous, a mix of careful and careless, hand-written and agent-generated, vetted and rubber-stamped, and the profile does useful work precisely because that variance exists. As the variance shrinks, so does the profile’s job, until the most successful outcome is that no one, me included, needs it anymore.

Most interactions with LLMs fail not because the model lacks capability, but because the user fails to provide enough context for the model to succeed. Context is not a prompt engineering trick. It is the entire mechanism by which a frozen set of weights produces behavior relevant to your specific situation.

What Context Means for an LLM

#

A large language model is a fixed function at inference time. Its weights were determined during training and do not change. Everything you want the model to know about your current task, your constraints, your codebase, your domain, and your preferences must be communicated through the context window.

This window has three conceptual regions.

1. The system prompt establishes persistent instructions and persona.
2. The conversation history provides the back-and-forth of the interaction.
3. The retrieved or injected context supplies external knowledge that the model was not trained on.

When people say “prompt engineering,” they are usually talking about the system prompt and a handful of few-shot examples. When people say “RAG,” they are talking about dynamically injecting retrieved documents into the third region. When people say “context engineering,” they are talking about the deliberate design of all three regions together.

In-Context Learning: Why This Works at All

#

The GPT-3 paper demonstrated something surprising: a sufficiently large language model can learn new tasks from examples provided in the prompt, without any gradient updates. This is in-context learning, and it is the reason context matters so much.

A model that has never seen your internal API conventions can follow them perfectly if you show it three examples. A model that does not know your company’s style guide can adopt it verbatim if you paste it in. The model is not truly “learning” in the statistical sense. The model is recognizing patterns in the provided context and extending them.

This means the quality of the context is the quality of the output. Vague context produces vague output. Contradictory context produces contradictory output. Missing context produces plausible-sounding output that is wrong in ways specific to your situation.

The Spectrum of Context Strategies

#

Context can be provided along a spectrum, from minimal to extensive.

Zero-shot prompting relies entirely on the model’s pre-trained knowledge. Zero-shot prompting works for generic tasks (summarize this text, translate this sentence) because the training data likely contained millions of similar examples. Zero-shot prompting fails for domain-specific tasks (generate a query against our proprietary schema, review code against our internal standards) because the model has never seen your specific conventions.

Few-shot prompting injects examples into the context window. The original GPT-3 evaluation showed that performance scales with the number of examples, but with diminishing returns. Three to five high-quality examples often capture 80% of the benefit. Example quality matters more than example quantity. One example that perfectly demonstrates the desired behavior outperforms ten mediocre ones.

Retrieval-augmented generation (RAG) dynamically fetches relevant documents and injects the documents into the context. The original RAG paper showed that combining a parametric model with non-parametric retrieval produces better grounded answers than either alone. RAG addresses the fundamental limitation that a model’s training data is a snapshot in time and cannot contain proprietary or recent information.

Long-context ingestion bypasses retrieval by stuffing everything into a large context window. Models such as Gemini with million-token context windows make the approach technically feasible. The question is whether dumping everything into the window actually works. Liu et al. demonstrated that models suffer from a “lost in the middle” effect: information at the beginning and end of the context is retrieved more reliably than information in the middle. Simply increasing the window does not linearly increase the model’s ability to use the information.

Context engineering is the emerging discipline of orchestrating all of these strategies deliberately. Context engineering recognizes that context is not just “what you put in the prompt,” but a system that includes retrieval logic, example selection, conversation management, and information ordering.

Where Context Breaks Down

#

Understanding the failure modes of context is as important as understanding how to provide context.

Attention dilution. Every token in the context window competes for the model’s finite attention. Adding irrelevant context does not just waste tokens. Adding irrelevant context actively degrades performance on the relevant portions. Liu et al.’s “Lost in the Middle” showed that even when the relevant information is present in the context, retrieval accuracy drops when the context is cluttered with noise. The lesson: more context is not always better. The right context, curated and ordered, outperforms a dump of everything that might be relevant.

Instruction following degrades at scale. System prompts and instructions are more reliably followed when the instructions are prominent in the context. As the context window fills with retrieved documents, conversation history, and examples, the model’s adherence to its original instructions weakens. Weakening adherence is why many production systems re-inject key instructions at the end of long contexts, not just the beginning.

Stale context. In a long agent session, early conversation turns become increasingly irrelevant. The model continues to weigh the early turns in its attention computation. The result is drift, where the model brings up constraints or preferences from turn 3 that are no longer applicable at turn 30. Effective agent systems summarize or prune earlier context to maintain relevance.

Implicit context gaps. The most dangerous context failures are the ones you do not notice. You assume the model knows that your API returns snake_case JSON, that your timestamps are in UTC, that your user IDs are integers not strings. The model does not know any of these things unless you tell it. Each implicit assumption is a potential bug in generated code or a hallucinated fact in generated text.

Practical Context Design

#

For engineers building with LLMs, context design is the highest-leverage activity. Here are concrete patterns that work.

Layer your context deliberately. Start with a system prompt that defines the task, constraints, and output format. Follow with retrieved documents that are directly relevant to the current query. Then provide conversation history, pruned to the most recent and relevant turns. End by restating the specific question or instruction. This ordering respects the model’s attention patterns, which weight the beginning and end of context most heavily.

Curate your examples. A few-shot example should demonstrate the hardest case, not the easiest. If your task involves edge cases (empty inputs, special characters, ambiguous queries), show examples of those edge cases. Showing five variations of the happy path teaches the model less than one example of each difficult variant.

Use structured context. When injecting retrieved information, format the information consistently. Markdown headers, delimited sections, and numbered lists give the model explicit boundaries between pieces of information. A blob of unstructured text forces the model to spend attention on parsing structure rather than reasoning about content.

Separate context from instruction. Mixing “here is some reference information” with “now do this task” in the same unstructured block reduces reliability. Explicitly mark where context ends and instructions begin. Many production systems use XML tags or special tokens for this purpose.

Test your context independently. Before deploying an LLM pipeline, test whether the model can answer simple factual questions based solely on the provided context. If the model cannot reliably retrieve a fact that is clearly present in the context, the context is too long, too noisy, or poorly structured. The test is a fast diagnostic that catches many context problems before the context problems become production incidents.

Context Engineering as a Discipline

#

The shift from “prompt engineering” to “context engineering” reflects a maturation in how we think about LLM interactions. Prompt engineering suggests that the right magic words unlock better performance. Context engineering recognizes that the entire information environment determines the output.

The distinction matters because the distinction changes where you invest effort. If you believe in magic prompts, you spend your time iterating on wording. If you believe in context engineering, you invest in retrieval systems, example libraries, context ordering, and information architecture.

The results from Anthropic’s work on contextual retrieval illustrate the payoff well. By adding a small amount of context to each retrieved chunk (explaining how the chunk relates to the broader document), Anthropic reduced retrieval failure rates by 67%. Not by changing the model. Not by changing the prompt. By changing how context was prepared and presented.

Chain-of-thought prompting is another example. The model’s reasoning improves not because you asked the model to “think step by step” as a magic incantation, but because you expanded the context window with intermediate reasoning steps. The model uses its own generated context as additional input for subsequent tokens. Context is not just what you provide. Context is also what the model generates and then consumes.

The Uncomfortable Implication

#

If context determines output quality, the ceiling on LLM performance in production is not the model’s capability but the quality of the context pipeline feeding the model.

The conclusion is uncomfortable because the hardest engineering problem in LLM applications turns out not to be model selection or fine-tuning. The hard problem is information retrieval, information architecture, and information presentation. These are old problems from search engineering and information science, now applied to a new interface.

The engineers who build the best LLM products will not necessarily be the ones who understand transformer architectures most deeply. The best LLM products will be built by the engineers who can design systems that surface the right information, at the right time, in the right format, and place the information where the model will actually attend to it.

Context is not a feature you add to an LLM application; it is the application.

References

#

• Brown et al., “Language Models are Few-Shot Learners” – established in-context learning as a core capability of large language models
• Lewis et al., “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks” – introduced RAG as a method for grounding LLM outputs in retrieved documents
• Liu et al., “Lost in the Middle: How Language Models Use Long Contexts” – demonstrated positional bias in how models attend to long contexts
• Wei et al., “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models” – showed that intermediate reasoning steps in context improve model performance
• Anthropic, “Contextual Retrieval” – demonstrated that context preparation significantly reduces retrieval failure rates

My previous piece was about code review: a human reading code before it ships. This one is narrower, and blunter. It is about the merge gate, the specific act of requiring a human to click “approve” before code can land in a codebase.

People treat review and approval as the same thing. They are not. You can review code without being able to block it. You can approve code without reading it. The valuable act (reading) and the gating act (merging) have been fused by our tooling, and that fusion is worth pulling apart.

Most arguments for keeping a human in the merge loop collapse once you ask a single question. What does the approval click certify that the automated gates did not?

What the Approval Click Actually Certifies

#

Watch what happens when a pull request gets approved.

The approver sees a green checkmark from CI. They see a passing test suite. They scan the diff for a few seconds. They click approve.

Ask them, afterward, what they certified. They will struggle to answer.

They did not re-run the tests; CI already did that. They did not verify the implementation against the spec; they assumed someone else did. They did not reason about every edge case; the diff was too long to reason about in three minutes. They did not assess whether the change could be undone; that is not what diffs show.

The approval click, in most teams, certifies exactly one thing: that a specific human was awake and present at the moment the PR was merged. That is a low-information event, and it is doing almost no safety work.

The actual safety is being produced by the systems around the gate. The test suite. The linter. The static analyzer. The deployment pipeline. The rollback path. The human approval is layered on top, taking credit for safety it did not generate.

The Cost of the Gate

#

A gate that does little safety work still extracts a cost. Several costs, in fact, and they compound.

Latency. Every pull request waits for a human. That human is in a meeting, asleep, on another team, or simply not looking at their inbox. Cycle time stretches from minutes to hours to days. Work that could ship this morning ships next Tuesday.

Batch amplification. This is the subtlest and most damaging cost. When approval is expensive to obtain, people batch. They hold three small changes until they have a fourth, because each approval is a fixed-cost interruption. The gate incentivizes the exact thing it should discourage: larger, riskier changes. A mechanism designed to keep changes safe ends up making them less safe, because the unit of review grows to fill the cost of getting reviewed.

Context switching. Every approval is an interruption for the approver. Their work is paused, their context is swapped, their focus is fractured. You are taxing your most experienced engineers to perform a low-information ritual.

Single point of failure. If only two people can approve a given area and both are out, the pull request stalls. Approval authority concentrates, and concentration creates bottlenecks and bus factors.

Soft target. The approver is the cheapest attack surface in the entire pipeline. You can harden CI, pin dependencies, and scan for secrets, but a tired approver who clicks approve on a social-engineered change defeats all of it. A human gate is a human vulnerability.

What Actually Makes a Merge Safe

#

Strip the ritual away and ask what keeps a merge safe. It is a short list, and the human approver is not on it.

A comprehensive test suite that runs on every change. Static analysis that catches the classes of bugs tests miss. A deployment pipeline that canaries before it fully rolls out. Feature flags so new behavior can be disabled without a redeploy. A rollback path that has actually been tested, not assumed. Blast-radius limits that cap how much any single change can touch.

Each of these operates on reality, not on a human’s prediction of reality. A canary either surfaces the problem or it does not. An approver might notice the problem, or might be thinking about lunch.

The gate is taking credit for safety the system produces. Once you see this, you cannot unsee it.

The Accountability Objection

#

The first objection is accountability. Without a human approval, who is responsible for the code that ships?

I made this argument in the previous piece, and I will not repeat all of it here. The short version is that a human glancing at a diff was never truly responsible for what shipped. Responsibility lives upstream, in who decided the problem was worth solving and who wrote the specification.

The approval signature adds nothing to that picture. It is a name on a line, demanded because it feels like accountability. When a change breaks production, the approver is not the one who gets blamed, disciplined, or even consulted. The signature exists to allocate blame after the fact, not to prevent harm before it.

What remains, once you accept this, is the audit and compliance framing. That is the stronger case for keeping a human signature, and it deserves its own answer.

The Compliance Objection

#

“But regulation requires human approval.”

Sometimes it does. More often, regulation requires traceability, a named owner, a documented decision, an auditable path. Those are not the same thing as a tired human clicking approve at four in the afternoon on a Friday.

When a rule says a change must be “reviewed and approved”, it is trying to ensure that someone with authority consciously decided the change was acceptable. That intent can be satisfied several ways. A named owner who signed off on a specification. An automated gate whose rules were themselves approved by a human. A risk classification that a human defined and a machine enforces consistently.

The function is accountability and traceability. The form is “a human clicks a button”. Teams satisfy the form and skip the function all the time. The sound path is the reverse: satisfy the function rigorously, and let the form follow.

If your auditor insists that safety lives in a specific human keystroke, you have an education problem, not an engineering one.

The Binary Mistake

#

The deeper error is treating approval as binary. Either every pull request needs a human, or no pull request does.

This is wrong, and it is not how anyone actually behaves. A README typo and a production schema migration are both pull requests. They do not need the same gate. Treating them the same is not caution; it is a failure to think about risk.

The right unit of gating is not “is this a pull request”. It is “what is the blast radius of this change, and is it reversible”.

A one-line documentation fix is low blast radius and trivially reversible. Let it merge on green. No human needs to see it.

A change that drops a database column is high blast radius and may be irreversible. That deserves a human looking at it, carefully, with time.

A change that alters an authentication boundary is medium blast radius but high trust impact. That deserves a human, and probably more than one.

The properties that should trigger a human gate are properties of the change: irreversibility, blast radius, trust-boundary crossing, external commitment. They are not properties of the artifact. When you gate on the change instead of the artifact, the fraction of changes that need a human collapses to a small minority.

The Small Set Where a Human Gate Genuinely Adds Value

#

Frankly, there is a set of changes where a human in the merge loop is not theater. It is small, but it is real.

Irreversible changes. Data destruction, schema drops, deletions of public content, sending real money, publishing to external systems. Once these execute, you cannot call them back. A human who understands the irreversibility should look at them, because the automated gates can only verify forward correctness, not undo impossibility.

Trust-boundary changes. Authentication, authorization, permission models, security-sensitive code paths. These are exactly where a subtle mistake is both likely and catastrophic. A human reviewer adds value here, not because they will catch every bug, but because the cost of a miss is high enough to justify the latency.

Changes to the gating system itself. You do not want the merge gate to auto-approve changes to the merge gate. That is the one place circularity will bite you. A human reviews the rules that the machine enforces.

External commitments. Public API changes, contractually obligated behaviors, compliance-relevant logs. These have consequences outside the codebase, and a human should confirm the external surface is intentional.

Note what is not on this list. Styling. Refactors within a single module. New tests. Documentation. Internal-only features. Dependency bumps that pass audit. These are the overwhelming majority of pull requests. They do not need a human gate. They need the automated gates, and then they need to merge.

The Transition

#

You do not get to “no human gate for most changes” by decree. You get there by making the default path safe.

Start by making the low-blast-radius path auto-merge on green. Documentation, tests, internal-only changes within a single module. CI passes, the merge happens, nobody clicks anything.

Then compute blast radius automatically. Which files changed. Did the change touch the public API. Did it touch the database schema. Did it add a dependency. Did it change infrastructure or deployment configuration. Did it cross a security boundary.

Each of these is a machine-checkable property. Route the change to the human gate only when it crosses a threshold.

When a change does reach the human gate, change what the human is actually doing. They are no longer reviewing code line by line. They are reviewing risk. Is this change as irreversible as the system thinks it is? Is the rollback plan real? Is the blast radius acceptable? Does the external commitment match what was approved upstream?

Reviewing risk is high-leverage work. Reviewing diffs for style is not. The transition moves the human from the low-leverage activity to the high-leverage one, and it frees the rest of the pipeline to move at the speed the machines can sustain.

This is not a proposal. It is a description of the workflow from the previous piece, where a pull request is checked against concrete gates: does it introduce security-sensitive changes, add dependencies, change public interfaces, is it reversible, do the tests pass, does it satisfy the acceptance criteria. What this article adds is the principle behind that workflow. The human gate is the exception, selected by the properties of the change, not the default triggered by the existence of a pull request. The skills that implement it are publicly available.

The Merge Button Is a Ritual

#

The merge button exists because our tools gave us a button. We built a workflow around it, assigned it meaning, and then treated the meaning as structural.

It is not. The safety of a codebase is produced by the systems around the merge, not by the merge approval itself. The click certifies almost nothing those systems did not already certify. And it extracts a real cost: latency, batched risk, fractured focus, concentrated bus factors, and a soft target for anyone who wants to slip something through.

The defensible position is not “no humans in the loop”. It is “humans in the loop where they add value, out of the loop where they do not”.

Most pull requests do not need a human to approve them. A small, identifiable minority do. The mistake of the current default is that it treats every change as if it belonged to that minority.

Free the majority to merge on green. Reserve the human gate for the changes where it actually certifies something the machine cannot. And stop pretending the button is what keeps you safe.

The gate was never the safety. The system was.

Code review is a bottleneck. I am no longer convinced it is a useful one.

This is not a conclusion I arrived at lightly. Code review has been one of the most reliable quality gates in software engineering for decades. But the assumption underlying code review, that a human reading code before it ships catches meaningful problems, is worth re-examining when most of that code was written by an LLM.

What Code Review Was Supposed to Do

#

Code review served several purposes simultaneously.

It caught bugs before they reached production. It enforced consistency across a codebase. It spread knowledge between team members. It forced the author to organize their thoughts before presenting them to a peer.

Each of these purposes assumed something important: that the person who wrote the code and the person reviewing it were both human, that the author had thought carefully about each line, and that the reviewer could rely on the author’s intent as context.

When an LLM writes the code, these assumptions break.

The LLM did not think carefully about each line. The LLM does not have intent in the way a human does. The reviewer cannot ask the author “what were you trying to do here?” and get a meaningful answer, because the author is a statistical model that generated the most probable next token.

The loss of a human author changes the nature of the review fundamentally, and not in the direction most people assume.

Why Reviewing LLM Code Is Different

#

When a human writes code, code review is a conversation between two people who share a mental model. The reviewer can trust that the author made deliberate choices, even imperfect ones. Differences between what the reviewer expects and what the code does are interesting signals, because they represent a gap between two human understandings of the same problem.

When an LLM writes code, there is no shared mental model. The code is the output of a pattern-matching process. Sometimes it is correct. Sometimes it is subtly wrong in ways that look correct. Sometimes it is obviously wrong.

The reviewer’s job shifts from “does this match the author’s intent?” to “does this do what I want?” This sounds like the same question, but it is not. The first question allows the reviewer to leverage the author’s reasoning. The second question requires the reviewer to independently verify every assumption the code makes.

This is harder, more tedious, and less effective than reviewing human-written code. The reviewer is not building on the author’s thinking. They are reconstructing it from scratch, line by line.

The Bottleneck Argument

#

Here is the practical problem.

LLMs can generate code orders of magnitude faster than humans can review it. A developer who used to spend six hours implementing a feature might now spend thirty minutes prompting an LLM and one hour reviewing the output.

The ratio of review time to implementation time has inverted. Where review used to be a small fraction of the development cycle, it is now the dominant fraction.

And the quality of that review is worse, not better, because reviewing code you did not write is cognitively different from reviewing code you understand deeply.

This creates a specific kind of bottleneck: one where the throughput-limiting step is also the lowest-quality step. You are spending most of your time on the part of the process where you add the least value.

What I Would Rather Be Doing

#

If I am going to spend my limited cognitive budget, I want to spend it on the decisions that matter most.

Deciding which problems to solve. Understanding whether a feature should exist at all. Designing the boundary between components. Choosing the right abstraction for the domain. Thinking about how users will actually interact with what we build.

These are high-leverage activities. They determine whether the code that gets written is useful, not just correct.

Reviewing LLM-generated code for style, naming conventions, and obvious bugs is low-leverage work. An automated tool can do it faster and more consistently than I can. A linter does not get tired. An automated test suite does not lose focus after the third function.

The hours I spend reviewing code that an LLM wrote are hours I am not spending on the problems that only a human can solve. That trade-off did not used to exist, because writing code and reviewing code were both human activities and the time allocation was roughly balanced. Now the balance is broken, and the opportunity cost of review is much higher.

Why Automatic Review and Approval Make Sense

#

I have no objection to automated code review. I have no objection to automated PR approval.

The instinct to keep a human in the loop for every change is driven by fear, not by a rational assessment of what the human actually contributes at that point in the process.

Consider what a good code reviewer does today. They check that tests pass. They look for obvious bugs. They verify naming conventions. They ensure the change aligns with the stated goal.

Every one of these checks can be automated. Tests pass or they do not. Static analysis tools catch bugs more reliably than tired humans scanning diffs. Linters enforce style more consistently than any reviewer. Alignment with the stated goal can be checked by having an LLM compare the PR description with the actual changes.

The Wrong Stage to Catch Subtle Issues

#

The remaining argument for human review is that humans catch subtle issues that automated tools miss: architectural problems, subtle security vulnerabilities, misunderstandings of the domain.

These are real concerns, but they are not best addressed at the PR level. They are best addressed at the specification level. If you write a precise specification, automated verification can confirm the implementation matches it. If the specification is vague, no amount of human review will save you from building the wrong thing.

Some would argue that catching these issues during review is better than not catching them at all. This is hard to disagree with in isolation. Of course a subtle bug caught at review is better than the same bug reaching production.

But the better-late-than-never framing hides the real trade-off. The question is not whether review catches some problems. It is whether review is the best place to catch them, and whether the time spent reviewing could catch more problems if spent elsewhere.

When you catch an architectural flaw at review time, the code is already written. Fixing it means rework, rebase, re-test, re-review. Catching the same flaw during specification costs a conversation. The later you catch it, the more expensive it is, and code review is one of the latest stages in the pipeline.

More importantly, relying on review to catch subtle issues is unreliable by design. A human reviewer catches what they happen to notice, when they happen to be alert, on the changes they happen to read carefully. Some issues get caught. Many do not. You are depending on luck and attention, not on a system.

The alternative is to build the catching into the system itself. A precise specification catches architectural misunderstandings before code exists. A comprehensive test suite catches behavioral bugs on every run, not just when a reviewer is paying attention. Static analysis catches security patterns deterministically. Each of these catches issues systematically, on every change, forever.

Review catches issues once, for the reviewer who happens to be in front of the diff. A good test catches the same class of issue every time it runs, for as long as the codebase exists. If you find the same kind of subtle issue during review more than once, the answer is not to keep reviewing harder. The answer is to encode that check into your automated gates so it never depends on a human noticing again.

“Catching it late is better than not catching it” is true. It is also an argument for accepting a process that catches too little, too late, at the highest possible cost.

The review should happen before the code is written, not after. Spend the human effort on the spec. Let the machines verify compliance.

The Quality Maximization Myth

#

Some would argue that this misses the point. The goal of code review is not just to check gates, it is to make the code the best it can be. A good reviewer suggests a cleaner abstraction, spots a performance issue the author missed, proposes a name that communicates intent better. Review is quality maximization, not just verification.

This sounds right until you press on what “the best it can be” actually means.

It is subjective. It is unbounded. There is always a cleaner abstraction, a faster algorithm, a better name. The pursuit has no natural stopping point, which is why code reviews so often devolve into bikeshedding over style preferences that do not measurably improve the outcome.

When the code was written by a human, a second perspective genuinely improved the implementation. Two brains could find a better approach than one. But when the code is generated by an LLM, the reviewer is the only brain in the loop, and their suggestions compete with the option of simply regenerating the code against a better specification.

If the code is not good enough, the answer is not to have a human improve it line by line during review. The answer is to improve the specification, the test suite, or the generation prompt, and let the machine produce a better version. That scales. Human suggestions on a diff do not.

The quality of the code is bounded by the quality of the specification that produced it. If you want better code, write a better spec. Reviewing the output is the most expensive, least scalable way to improve it.

The Reallocation

#

Here is what I am proposing.

Instead of spending 30% of development time on code review, spend 5% on automated verification and reallocate the saved hours to specification, problem selection, and domain understanding.

The quality of the software will not decrease. The tests still run. The linters still lint. The static analysis still analyzes. What changes is where the human attention goes.

Right now, human attention is concentrated at the end of the pipeline, reviewing output. It should be concentrated at the beginning of the pipeline, defining what the output should be.

This is not a radical idea. It is the same principle behind test-driven development: define what you want first, then build it. The difference is that now the builder is a machine, and the definition is the only place where human judgment is irreplaceable.

This Is Not Hypothetical

#

None of this is a prediction about the future. I run this approach today.

My development workflow front-loads human effort into specification, requirements, and problem selection. Before any code is written, there is an issue with acceptance criteria, a requirements document, and a technical specification, each reviewed and approved. That is where my judgment gets spent.

Code generation happens against that specification. When a pull request is opened, an automated review pipeline checks it against concrete gates. Does it introduce security-sensitive changes? Does it add new dependencies? Does it change public interfaces? Is the change reversible? Do the tests pass? Does the diff actually satisfy the acceptance criteria from the linked issue?

If all gates pass, the PR is approved automatically. No human reads the diff. No human clicks approve.

When the gates flag something, the PR is held for manual review. This is not the same as reviewing every change. It is reviewing the changes that carry real risk, which is a much smaller set.

The distinction matters. I am not advocating for shipping unreviewed code. I am advocating for shipping code that has been verified by systems more reliable than a tired human scanning a diff, and reserving human attention for the small fraction of changes where it actually adds value.

The infrastructure for this exists. The gates are concrete. The approach works because the specification does the heavy lifting that code review used to do poorly. The skills that implement this workflow are publicly available.

Who Is Responsible

#

The first objection is accountability. If no human reads the code before it ships, who owns the problems it creates?

The real answer is that a human reviewing a pull request for ten minutes was never truly responsible for that code. They provided a rubber stamp. They glanced at the diff, checked that the tests passed, and clicked approve. When that code caused an outage six months later, nobody blamed the reviewer. They blamed the author, the test suite, the deployment process, or the requirements.

What about the reviewer who spends thirty minutes, or an hour? They are doing substantive work, not rubber-stamping. The characterization above does not apply to them. They genuinely understand the change, question the design, and catch real issues.

But the argument against mandatory review does not depend on reviews being shallow. It depends on where that hour of expert attention is best spent.

An hour of review catches issues once, for one change, depending on that reviewer being sharp that day. An hour spent improving the specification prevents the entire class of issue from reaching implementation. An hour spent writing a regression test catches the bug on every future run, not just the one time a human happened to read the code.

The thorough review is real work. It is just not the highest-leverage work that person could be doing with that hour. And at scale, it is unsustainable: if every PR requires an hour of human review and the LLM produces ten PRs a day, you need ten hours of review to keep up. That is not a process that scales.

Code review creates an illusion of accountability without delivering it. The signature on the PR is accountability theater.

Real responsibility lives upstream. The person who decided this problem was worth solving owns the outcome. The person who wrote the specification owns whether the implementation matches intent. The person who designed the deployment pipeline owns how quickly a bad change can be contained.

Removing human review does not remove responsibility. It forces you to locate responsibility where it actually belongs: in the decisions that guided the work, not in the person who scanned it at the end.

What About Outages

#

The second objection is safety. What if the LLM makes a decision that takes down production?

This is a real risk, but code review is the wrong tool to mitigate it.

Most production outages are not caused by bugs that a reviewer would catch. They are caused by configuration changes, unexpected data formats, load patterns, dependency failures, and integration issues that only surface under real traffic. A human reading a diff is making a guess about what might happen. Production behavior is the ground truth.

If you want to prevent outages, invest in the systems that observe and contain actual behavior.

Feature flags so a change can be turned off without a redeploy. Canary deployments so a bad change reaches 1% of traffic before it reaches 100%. Monitoring and alerting so a regression is detected in minutes, not hours. Fast, tested rollback paths so recovery does not depend on someone remembering how the old version worked.

These tools are more reliable than code review because they operate on reality rather than prediction. A reviewer might miss a subtle interaction. A canary deployment will surface it.

What About Reversibility

#

A related concern is that an LLM might produce changes that are hard to undo. A sprawling refactor, a database migration that is not backward compatible, a change that entangles two previously independent systems.

This is a serious problem, but again, human review is not the safeguard people think it is.

Reviewers focus on forward correctness. They ask “does this do what it should?” They rarely ask “can we undo this cleanly?” Even when they do, reversibility is hard to assess by reading a diff. It depends on what happens in production after the change lands, on data migrations that have already run, on other changes that build on top of it.

The solution is to make reversibility a structural property of how changes are made, not a property enforced by review.

Keep changes small and independent. Require database migrations to be backward compatible, deployed before the code that depends on them. Use feature flags so new behavior can be disabled without reverting code. Treat large, entangling refactors as high-risk changes that warrant extra process, not as the default mode of operation.

These are constraints you encode in your pipeline and your specification process. They do not require a human to read every line of every PR. They require discipline at the level where decisions are made.

What About Code Quality and Maintainability

#

A third concern is that without human review, LLM-generated code will degrade into slop. Inconsistent naming, duplicated logic, unnecessary abstractions, dead code, patterns that do not match the rest of the codebase. Each individual change passes its tests, but the codebase slowly rots.

This is a real risk, and it deserves a real answer. The answer is not that review prevents it. The answer is that review catches it inconsistently, after the fact, one PR at a time.

Most of what we call code quality is measurable. Cyclomatic complexity is a number. Duplication is detectable. Dead code is identifiable by static analysis. Naming conventions are enforceable by linters. File length, function length, import depth, test coverage gaps, all of these are machine-checkable properties.

If you care about maintainability, encode the constraints that produce it. Set complexity limits that fail the build. Run duplication detectors on every PR. Require test coverage above a threshold. Lint aggressively. Block PRs that introduce unused exports or dead code.

These checks run on every change, consistently, without getting tired or distracted. A human reviewer might flag a function that is too complex. A complexity gate will flag every function that exceeds the threshold, every time, for as long as the rule exists.

Slop also has an upstream cause. When the specification is vague about architecture, naming, and patterns, the LLM fills the gap with whatever it has seen most often in its training data. That output is generic by default. It will not match your codebase’s conventions unless the specification tells it what those conventions are.

This means the fight against slop is won at the specification level, not at the review level. A specification that includes the patterns to follow, the existing abstractions to reuse, and the naming conventions to respect produces cleaner code than a vague spec plus a human reviewer cleaning up the output.

For what slips through the automated gates, frequent maintenance sweeps catch accumulated decay. Run a dead code analysis weekly. Run a duplication detector weekly. Review complexity trends after every merge. This is a more systematic approach than hoping each PR’s reviewer notices the slow accumulation of mess.

Code quality is a property of your constraints, not of your reviewers.

What I Am Not Saying

#

I am not saying all code review should be eliminated tomorrow. Legacy codebases, critical security infrastructure, and domains where correctness is life-or-death may still benefit from human review.

I am not saying code review was never useful. It was, for decades, one of the best tools we had. I am saying the tool’s value has changed because the context has changed.

I am not saying I trust LLMs to always produce correct code. I am saying that human code review is not the best way to ensure correctness when the code was machine-generated. Automated verification, comprehensive test suites, and precise specifications are better tools for that job.

The Question Worth Asking

#

The next time you open a pull request full of LLM-generated code and start reviewing it line by line, ask yourself: what am I actually checking?

If you are checking style, a linter does it better. If you are checking correctness, tests do it better. If you are checking whether the code solves the right problem, you should have answered that question before the code was written.

Code review made sense when humans wrote all the code. It makes less sense when humans define the problem and machines implement the solution. The bottleneck has moved. Our processes should move with it.

In Exponential growth software company I explored the constraints that make scaling a human company hard: onboarding bottlenecks, culture dilution, coordination overhead, institutional knowledge decay. Each of these constraints grows with the number of people you employ. A company where the workforce is entirely composed of LLM agents inverts most of these problems. The old constraints evaporate, but new ones emerge from a different direction.

What Disappears

#

Onboarding becomes instantiation. An LLM agent starts productive the moment it is created. There is no training period, no ramp-up, no senior employee pausing their work to bring a new hire up to speed. Spawning 100 agents costs roughly the same operational effort as spawning 1. The ratio of onboarded to onboarders that mattered for human companies becomes irrelevant.

Culture is exactly what you specify. Agents follow their instructions. If you want 1,000 agents to behave identically, you give them the same system prompt. There is no culture clash, no informal norms developing in opposition to the official ones, no gradual drift as new hires bring outside habits. The culture is the prompt.

Coordination scales differently. Agents do not have cognitive limits on the number of peers they interact with. A single orchestration agent can coordinate with hundreds of worker agents without getting overwhelmed. Communication between agents is structured, lossless, and near-instantaneous. The team structures, single points of contact, and redundancy tradeoffs that human organizations require are replaced by protocol design.

Institutional knowledge does not decay. Everything an agent knows is explicit in its instructions, tools, and retrieved context. There is no tacit knowledge locked in someone’s head, no risk of losing the person who understands the legacy system. When you replace an agent, the replacement has access to exactly the same information.

What Replaces It

#

The fundamental principle from the original article still holds: anything that scales linearly or superlinearly with itself needs to be optimized to grow sublinearly. The bottleneck has moved, but it has not disappeared.

Unit economics

#

Each agent invocation costs compute and API calls. At scale, the marginal cost of an additional agent is small but not zero. If your revenue per task is lower than the cost of the compute required to complete it, you have the same fundamental problem as an unprofitable human company. A human company that loses money on every employee-hour goes bankrupt. An agent company that loses money on every inference call goes bankrupt just as fast, it just happens in milliseconds instead of months.

Correlated failures

#

With humans, errors are diverse and partially self-correcting through independent judgment. Two engineers given the same task will produce different bugs. With agents sharing the same prompt, errors are correlated and systemic. A single flawed instruction propagated across 1,000 agents produces 1,000 instances of the same mistake at scale. Consistently wrong is worse than inconsistently right. Correlated errors make quality control the central bottleneck. You need evaluation pipelines, guardrails, and feedback loops that themselves must scale. The testing infrastructure becomes the company’s most critical asset, more important than the agents it tests.

Orchestration complexity

#

While individual agents do not get overwhelmed, the system as a whole can still produce emergent failures. Agents acting on stale information, conflicting instructions, or subtle misinterpretations of their goals can cascade into problems that are hard to diagnose because the system operates at a speed and scale humans cannot directly oversee. The orchestration layer is the new management layer. Its complexity grows with the number of agents and the richness of their interactions. Unlike human management, where adding managers adds judgment and adaptability, adding orchestration logic adds rigidity. Every new rule is a new potential point of failure.

Infrastructure brittleness

#

A human can work around a broken tool with creativity. An agent generally cannot. If an API goes down, every agent depending on it stalls. If a database schema changes unexpectedly, every agent writing to it produces corrupted data. The brittleness of automated systems means that reliability engineering and observability become more important, not less, as you scale. The company that cannot detect a degrading agent within seconds will compound the damage across its entire workforce simultaneously.

Model dependency

#

Your company’s capacity is bounded by what the underlying models can do. If the model provider changes behavior, degrades performance, or raises prices, your entire workforce is affected at the same time. This is a vendor dependency unlike any single human employee leaving. It is more like all your employees sharing the same brain, and that brain being operated by a third party. Diversifying across models is a partial hedge, but it introduces the same coordination complexity as a multilingual workforce.

Where the Moat Lives

#

When anyone can spawn an equally capable agent workforce, the advantage is no longer in having employees. The moat shifts to the quality of your instructions (prompts as institutional knowledge), the design of your orchestration (workflows as management), your data flywheels (evaluation data as competitive advantage), and domain-specific tools and integrations (proprietary capabilities the agents use). The company is no longer its people. It is its prompts, its pipelines, and its proprietary context.

The Pattern

#

The original article concluded that human scaling requires optimizing every linear cost down to sublinear. The same conclusion applies here, but the costs are different. Human companies optimize hiring, onboarding, and culture. Agent companies optimize inference cost, error correlation, orchestration complexity, and infrastructure reliability. The companies that scale exponentially in this era will be the ones that treat their agent workforce as a system to be engineered, not a team to be managed.

References

#

• Exponential growth software company – original article on human company scaling constraints
• The Shifting Bottleneck – how AI moves bottlenecks up the decision chain
• Software Engineering Teams in the Age of AI – team design implications of AI-assisted development Audience notes: Assumes familiarity with LLM agents and basic organizational scaling concepts.

I’ve been building AI agents that review pull requests, and the official gh CLI doesn’t have what they need. Agents can’t leave inline comments on specific lines, can’t manage pending reviews, and can’t edit or delete comments. These are table-stakes operations for any code review workflow, and they’re only accessible through the GitHub web UI or the raw GraphQL API.

So I built ghx, a CLI designed to make agentic code reviews practical.

What ghx does that gh doesn’t

#

Inline comments on files and lines. The most fundamental operation for a code review agent: comment on a specific line of a diff.

$ ghx pr comment 42 --file src/main.go --line 10 --body "Nit: use fmt.Errorf"
Created inline comment on src/main.go:10 (thread PRRT_kwDOC0I7As5vKgVn)

$ ghx pr comment 42 --file src/main.go --line 10-15 --body "Consider extracting this"
Created inline comment on src/main.go:10-15 (thread PRRT_kwDOC0I7As5vKgVq)

File-level comments (without --line), top-level PR comments, and replies to existing threads are all supported.

Pending reviews. Accumulate review comments without submitting them immediately, then approve or comment when ready:

$ ghx pr comment 42 --file src/main.go --line 10 --body "Nit" --pending
Added pending inline comment on src/main.go:10 (thread PRRT_kwDOC0I7As5vKgVz, review PRR_kwDOC0I7As4B9Y2z)

$ ghx pr review submit 42 --event APPROVE --body "LGTM"
Submitted review PRR_kwDOC0I7As4B9Y2z as APPROVE

Edit and delete comments. Fix a typo or remove a comment. Use ghx pr threads 42 --ids to list IDs:

$ ghx pr threads 42 --ids
PRRT_kwDOC0I7As5vKgVn  src/main.go:10  [open]
  PRC_kwDOC0I7As5TKxYc  reviewer  Nit: use fmt.Errorf

  PRC_kwDOC0I7As5TKxYd  author  Good catch, will fix.

Then edit or delete the comment:

$ ghx pr comment edit PRC_kwDOC0I7As5TKxYc --body "Use fmt.Errorf instead of errors.New"
Updated comment PRC_kwDOC0I7As5TKxYc

$ ghx pr comment delete PRC_kwDOC0I7As5TKxYc
Deleted comment PRC_kwDOC0I7As5TKxYc

Review thread management. List, filter, and inspect review threads:

$ ghx pr threads 42
src/main.go:10  [open]
  reviewer  Nit: use fmt.Errorf
  author    Good catch, will fix.

src/main.go:45-52  [resolved]
  reviewer  Consider extracting this into a helper
  author    Done in 3a1b2c4

Filter by state with --state open, --state resolved, or --state all.

Issue comments and viewing. Add, edit, and delete issue comments, and view issues with their full comment history:

$ ghx issue view 42
Fix race condition in worker pool  [open]  author

The worker pool has a race condition when multiple goroutines access
the shared counter without proper synchronization.

2 comment(s):

  contributor  I can reproduce this with `go test -race ./...`
  author       Fixed in #50

$ ghx issue comment 42 --body "This is fixed in #50"
Created comment IC_kwDOC0I7As5TKxZa on issue #42

The stash system

#

ghx has a local stash system for review comments, modeled after git stash.

The main use case is enabling agents to batch many comments at once before submitting them all as a single review. Instead of making individual API calls per comment, an agent can stash comments locally and pop them into a pending review in one operation:

$ ghx pr comment 42 --file src/main.go --line 10 --body "Nit" --stash
Stashed comment on src/main.go:10 (stash@{0} now has 1 threads)

$ ghx pr comment 42 --file src/main.go --line 20-25 --body "Extract this" --stash
Stashed comment on src/main.go:20-25 (stash@{0} now has 2 threads)

$ ghx pr review stash list 42
stash@{0}:  2 threads, 2 comments
  src/main.go   10      1 comment(s)
  src/main.go   20-25   1 comment(s)

$ ghx pr review stash pop 42
Popped stash@{0} (2 threads, 2 comments) into review PRR_kwDOC0I7As4B9Y2z

The stash also solves a GitHub API constraint: you can’t mix immediate comments with pending review comments on the same PR. When you submit an immediate comment on a PR that has a pending review, ghx automatically stashes the pending review, submits the comment, and restores the pending review. The stash lets agents use a push-pop workflow (stash, comment, comment, pop) instead of repeating push-comment-pop for every immediate comment.

The stash supports multiple entries, just like git stash: push, pop, drop, and list.

Getting started

#

ghx is a single Go binary. Download the latest release from GitHub, or install with Go:

go install github.com/tomzxcode/ghx@main

ghx picks up your existing GH_TOKEN, GITHUB_TOKEN, or gh auth login credentials. No extra configuration needed. All commands accept --repo OWNER/REPO or auto-detect from the current git remote.

ghx is MIT-licensed and written in Go with no runtime dependencies beyond the GitHub API.