Below the Surface

We trained a language model so you can watch it pick each word

hello@squidtrain.com (SquidTrain) — Sun, 04 Oct 2026 20:46:19 GMT

Every AI model you use at work writes the same way: it looks at the text so far, scores every possible next piece, picks one, and repeats. We wanted people to see that happen, with a real model they can poke at. So we trained one.

On October 4, 2026, SquidTrain published Token Lab, a small language model we trained ourselves on 144 public-domain books and an interactive page that runs it entirely in your browser. You type a sentence and watch the model read it as tokens, score its options for the next one, pick, and keep going. You can change how it chooses, see which earlier words it looked at, and scrub back through snapshots saved during training to the moment it knew nothing.

Try Token Lab. Nothing you type leaves your browser.

What can you do in Token Lab?

The page is built as a set of panels, and each one shows a step the large commercial models also take.

What the model reads. Your text is split into numbered pieces called tokens. The model never sees letters or words, only those numbers. "Sherlock" becomes three pieces. A context bar shows how much of the model's 256-token window your text fills.

What comes next. For the current position, the page lists the most likely next tokens with their odds. Press "Next token" to add one at a time, or "Write 40 tokens" to let it run.

How it chooses. Three controls change the pick. Temperature reshapes the odds: at zero the model always takes the top choice and soon repeats itself, and higher values let unlikely words through. Top-k and top-p cut the list down before the pick. This is why the same prompt can give a different answer every time you ask.

Where it looked. Click any token to see which earlier tokens the model paid attention to when it was there, layer by layer and head by head. Each head learned on its own what to track, and you can compare them.

Watch it learn. We saved real snapshots while the model trained. Drag the slider back to step 0 and the sample text is gibberish. A few hundred steps in, it has words. By the end it has grammar, names and the style of the books it read. A chart shows the error on text the model never saw during training, falling as it learns.

Model size. Switch between three models trained on the same books, from under 1 million to 40 million parameters, and see how each one continues your sentence.

How did we build it?

We built Token Lab with Claude Opus 5.5, which wrote the training code, the browser engine, and the page. The training itself ran on one office PC with an NVIDIA RTX 3090 graphics card.

The books. We used 144 English books from Project Gutenberg, all in the public domain, including Pride and Prejudice, the Sherlock Holmes stories, Frankenstein, Dracula and Moby Dick. That came to about 31 million tokens. We stripped the Gutenberg headers and footers, dropped non-English text, and removed sentences containing racial slurs that are common in books of that era. The page runs the same filter on anything the model writes.

The tokenizer. We trained our own, with a vocabulary of 4,096 tokens. Common words like "the" get a single token. Rarer words are built from pieces, which is why "Sherlock" splits into three.

The models. All three use the same design as the GPT family, at a much smaller scale:

Tiny: 0.95 million parameters, 2 layers, trained in about 3 minutes.
Small: 12.3 million parameters, 6 layers, trained in about 16 minutes. This is the one the page loads first.
Medium: 40.1 million parameters, 12 layers, trained in about 42 minutes.

Each model read the full set of books about 12 times. Training all three took 61 minutes, and the whole run, including downloading the books and building the tokenizer, took 66.

Running it in the browser. The trained weights are compressed to 8 bits and loaded by the page as plain files. A small engine we wrote in JavaScript does the math the model needs, in a background thread so the page stays responsive. We checked it against the original training code: the two agree to within rounding error on every test sentence, so what you see on the page is the real model, step for step.

Why build a small model on purpose?

A small model is fluent and often wrong, and that makes it a better teacher than a large one.

Ask the small model to finish "The capital of France is the city of" and it rates "France" at 24%, "England" at 14%, and "Paris" at 6% (at the page's starting temperature setting). It has read plenty of sentences shaped like that one, so it knows a place name comes next. It has not read enough to know which one is true. It is predicting what usually comes next, and the right answer is only one of the likely options.

The models you use at work run on the same mechanism with vastly more data and parameters, so they get "Paris" right. But they still produce answers by choosing likely pieces, one at a time. When a large model is wrong, it is wrong in the same fluent, confident voice it uses when it is right. Seeing that at a small scale, where the odds are on the screen, makes it much easier to understand when you meet it at a large one.

What does it teach about the AI you already use?

A few lessons we come back to in almost every training session become obvious after ten minutes with Token Lab.

Different answers to the same question are normal. The model picks from a list of likely options, and the temperature setting decides how adventurous that pick is. If you need the same answer twice, you need a process that checks it, since asking again will not always give you the same one.

Confidence is a writing style. The model does not know when it is wrong. A fluent answer tells you the words are likely. Whether they are correct is a separate question. Check facts, figures and names against a source.

Context has a limit. Our model can see 256 tokens. Commercial models can see far more, but every model has a window, and anything outside it does not exist for the model. Long conversations and long documents eventually push the beginning out of view.

Tokens are the unit of everything. Context windows and API prices are counted in tokens, and many usage limits are too. Once you have seen a word split into pieces, those numbers make more sense.

How does this fit what SquidTrain does?

SquidTrain is an AI training and software development company. We train companies, teams and individuals to use AI on the work they already do, and on the rest of life too: planning, writing, learning and helping kids understand the tools they are already using. Builds like Token Lab are part of how we teach. People make better decisions about AI once they have seen how it works, and an interactive model beats a slide about one.

The build also shows the second half of what we do. We built the data pipeline, the model code, a custom engine, and a finished web page with Claude Opus 5.5, and ran it all on hardware already in the office. When training alone will not solve a problem, we build the tool.

If you want your team to understand the tools they are using, or you have a problem that might need a build of its own, see our training or book a free call. And the Lab has more builds to try.

The books in Token Lab come from Project Gutenberg and are in the public domain in the United States. The full list is on the Token Lab page under "The books it read."

Bill Gates was stunned three times

steven@squidtrain.com (Steven Van Solkema) — Fri, 04 Sep 2026 16:00:00 GMT

The idea that now these things are in a meaningful sense better than I am is like: What the hell? - Bill Gates

That's Bill Gates, in the New York Times on August 26, talking about Claude Code. I read it twice. Then I put the paper down (fine, the laptop) and went back to the terminal window I'd left open, where that same tool was halfway through refactoring something for me.

On August 26, 2026, Bill Gates published a GatesNotes essay warning that the AI era will be "one of the most turbulent times in human history," and gave an interview to the New York Times in which reporter Karen Weise wrote that technology had truly stunned Gates three times: the graphical user interface in 1980, the OpenAI demo at his house in 2022 that became ChatGPT, and Claude Code, Anthropic's coding tool, in 2026.

I want to be careful with that list, because it has already been passed around a little loosely. The "three times" line is Weise's reporting of what Gates told her, and the Claude Code quote above is Gates's own words in that interview. Gates himself put it more bluntly on the GeekWire podcast three days later: "I was shocked by ChatGPT, and I was shocked by Claude Code. Those are both things where I went, oh my God."

So the most successful software person of the last fifty years has had three of those moments. I have had exactly one. It's the same one he had this year, and I've been living inside it every working day since.

What did Bill Gates actually say about Claude Code?

Coding, Gates told the Times, is the subject he knows best, and Claude's abilities shocked him. In the MIT Technology Review the same week he got specific: "I was stunned at the coding. Claude code, the context buffer, the agentic approach, just the model underneath."

Notice what he did not say. He did not say it writes nice functions. Every model has written nice functions for a couple of years. He named the context buffer and the agentic approach. That's the part that matters, and it's the part that most people who have not sat down with it for an afternoon still miss.

Here is the plainest way I can put what changed. The old pattern was: I describe a task, the model produces text, I copy it somewhere and find out whether it works. The new pattern is: I describe an outcome, and the tool reads the files, decides what to look at next, runs the thing, reads the error, fixes it, and comes back to me with a result and a question. I am still deciding. I am just not typing.

I came to this from trading operations, where being surrounded by systems that are faster than you at a specific job is a normal Tuesday. Nobody in an options business feels threatened by a pricing engine. You feel threatened by not understanding what it did. That framing has served me well here. Whether the tool is "better than me" at producing code stopped being an interesting question a while ago. Whether I can see what it did, and stop it when I should, is the one I get paid to answer.

Why a warning from Gates is good news for small teams

Most of the coverage led with the fear, and the fear is real. Gates wrote that "the smartest cybersecurity experts I know are scared about the next few years." He told the Times that in private, "people who understand how good this stuff is, and how much better it's getting, they're very worried," and that the industry is downplaying it. On self-regulation: "No, thanks!"

First, own the thing. Gates's turbulence is a story about concentrated power and a missing plan. The practical translation for a team like ours, or like yours, is to stop renting capability you cannot inspect. That's why we license tools, agents, skills and prompts with source included, running on your infrastructure. It sounds like a sales line (it's also on our products page, I'll admit that), but it comes from the same place as Gates's worry. If the tool is going to be "better than you" at a chunk of your work, you had better be able to open it up.

Second, decide where the human sits before the agent runs, and write it down. We wrote a whole post on this earlier this year, and the Gates interview made me want to reprint it. Oversight placed according to consequence is the difference between a tool that shocks you in the good way and a tool that shocks you at 3 a.m.

Third, take the attack surface seriously. The same connectivity that makes an agent useful (it can read your email, your files, your ticketing system) is what makes it dangerous, and Gates is right that defenders are behind. Our post on the Model Context Protocol goes into the specifics. Read it before you wire anything to production.

The personal part

I said I've had one of these moments. Here's what it was, for the record.

Demos are easy to be impressed by, and I have watched a lot of them throughout my career in finance and technology, so mine came somewhere less dramatic. It was the first week I stopped opening a code editor first thing in the morning and started opening Claude Code instead, and then I realized, only two days later, that the shift had happened without a decision.

Gates said something on GeekWire that I keep repeating to people: "I joke with people that I used to have Claude-like people that I would send email to, but they were so slow, and there were some topics they didn't actually know." I spent a career sending those emails. Now the reply comes back in ninety seconds, with the diff attached, and the honest reaction is exactly the one Gates had. Followed, about a minute later, by the second reaction, which is the one I'd like more people to have: OK ... so what do I check, and who is accountable when it's wrong.

That second reaction is the whole business we're building. Seriously. The tools and the training days are just the delivery mechanism ... the product is the habit of following the "what the hell" with "what do I check."

Gates said he rarely stops thinking about AI, "not because I have all the answers, but because the questions it raises are too consequential to leave to a small group of technologists." I'd put a smaller version of that on our door. The questions are too consequential to leave to the vendors, either. Open the tool. Look at what it did. Then decide.

He's been stunned three times in forty-six years. I don't know if I get a fourth. I'm fine with three.

Sources: Karen Weise, "Bill Gates Warns A.I. Is More Dangerous Than Big Tech Will Admit," The New York Times, Aug. 26, 2026. Bill Gates, "The turbulent AI era is here. The choices we make now are critical," GatesNotes, Aug. 26, 2026. GeekWire Podcast, "Bill Gates in his own words," Aug. 29, 2026. Mat Honan, MIT Technology Review, Aug. 26, 2026.

Header photo: Gisela Giardino, CC BY-SA 2.0, via Wikimedia Commons.

How MCP widened the AI attack surface

logan@squidtrain.com (Logan Matson) — Thu, 23 Jul 2026 16:00:00 GMT

What the Model Context Protocol solved

The move from isolated chatbots to integrated agents was enabled by a connectivity standard. Anthropic released the Model Context Protocol in November 2024 and donated it to the Agentic AI Foundation under the Linux Foundation in December 2025. It runs over JSON-RPC 2.0 and defines how an AI application discovers, invokes, and exchanges data with external resources.

Before that standard, connecting a set of agents to enterprise tools meant writing bespoke point-to-point integration code, with custom authentication, error handling, and data formatting for every agent-tool pair. The protocol resolves that N times M problem: build one server per system, and any compatible client can reach it.

Adoption followed quickly. By early 2026 the protocol had passed 97 million monthly SDK downloads across more than 17,000 public servers, and OpenAI, Google DeepMind, and Microsoft had all shipped support within thirteen months of launch. That infrastructure is what makes a market in licensed tools, agent skills, and prompt packages possible at all.

What it expanded

The same connectivity widened the supply chain risk surface considerably, because the protocol inverts the usual direction of network interaction. Instead of human clients requesting data from controlled servers, servers expose tool metadata and execute actions on behalf of non-deterministic AI clients. The NSA’s guidance on this is blunt: the protocol shipped with a flexible and underspecified design, and its proliferation outpaced its security model.

The consequences showed up fast. Researchers filed more than 30 CVEs against MCP servers, clients, and proxy packages in the first months of 2026. A scan of public address space in early 2026 found nearly 7,000 internet-exposed servers running default transport configurations that permitted unauthenticated remote code execution. The most severe finding, CVE-2025-6514, carried a CVSS score of 9.6 and affected the widely used mcp-remote proxy package.

Prompt injection and tool poisoning

Beyond configuration flaws, the models themselves bring failure modes that no amount of network hardening addresses. Because a language model consumes trusted system instructions and untrusted user input in the same channel, as natural language, it has no reliable way to tell a command from data that merely looks like one.

That gap is exploitable at a distance. If an agent is authorized to read email or scrape a website, an attacker can plant instructions in that external content, an indirect prompt injection. Once the agent processes the poisoned input it may change behavior, exfiltrate data, or take a destructive action, all while appearing to operate normally.

Tool poisoning attacks the discovery layer instead. An attacker who can alter a third-party server’s tool descriptions or parameter schemas can mislead the model during planning, so it misuses a legitimate tool rather than calling a malicious one. Detection is genuinely hard here, which is why work on evidence-based detection for MCP traffic is active.

Signing would help, and it is worth asking for. As of early 2026, though, there is no widely adopted initiative to sign agent skills, which makes provenance a control buyers should demand rather than one they can assume.

Defense in depth

Relying on a foundation model’s built-in alignment filters is not a security posture. Those defenses are routinely bypassed, and treating them as a boundary means having no boundary.

The architectural answer is to stop asking one model to distinguish instructions from data. The dual-LLM pattern, proposed by Simon Willison in 2023 and now well documented, splits the job: a privileged model plans actions and calls tools but never sees untrusted content, while a quarantined model processes untrusted text and has no tool access at all. Results pass between them as opaque variables the privileged model cannot read. It separates command logic from content by construction rather than by instruction.

The same principle applies to the prompts themselves. A licensed prompt package is a structured asset, not a string of text: role and style instructions, prompt chains for multi-stage tasks, domain-specific components, and audit and guardrail prompts that check consistency and facts. Treated that way, prompts get versioned, reviewed, and tested like any other software component, and they carry their own input validation and output schema enforcement.

At the network layer, a gateway centralizes what would otherwise be reimplemented per server: identity mapping against an existing provider, role-based access control, agent isolation, and an audit trail on every tool call. The build-versus-buy question there turns on governance speed rather than engineering cost, since audit logging bolted on per server tends to be inconsistent where it exists at all.

None of this is exotic. It is the same discipline applied to any component with production access: bound what it can reach, verify what it produces, log what it did, and assume the input is hostile. The protocol made the connections easy. It did not make them safe.

Prompts are code, so version them like code

logan@squidtrain.com (Logan Matson) — Sun, 07 Jun 2026 16:00:00 GMT

A prompt determines system behavior. It gets edited under time pressure, it breaks in ways that are hard to reproduce, and it is frequently the least controlled artifact in the stack. The same organization that requires two approvals to change a database index will let someone edit a prompt in a web form at four in the afternoon, with no history and no review.

This is not a discipline problem. It is a tooling default. Most platforms present prompts as configuration, and configuration is culturally exempt from the controls applied to code. The exemption made sense when configuration meant a timeout value. It does not survive a prompt that decides which customers get flagged for manual review.

The usual state of things

Prompts tend to live in a config UI, a shared document, or inline in a function that three people have edited.

There is no history, so when behavior changes there is no way to see what changed. Someone reports that outputs got worse last week. You look at the prompt. It contains the current text and no record of any other text it has ever contained. The investigation ends there, or it turns into interviews.

There is no review, so a wording change ships without a second reader. This matters more than it sounds, because prompt edits have non-local effects. Removing a clause that seems redundant can change behavior on inputs nobody was thinking about. A second reader catches the edit that quietly dropped a constraint.

And there is no link between a prompt version and the outputs it produced, so debugging a bad result from last week means guessing. You have the output. You have the current prompt. You do not have the prompt that produced the output, which is the one you need.

What versioning actually buys you

Putting prompts in the repository as files gives you the same properties you already rely on for code.

Diffs show exactly what changed, down to the word. This is more useful for prompts than for code, because prompt regressions are frequently caused by changes that look cosmetic. A reordered instruction list, a softened verb, a removed example.

Review catches the edit that removed a constraint. Blame tells you when a behavior was introduced, which converts an open-ended investigation into reading one commit. Rollback is a revert rather than an attempt to remember the previous wording.

It also makes prompts testable. A prompt in a file can be loaded by a test that runs it against known cases and asserts on the output. A prompt in a text box cannot. This is the property that matters most, because it is what lets you change a prompt without fear.

There is a secondary benefit worth naming: prompts in the repository are visible to the people maintaining the surrounding code. A prompt in a separate system is invisible during code review, so an engineer changing the calling function has no way to see the instructions that function depends on.

A workable structure

Keep each prompt in its own file, named for the task it performs. One file per task, not one file containing every prompt, because a single file makes diffs noisy and blame useless.

Store the model, temperature, and any other parameters alongside it, because those change behavior as much as the text does. A prompt that works at temperature zero and fails at temperature one is not a prompt problem, and you cannot see that if the parameters live in a different system. Keeping them together means one diff shows the whole change.

Record the prompt version with every logged output so a bad result can be traced back to the exact input that produced it. A commit hash or content hash stored on the log line is enough. This single practice converts most prompt debugging from guesswork into lookup.

If the same instruction appears in several prompts, factor it out into a fragment and compose. Duplication drifts, and drifted prompts fail in ways that are hard to attribute: two tasks that should behave identically diverge because someone updated one copy. The usual candidates are output format instructions, tone constraints, and domain definitions.

Resist over-factoring. A fragment used by one prompt is indirection with no benefit, and heavily composed prompts become hard to read as a whole, which matters because the model reads them as a whole.

What to log

Version control answers what the prompt said. Logging answers what happened when it ran. You want both, and the link between them.

At minimum: the prompt version, the model and parameters, the input, the raw output, and whether any validation or retry logic fired. The raw output matters specifically: logging the parsed result discards the evidence you need when parsing is what failed.

Inputs and outputs may contain sensitive data, so this intersects with your retention and privacy rules. Decide the policy deliberately rather than discovering it during an audit. Redaction at write time is usually easier than deletion later.

The objection

The common argument against this is that non-engineers need to edit prompts, and a repository puts them behind a pull request.

That is a real constraint. The people with the domain knowledge to write a good prompt are frequently not the people comfortable with git, and routing every wording change through an engineer is both slow and a poor use of everyone involved.

But it is a workflow problem rather than a reason to abandon version control. A review interface that writes to the repository serves both needs: the editor gets a text box, and the system still gets history, review, and rollback. Several tools do this, and a thin internal one is not difficult to build for a small number of prompts.

The intermediate position, if that is too much to build now, is to keep prompts in files and give non-engineers a documented path to request changes with a named owner who applies them. Slower than a text box, considerably faster than reconstructing history you never kept.

Giving up history is a large price for a convenience that can be built.

Harness engineering and the AI pilot gap

hello@squidtrain.com (SquidTrain) — Sat, 23 May 2026 16:00:00 GMT

The disconnect between experimentation and scaled deployment

Enterprise AI in 2026 presents a paradox. On the surface, the technology is everywhere. 80% of enterprise applications shipped or updated in the first quarter of 2026 embed at least one AI agent, up from 33% in 2024. Gartner projects that 40% of enterprise applications will feature task-specific agents by the end of 2026, up from less than 5% the year before.

Beneath those adoption numbers sits a hard operational divide. While 88% of organizations report using AI in at least one business function, only 31% have an agent actually running in production. And 88% of agent pilots never graduate to production at all.

That failure rate is not a technology problem. It is an architecture problem, compounded by a definitional one. Many organizations fall for what Gartner calls agent washing, where an embedded assistant that depends entirely on human input gets labeled autonomous. A raw language model is a stateless token predictor. It becomes a reliable agent only when wrapped in deterministic software infrastructure, what the field now calls an agent harness. Teams that deploy off-the-shelf wrappers instead of engineering that harness find their agents fail on contact with a real enterprise environment.

From prompts to harnesses

Agent development has moved through three phases, each widening the scope of what you have to control to get a reliable result.

The first phase was prompt engineering, focused on single-turn optimization. Refining instructions with few-shot exemplars and chain-of-thought reasoning pulled better output from the model. This remains a real skill for getting predictable formatting, but it cannot manage evolving state or long-horizon tasks.

The second phase, context engineering, shifted the target to the information lifecycle around multi-step execution. Retrieval-augmented generation and long-term memory management address what enters the model’s context window at each step. But context engineering is feedforward. It optimizes the input and offers no structural mechanism to detect drift, verify intermediate results, or recover from error.

The third phase, harness engineering, treats the entire runtime as the design object. A harness orchestrates tool dispatch, context management, memory budgets, and safety enforcement. It wraps a probabilistic model in deterministic software.

Phase	Optimization target	Problem addressed
Prompt engineering	Single-turn instruction and output format	How to ask the model effectively
Context engineering	Information lifecycle and retrieval	Supplying missing domain knowledge
Harness engineering	Execution stability and error recovery	Preventing drift, rot, and cascading failure

Context rot

One failure mode harness engineering exists to solve is context rot. As an agent works through a long task, calling APIs, generating code, and accumulating reasoning steps, the context window fills with an expanding volume of tokens. As input length grows, attention degrades and output gets less reliable.

The effect is measurable and it depends on position. Accuracy drops from roughly 75% to 55% when a critical fact sits in the middle of a bloated context window rather than near the start. In a production deployment this compounds with every turn, eventually stalling the agent before it finishes the job.

The fix is structural, not a better prompt. Rather than replaying the entire conversation on every turn, a well-built harness maintains state deliberately. Trajectory architectures that decouple file reads from their observations, keeping a synchronized registry of relevant files and injecting only current contents at each cycle, produce measurably lighter trajectories: a 9% to 50% reduction in average input tokens per task and up to 37% fewer reasoning cycles, at comparable pass rates.

Data format matters too. Compact encodings such as CSV or Markdown lists instead of JSON can cut token usage by more than 56% while improving extraction accuracy by up to 5 percentage points.

A taxonomy for production agents

The field has converged on a formal taxonomy for harness design. The ETCLOVG framework describes seven layers that a production-ready agent needs beneath it.

Execution defines where agent code runs and what sandbox bounds it, keeping agents in read-only contexts until outputs pass deterministic quality gates. Tooling specifies the registry of available actions with strict schemas that limit the action space. Context management curates information flow to prevent rot. Lifecycle and orchestration handle continuity across sessions, which matters because state has to survive a pause: when an agent hibernates and resumes, the harness must reconstruct what it was doing without a full human briefing. Observability provides diagnostic output such as token consumption and friction events, so engineers can debug a failure without reconstructing the session from raw logs. Verification checks work before it lands. Governance sets the policies all of it runs under.

The practical takeaway is that an out-of-the-box model is not sufficient for complex operations, and the gap is not closed by prompting harder. It is closed by the infrastructure around the model: how state is managed, how tools are bounded, how failures surface, and how output gets verified before it counts. Skipping that layer is how plausible-looking output reaches production unchecked.

Why most AI pilots stall before production

hello@squidtrain.com (SquidTrain) — Tue, 12 May 2026 16:00:00 GMT

A pilot that works in a notebook and a system that works on a Tuesday afternoon are different engineering problems. The gap between them is where most internal AI projects stop. The pilot is not wrong, and the people who built it are not careless. The two things are simply built against different assumptions, and almost none of the assumptions that hold in a demo survive contact with production traffic.

What follows are the five places projects most reliably stall, and what to do instead.

The demo runs on curated inputs

Pilots are usually built against a handful of examples someone picked by hand. Those examples are clean, representative, and quietly filtered: the ambiguous cases never made it into the set. Nobody did this deliberately. When you are assembling test data, the record that takes twenty minutes to interpret is the one you skip, and the skipped records are exactly the ones that define production behavior.

Production traffic is the opposite. It contains malformed records, edge cases nobody documented, and inputs that are technically valid but semantically strange. A field that is nominally a date arrives as a free-text string in four formats. A document that should be three pages is a hundred and forty, because someone attached the entire contract. An identifier that is unique in the specification turns out not to be.

The failure is not that the model gets worse. It is that the input distribution was never the one you tested against. This distinction matters because the two problems have different fixes. A weaker model is solved by a better model. A distribution mismatch is solved by looking at real traffic, and no amount of model upgrading substitutes for that.

Before automating a task, pull a sample of real inputs. Not a curated sample. Take a window of actual traffic and read it. The point is to find the shapes you did not know existed while the cost of finding them is still low.

Nobody owns the failure path

In a pilot, when the output is wrong, a person notices and tries again. That person is load-bearing infrastructure, and they do not scale. They are also invisible in every demo, because their intervention happens between the run and the screenshot.

Production needs an answer to what happens when the model returns something unusable. There are four options and you have to pick one for each failure mode: retry with modified input, fall back to a deterministic path, escalate to a human queue, or fail loudly and stop. Deciding this late is expensive, because the surrounding system was built assuming success.

The harder question is detection. Retrying is easy once you know the output is bad. Knowing requires a check that runs on every response: schema validation if the output is structured, a constraint on length or format, a rule that catches the specific ways this task fails. An unusable output that passes silently into a downstream system is substantially worse than a crash, because the crash gets attention and the bad record does not.

Escalation deserves particular scrutiny. A human review queue is a reasonable answer only if someone actually owns the queue and it has a defined maximum depth. An unowned queue is a way of deferring failure, not handling it.

Evaluation is an afterthought

Teams that ship tend to build the evaluation harness before the feature. Not a benchmark score, but a set of cases drawn from real traffic with known-correct outputs, run on every change.

Without it, you cannot tell whether a prompt edit helped. You change the wording, spot-check three outputs, and they look better. Whether they are better across the distribution is unknown, and whether the change broke a case that used to work is also unknown. Teams in this position stop editing prompts, because every change is a gamble against invisible regressions.

A workable evaluation set does not need to be large. Thirty to fifty cases covering the common paths and the known failure modes will catch most regressions. What matters is that the cases come from real traffic and that the expected outputs were verified by someone who understands the task. Synthetic cases test whether the system handles inputs you imagined, which is a weaker guarantee than it sounds.

Add every production failure to the set as it is found. This is the mechanism by which the harness becomes valuable over time: it accumulates precisely the cases that broke, so they cannot break the same way twice.

This is unglamorous work and it is the difference between a system you can modify and one you are afraid to touch.

The integration surface is larger than expected

The model call is a small part of the work. Authentication, rate limits, retries with backoff, cost controls, logging, and access to the systems the output has to touch account for most of the engineering.

Pilots skip all of it because a single developer running locally already has the permissions. Their credentials are in the environment, they are the only caller so rate limits never trigger, cost is invisible at demo volume, and if something fails they see the traceback. Every one of those becomes a work item when the system runs unattended.

Rate limits deserve early attention because they fail in a misleading way. Under demo load you never see them. Under production load you see them intermittently, which reads as flakiness rather than a limit, and the instinct is to add a retry. A retry without backoff makes it worse.

Cost is the other item that only appears at scale. A per-call cost that is negligible in testing becomes the line item somebody asks about. Track spend per run from the first day, because retrofitting cost attribution after the fact means correlating logs that were never designed to be correlated.

The task was never defined precisely

This one is upstream of the rest. A pilot can succeed against a vague brief because a human is interpreting the output and filling gaps. Production cannot, because there is no interpreter.

If two people on the team would grade the same output differently, the task is underspecified. That disagreement will surface eventually, usually as a dispute about whether the system is working. Resolve it early by writing down what a correct output looks like, including the cases where the honest answer is that the input does not permit one.

Tasks where the correct answer is genuinely contested are not good first candidates for automation, however appealing the demo looks.

What to do differently

Start from a narrow task that a specific person does repeatedly, and instrument it before you automate it. Watching how the task is actually performed usually reveals that the real work is not the step you planned to automate.

Build the evaluation set from real cases, including the ones that went wrong. Decide the failure path first, for each way the task can fail. Then write the prompt.

The order matters. A pilot built this way is slower to demo and considerably faster to ship, because the work that usually happens between demo and production has already been done.

Where the human sits: oversight and pricing

hello@squidtrain.com (SquidTrain) — Wed, 29 Apr 2026 16:00:00 GMT

Per-seat pricing stopped describing the value

For twenty years B2B software ran on a simple equation: growth meant more employees, more employees meant more seats, and seats meant predictable recurring revenue. Agentic AI breaks the correlation. An autonomous agent does not occupy a seat. When one person with an agent does work that used to take five, headcount stops tracking the value delivered and per-seat licensing stops capturing it.

The market moved accordingly. Pure per-seat pricing fell from 21% to 15% of the market in twelve months. Outcome-based and value-based models took the share, and the shift shows up in how deals get discussed: 73% of clients now prefer value-based or outcome-driven pricing, and 58% raise pricing in the first discovery call. Federal procurement formalized it in Q1 2026, when three of ten named consulting firms offered performance-based fees as an option to retain contracts, the first time outcome pricing appeared inside a federal consulting procurement.

What replaces the seat varies by domain. Fees get tied to measurable results: a percentage of tax savings identified, deals closed, tickets resolved. The common thread is that revenue now depends on the software working in production rather than on access being granted. That is a genuine change in incentive, and it is uncomfortable for vendors whose product demos better than it deploys.

Why adoption is a people problem before it is a technical one

Measurable outcomes require that people actually use the thing, and this is where most deployments stall. Rolling out AI is not like rolling out enterprise software, because the old playbook assumed determinism. An ERP transaction returns the same output for the same input every time. A model returns probabilistic output that shifts with retraining, data drift, and edge cases. Distrust of a system that behaves differently on Tuesday than it did on Monday is not irrational resistance. It is a correct read of the system.

There is a second layer beneath that. Because these systems replicate judgment rather than automating transactions, they land on the part of the job people built a career on. A financial analyst resisting a tool may be resisting the implied message that twelve years of accumulated judgment no longer matters. Treating that as a training gap misdiagnoses it entirely.

Frameworks that work start from the individual rather than the rollout. The Prosci AI Integration Framework has people sort their own work into three buckets, which is a small move with a large effect on how the change lands:

My work. Tasks that stay human because they depend on emotional intelligence, ethical judgment, real-time improvisation, and personal connection. Naming these first establishes what is not up for automation.
With me. Tasks where the model is a collaborator. The person sets direction and the agent helps draft, research, analyze, or generate options. Higher quality output, less cognitive load, human still driving.
For me. Routine rules-based work, such as standardized reports and data organization, that can be delegated outright to free up capacity.

The sorting matters more than the categories. When someone maps their own week, the abstract threat of replacement becomes a concrete and much smaller list of tasks they were mostly glad to hand off.

The same pattern outside the enterprise

Individual adoption has run ahead of institutional adoption. By 2025, 92% of surveyed students reported using an AI tool, up from 66% the year before, and 88% had used one for assessments, up from 53%. ChatGPT led at 66% usage. That is among the fastest adoption curves any education technology has recorded.

The measured outcomes are strong. Students using personalized AI tutoring scored 54% higher on tests, and course completion rates ran 70% better than traditional approaches. A 2025 Harvard physics study found students working with AI tutors learned more than twice as much, in less time, than those in a traditional active-learning classroom.

And confidence still lags the usage, in the same shape it does inside companies. 65% of students worry that leaning on AI makes their learning shallow and discourages critical thinking, and 56% are concerned about data privacy. Only 36% received any AI skills training from their institution. High usage with low confidence and no instruction is not a success state. It is a gap.

Deciding where the human sits

Both settings point at the same design question, which is not whether to keep a human involved but exactly where. Human-in-the-loop design means a qualified person, with adequate context and real authority to act, positioned at the points in a workflow where review changes the outcome.

Placement should be calibrated to consequence rather than applied uniformly. High-consequence actions, such as financial transactions, access changes, external communications, and infrastructure modification, warrant explicit approval before execution. Confidence-threshold escalation routes work for review when the model’s own reliability score falls below a calibrated line. Audit-only checkpoints suit high-volume low-risk operations, keeping a log without gating throughput.

Uniform oversight fails in a predictable way. Ask a person to approve everything and they approve everything, which is automation complacency wearing a governance badge. The failure modes worth designing against are that, unclear ownership when something goes wrong, and guardrails brittle enough that people route around them.

The infrastructure and the economics both now point the same direction. Outcome-based pricing means the vendor is paid when the work lands, and work lands when the people around the system trust it enough to use it and understand it well enough to catch it when it is wrong. Those are the same problem.

When to use an agent and when to use a script

logan@squidtrain.com (Logan Matson) — Mon, 06 Apr 2026 16:00:00 GMT

Agents are useful when the sequence of steps cannot be known in advance. When it can, an agent is a slower, more expensive, less predictable way to run a script.

This sounds obvious stated plainly. It is worth stating because the current default runs the other way: teams reach for an agent first and discover the determinism problem in production, at which point the agent is load-bearing and hard to remove.

The distinction that matters

A script encodes a decision you already made. An agent makes the decision at runtime. That flexibility costs latency, tokens, and determinism, and it is worth paying only when the decision genuinely cannot be made ahead of time.

A useful test: can you draw the flowchart? If you can, write the flowchart. Reaching for an agent to execute a known sequence adds a nondeterministic layer between you and an outcome you already understand.

The test has a failure mode worth naming. Sometimes you can draw the flowchart but it has forty branches, and writing it feels worse than delegating to an agent. That instinct is usually wrong. Forty branches you can enumerate are forty branches you can test. An agent replaces them with a decision process you cannot enumerate, which is a different thing from simplifying them.

The genuine case for an agent is not that the branches are numerous. It is that you cannot list them.

The middle ground

Between a script and an agent sits an option that gets skipped: a single constrained model call inside otherwise deterministic code.

Classification, extraction, summarization, and format conversion are model-shaped problems with script-shaped control flow. The model handles the part that resists rules, and the surrounding code does everything else. One call, no loop, no tool selection, no planning step.

This covers a large share of what gets built as an agent. If your agent makes one model call and then stops, it was never an agent. It is a function with an unnecessary framework around it. Removing the framework makes it faster, cheaper, and easier to test.

Reach for the loop only when the second call depends on what the first one found.

Where agents earn their cost

Agents fit when the input space is open and the path depends on what is found along the way.

Investigating a failure, where each finding determines the next thing to check. You cannot pre-specify the sequence because the second step depends on what the first one turned up.

Research across sources whose structure is unknown until opened. A script can fetch a known endpoint; it cannot decide that a document is irrelevant and try a different source.

Tasks where the number of steps varies by an order of magnitude between runs. If one input needs two operations and another needs sixty, fixed control flow either wastes work or truncates it.

The common thread is that enumerating the branches in advance would be impractical, not merely tedious.

Where scripts win

Fixed transformations, scheduled jobs, and anything whose correctness you need to guarantee. If the same input must always produce the same output, an agent is the wrong tool. Determinism is a feature, and agents trade it away by design.

Cost compounds too. A script that runs a thousand times a day costs the same each time. An agent doing the same work re-derives its plan on every run, paying for reasoning that produced an identical conclusion the previous nine hundred and ninety-nine times.

Latency follows the same pattern. A script’s runtime is roughly fixed. An agent’s varies with how many steps it decides to take, which makes it awkward to put in a request path where someone is waiting.

They fail differently

This is the operational difference and it gets underweighted.

A script fails at a known line, with a stack trace, in a way that reproduces. You fix it once and it stays fixed.

An agent fails by making a defensible-looking wrong decision partway through a sequence. There is no stack trace, because nothing threw. Reproducing it may not be possible, because the same input can produce a different path. And the fix is a change to instructions or tools whose effect on other cases is not obvious.

This means agents need observability that scripts do not: a record of each step, the reasoning, the tool calls, and the results. Without it, debugging is guesswork. Build that before the agent handles anything that matters, because retrofitting it means reproducing failures you cannot reproduce.

The hybrid case

Most real systems want both. An agent decides what needs to happen and then calls deterministic tools to do it. The agent handles the open-ended part; the scripts handle the parts where you need a guarantee.

The design question is where to draw the line, and there is a reliable heuristic: anything worth being certain about belongs in a tool the agent calls, not in the agent’s reasoning.

If a calculation must be right, do not ask the model to perform it. Give it a function. If an operation is destructive, do not let the agent construct the command. Give it a tool with constrained parameters. If a sequence must always run in order, make it one tool rather than three the agent could call out of order.

Each of those moves work out of the nondeterministic layer without giving up flexibility, because the agent still decides whether and when to call them.

Migrating between them

The choice is not permanent, and the useful direction is agent to script.

Start with an agent when the problem is genuinely open. Log the paths it takes. If it turns out that ninety percent of runs follow the same three sequences, those sequences are now the flowchart you could not draw at the start. Encode them as scripts and let the agent handle the remainder.

This works because the agent doubles as a discovery mechanism. What it cannot do is stay the permanent implementation of a path you now understand.

The shape most durable systems converge on is a thin layer of judgment over a thick layer of things that behave the same way every time, and the layer of judgment gets thinner as you learn what the system actually does.