Book a free call
TrainingProductsTeamSupportContactLabBlog Book a free call
Lab build · November 7, 2026

AI Explainer Lab: First Option Bias

AI has a first option bias. Give it two equally good choices and it usually picks whichever one you listed first, then explains why with reasons that flip when you swap the order. See it happen in 7,680 recorded runs on eight AI models, then copy the one-paragraph fix so AI weighs your options fairly.

Draft: partial data while the recording finishes. Not for posting.

Ask, then swap

Two finalists with the same experience, the same results and the same salary ask, written in different words. Ask the AI to pick one. Then swap the order and ask again. Every reply is a real, recorded run.

  1. Pick a decision and a model
  2. Ask the AI
  3. Swap the order and ask again
Decision
AI model

Option 1listed first
Option 2listed second
First ask
Press “Ask the AI” to see its recorded reply.
After the swap
Then swap the order and ask again.

Drop every run

Each ball is one recorded run of the decision and model you picked above, in both orders. It falls into the bin for the slot the AI chose. A fair judge would fill both bins about evenly.

0%

Every model, every decision

The fix: score each one on its own, and allow a tie

A simple chat prompt asks the AI to pick, so it picks, and the first option gets the benefit of the doubt. Two changes fixed it on every model we tested:

  1. Score each option on its own first, as if the AI had not seen the others, then compare the scores.
  2. Give it permission to say they're equal. If you only allow a pick, you'll get one, even when there's nothing to pick between.

For a decision that matters, also swap the order and ask again. If the answer changes, the options were closer than the reply made them sound.

The Fair Pick prompt · the exact wording we tested

Before comparing, judge each option on its own merits: give Option 1 a score from 1 to 10 with one sentence of reasoning, then do the same for Option 2 as if you had not seen Option 1. Then compare the scores. If they are equal, or the options are equally good, say so.

Paste it under your question, with the options labeled Option 1 and Option 2. Comparing more than two? Score each one the same way.

Get more free Lab builds. A new one most weekends, straight to your inbox.

Let us help your team use AI for real decisions

The finalists, bids and applicants on this page are made up. The mistake isn't: people ask AI to pick a vendor, rank applicants or choose between proposals every day, and the order they pasted things in quietly decides the answer. We train teams to write prompts and skills that weigh options fairly, in the AI tools they already use.

Hands-on AI trainingwith your team's real decisions
A shared prompt libraryfor the choices you make every week
Custom skillsthat score options the fair way
Guardrails that stickinside IT's rules
Book a free call

About this build

Built with Claude Opus 5.5 on November 7, 2026. We wrote six everyday decisions, each with two options of equal substance in different words: two office manager finalists, the same finalists with their names swapped, two bookkeeper finalists, two cleaning quotes, two roofing bids and two rental applicants. Each pair was shown in both orders, 40 times per order, with a simple chat prompt and with the fix, on eight models: Claude Haiku 5.5, Claude Sonnet 5.5, Claude Opus 5.5 and Claude Fable 5.1 through the Claude API, GPT-6 Luna, GPT-6.1 Sol and GPT-6 Astra through the OpenAI API, and gpt-oss 20B, an open model, on our own PC. That is 7,680 recorded replies on this page.

A pilot on three earlier pairs came first. Two of those pairs turned out not to be equal (one wording gave a cleaning quote a real edge, and two taglines came down to taste), so we rewrote or replaced them, and a short screen before the full run led us to reword one bookkeeper pair the same way. Every pair on this page then ran in full on every model, and none was dropped. Including the pilot and the screen, the build recorded 8,940 runs and about 6.9 million tokens, and took about 3 hours and 38 minutes from the first pilot run to the last record.

Each reply ends with a "Choice:" line, and a pick counts as "listed first" when it names the option shown as Option 1. The finalists, companies, bids and applicants are fictional. Every reply on this page is a recorded run of a real model, saved and replayed here; nothing is generated while you watch. Everything shown works in a normal Claude, ChatGPT or Microsoft 365 Copilot chat; we used the APIs only to record and measure.

More builds in the Lab

Sunlight shining down through deep blue water

Find light in the deep waters of AI

Let's talk! A short call will help you learn more about our training and consulting solutions. We can also discuss whether our existing licensed tools can help, or whether you need a custom build.

Book a free call