AI Explainer Lab: First Option Bias
AI has a first option bias. Give it two equally good choices and it usually picks whichever one you listed first, then explains why with reasons that flip when you swap the order. See it happen in 7,680 recorded runs on eight AI models, then copy the one-paragraph fix so AI weighs your options fairly.
Ask, then swap
Two finalists with the same experience, the same results and the same salary ask, written in different words. Ask the AI to pick one. Then swap the order and ask again. Every reply is a real, recorded run.
- Pick a decision and a model
- Ask the AI
- Swap the order and ask again
Drop every run
Each ball is one recorded run of the decision and model you picked above, in both orders. It falls into the bin for the slot the AI chose. A fair judge would fill both bins about evenly.
Every model, every decision
The fix: score each one on its own, and allow a tie
A simple chat prompt asks the AI to pick, so it picks, and the first option gets the benefit of the doubt. Two changes fixed it on every model we tested:
- Score each option on its own first, as if the AI had not seen the others, then compare the scores.
- Give it permission to say they're equal. If you only allow a pick, you'll get one, even when there's nothing to pick between.
For a decision that matters, also swap the order and ask again. If the answer changes, the options were closer than the reply made them sound.
Before comparing, judge each option on its own merits: give Option 1 a score from 1 to 10 with one sentence of reasoning, then do the same for Option 2 as if you had not seen Option 1. Then compare the scores. If they are equal, or the options are equally good, say so.
Paste it under your question, with the options labeled Option 1 and Option 2. Comparing more than two? Score each one the same way.
Let us help your team use AI for real decisions
The finalists, bids and applicants on this page are made up. The mistake isn't: people ask AI to pick a vendor, rank applicants or choose between proposals every day, and the order they pasted things in quietly decides the answer. We train teams to write prompts and skills that weigh options fairly, in the AI tools they already use.
The Fair Pick download
Free, no sign-up needed. The Fair Pick prompt, ready for Claude, ChatGPT and Microsoft 365 Copilot, plus a one-page checklist for decisions you hand to AI.
Download (zip)
Find light in the deep waters of AI
Let's talk! A short call will help you learn more about our training and consulting solutions. We can also discuss whether our existing licensed tools can help, or whether you need a custom build.