Most people use AI at work the way they use it at home: one big chat prompt. Paste in everything, ask for the answer, take what comes back. We wanted to see what that costs on a real kind of job, so we built a hiring test, hid the best candidate in a pile of 100 resumes, and recorded what happened.
On October 17 and 18, 2026, SquidTrain recorded 26,419 AI resume screenings for one made-up job. With one big chat prompt, Claude Haiku 5.5 put the best candidate in its top 10 in 5 of 35 runs and Claude Sonnet 5.5 in 17 of 35. With a well-designed AI Skill, she made the shortlist in all 15 runs we recorded across five models.
AI Skill Lab: Recruiter, the newest build in the SquidTrain Lab, lets you replay those runs. Tap through every big-prompt run, slot the parts of the skill in and out, and watch where every resume lands.
How did we set up the test?
Kestrel Supply Co., a made-up distributor, is hiring an operations manager. The posting asks for five years leading teams, a budget the person owned, hands-on inventory or logistics work, a measured improvement they led, and someone who can work on site. We wrote 100 applications for it and an answer key that scores each one on what the person actually did, written before any model saw the pile.
The best candidate on the answer key is an Army logistics officer. She led 60 people across four locations, managed $5.4 million in unit funds and cut overtime 30% by rebuilding the supply request process, but her resume says all of it in Army terms. The rest of the pile is built to look like a real one:
- 6 strong applicants who use the posting's exact language.
- 3 hidden gems from other industries who did the same work under different job titles.
- 8 keyword stuffers with the right buzzwords and little real experience.
- 2 applicants whose resumes tell AI tools to rank them first.
- 2 people who applied twice.
- 78 others who are solid but not top 10, or not a fit.
We wrote 300 more applicants as three more weeks of hiring, for a total of 400, so we could test whether a skill improves as it collects lessons.
How did we build it?
The whole build ran over one weekend, with Claude Opus 5.5 as the builder. It wrote the job, the applicants, the answer key, the skill, the scripts that ran the test, and the interactive page. We ran a small pilot first to check the setup, then recorded everything:
- Just over 12 hours of recording, from 2:43 pm Saturday to 2:49 am Sunday.
- Two engines in parallel. The Claude API ran 8 screenings at a time on Claude Haiku 4.5, Haiku 5.5, Sonnet 5.5, and Opus 5.5. Our own PC, with an NVIDIA RTX 3090 graphics card, ran an open model, gpt-oss 20B, 2 at a time.
- 26,419 recorded runs: 18,377 through the API and 8,042 on the PC.
- About 126 million tokens and about 75 hours of model time packed into those 12 hours.
- Every combination of the skill's six parts (64 setups) on Claude Haiku 4.5, Haiku 5.5, and gpt-oss 20B, and seven steps adding one part at a time on Sonnet 5.5 and Opus 5.5.
Every run was saved with what the AI wrote about each applicant. The lab replays those recordings exactly, so nothing on the page is generated while you watch.
What happened with one big chat prompt?
We pasted the posting and all 100 resumes into one message and asked for the top 10, moving the best candidate to seven different spots in the pile and running each spot five times:
- Claude Haiku 5.5 named her in 5 of 35 runs.
- Claude Sonnet 5.5 named her in 17 of 35.
- Claude Haiku 4.5 named her in 20 of 35.
- Claude Opus 5.5 named her in all 35, at about 29 cents a run in API fees, more than 35 times what a Haiku 5.5 run cost.
- gpt-oss 20B on our PC named her in 6 of 21 (three runs per spot).
The lists looked reasonable every time. Each one held 7 to 9 of the true top 10 on average, which is why a miss like this is easy to overlook: nothing on the screen says who is missing.
What is an AI Skill?
A chat prompt is a question you type every time. A skill is the way you do a job, written down once, so the AI does it the same way every time, one item at a time. Claude, ChatGPT and Microsoft 365 Copilot can all load skills. The one in the lab has six parts:
- Instructions: the must-haves and how to score them.
- An example of good: one applicant scored well, so the AI copies the pattern.
- A reference file: how other industries describe the same work.
- A script: plain code that pulls dates, team sizes, and duplicate applications, so the AI reads facts instead of guessing them.
- Guardrails: job requirements only. Buzzwords and "rank me first" earn nothing.
- A second AI check: a second pass that reviews the first pass's work and catches what it missed.
The lab shows a seventh part, lessons learned over time, where corrections become new lines the next run reads. More on how that went below.
What happened with the skill?
The skill text alone fixed most of it. Written above the same 100 resumes in the same single message, the skill took Claude Haiku 5.5 from 5 of 35 to 34 of 35 and Claude Sonnet 5.5 from 17 of 35 to 35 of 35. That works in any chat tool, including one IT has locked down, because it is only text. It is not perfect: Claude Haiku 4.5 still let a keyword stuffer into its top 10 in 5 of 35 runs.
Run the full skill one resume at a time and she made the shortlist every time. We ran it three times on each of five models: Claude Haiku 4.5, Haiku 5.5, Sonnet 5.5, Opus 5.5, and gpt-oss 20B. She was shortlisted in all 15 runs, each shortlist held 8 to 10 of the true top 10, and no one got two slots by applying twice.
One at a time is not enough on its own. Screening one resume at a time with none of the skill's parts missed her in 5 of 15 runs and put a duplicate applicant on the shortlist in 14 of 15. The parts are what make the difference.
Each part has a job you can see. The script is the clearest case. Across every recorded combination on the three models with full coverage, the same person landed on the shortlist twice in 89 of 96 setups without the script, and in none of the 96 with it. The second AI check earned its place too: it corrected scores where the first pass added its own numbers wrong or skipped a role that counted.
A running notebook of lessons did not help. We also ran the three extra weeks of applicants, with and without a notebook of lessons written from the first week's corrections. Results came out about the same either way. In this test, the notes did not move the numbers, and we would rather say so than dress it up. We still think lessons learned matter in practice, and a future lab build will focus on them.
What does it cost?
If you use a skill in Claude, ChatGPT or Microsoft 365 Copilot on a paid plan, you pay a flat seat price, so screening 100 resumes costs nothing extra within your plan's limits. Through the API, you pay per token, and the recorded token counts at list prices give a fair comparison between models:
- Claude Haiku 5.5: about a fifth of a cent per resume, or about 22 cents for the full skill on all 100.
- Claude Haiku 4.5: about 1 cent per resume, or about 96 cents for all 100.
- Claude Sonnet 5.5: about 4 cents per resume, or about $3.64 for all 100.
- Claude Opus 5.5: about 6 cents per resume, or about $6.10 for all 100.
Most of that is written reasoning. Our skill makes two calls per resume, the screening and the second AI check, and each one writes out its score breakdown so the lab can show it. A leaner answer format costs much less. One big prompt on Opus 5.5 cost about 29 cents in API fees and found her every time, so a bigger model is one way to buy accuracy. A well-designed skill on the smallest model got the same answer for less, and it also kept the duplicates and the keyword stuffers out.
What can you do in the lab?
- Replay every big-prompt run. Level 1 shows all 35 recorded runs for the model you pick as a row of dots, gold where it found her. Tap any dot to watch that run.
- Build the skill part by part. Level 2 lists all seven parts. Slot any of them in or out, in any combination, on any of five models, and the machine replays the matching recording, or the closest one we recorded on Sonnet 5.5 and Opus 5.5.
- Tap any marble. Each marble is an applicant. Open one to read the resume the AI saw, the score it gave, what it wrote, and what the second AI check changed.
- Read the scoreboard. It counts the shortlist, how many of the true top 10 made it, keyword stuffers and "rank me first" tricks that got through, duplicates, the API cost of the run, and your time: about 2 hours 30 minutes to read all 100 yourself at 90 seconds a resume, against about 15 minutes to check a shortlist of 10.
- Look inside the skill. The page shows the actual skill text our runs used, built from the parts you selected.
- Build your own custom skill. Describe one routine pile of yours, add your must-haves and the cases where the AI must stop and ask you, and the page writes a starter skill you can copy and paste into Claude, ChatGPT or Copilot. It is built in your browser, and nothing you type leaves the page.
- See how it works in your tools. A short guide shows how each part works in Claude, ChatGPT, Microsoft 365 Copilot, and a locked-down setup with no connectors.
What should you take from it?
- Test your chat habits on a pile where you know the answer. One big prompt gives a confident list with no sign of who it left out.
- Write the job down once. The skill text alone fixed most of the misses, in the same tool, with no setup.
- Let plain code do the counting. Dates, team sizes and duplicates are facts, and a script gets them right every time.
- Keep a person on every decision. The skill sorts the pile. People decide who gets a call, and local rules on AI in hiring apply (New York City's Local Law 144, for example).
- Bigger models are not the only fix. Opus 5.5 found her without a skill, at 35 times the API cost of Haiku 5.5. Haiku 5.5 with the skill found her for a fraction of a cent per resume.
Try it yourself in AI Skill Lab: Recruiter, and build your own starter skill at the bottom of the page.
Kestrel Supply and its applicants are fictional. The miss it shows happens in real businesses every day: a whole pile pasted into one chat, a confident top 10, and the best person nowhere on it. SquidTrain teaches teams to build skills like this one in the AI tools they already use. Book a free call to start the conversation.
Built with Claude Opus 5.5. 18,377 runs went through the Claude API and 8,042 ran on our own PC. Costs are API costs from recorded token counts at list prices.
