OpenAI and Anthropic Shipped New Models the Same Afternoon: Build the 10-Question House Exam That Tells You Whether to Switch
On September 22, Anthropic released Claude Opus 5.5 and OpenAI released GPT-6 Sol and Luna, each priced below the model it replaces. Lab benchmarks can't tell you which one handles your work. A 10-question exam built from your own jobs can, in about 20 minutes.
What shipped on September 22
- Anthropic: Claude Opus 5.5. Anthropic says it performs at the level of Claude Fable 5.1, its pricier top model. API prices dropped 20%, to $4 per million input tokens and $20 per million output tokens. It is available on the Pro, Max, Team, and Enterprise plans.
- OpenAI: GPT-6 Sol and GPT-6 Luna. OpenAI says Sol makes about half as many mistakes as GPT-5.6 Sol. API prices are half or less of the models they replace. Luna, the low-cost model, is also available to Free and Go users in the ChatGPT desktop app.
The so-what. Every lab publishes benchmarks. None of them test your quote follow-ups, your job notes, or your month-end summary. Three new models in one afternoon means someone will tell you to switch this week. Decide with your own test. Build it once and reuse it every time a model ships.
Build the house exam (20 minutes, once)
- Pick 10 real tasks you gave AI in the last month. Use a mix: 4 writing tasks (a customer email, a proposal section), 3 reading tasks (summarize a contract, pull numbers from an export), and 3 judgment calls (which of these leads to call first).
- Strip names, addresses, and account numbers. Keep the file in your private business AI workspace.
- Write the answer key. For each task, list three things a good answer must include.
- Save the 10 prompts and the key in one doc called House Exam.
Run it (20 minutes, every model release)
Paste each prompt into the new model and into the one you use now. Score each answer 0, 1, or 2 against your key. Then paste both sets of answers into either model with this:
Compare these two sets of answers against my answer key. For each task, tell me which answer meets more of the key. List any fact, number, or promise that appears in an answer but was not in my prompt. Do not pick an overall winner. Give me a task-by-task table.
The rule. Move a task to the new model only if it scored higher on that task and invented nothing. Leave everything else where it is. Keep the score sheet, so the next release takes 20 minutes instead of an afternoon.