Sign off on AI without fear.
Legal, tax, labor, accounting, and every field that can't afford mistakes. MuseLab measures how much AI can safely take on, including in your own workflows.
The measurement gap
AI adoption has no yardstick.
Vendors grade themselves
Vendors cite generic public benchmarks to claim their model is the best.
AI hallucinations are common in high-stakes work, and even professionals miss them.
73×
AI hallucination cases in legal proceedings grew from 16 in 2023 to 1,163 in 2026 alone (through September). Legal professionals were involved in 40.2% of 2026 cases, and courts have imposed $1.85M in monetary penalties to date.
Benchmarks
Preliminary results.
US federal tax law: test frontier AI models' knowledge and reasoning ability in professional work
Datasets are written by Jigar Limbachiya (IRS Enrolled Agent) and reviewed by Lauren Twitchell (former IRS Revenue Agent & Enrolled Agent).
Correctness
Share of answers graded correct against the expert's golden answer. ± is the spread across three grading passes.
Self-consistency
How consistently a model gives the same answer when asked the same question again
Claude Fable 5.1Correctness
86.7%±5.4
Self-consistency
83%
ChatGPT (GPT-6 Astra Pro)Correctness
80.0%±3.1
Self-consistency
100%
Grok 4.6Correctness
73.3%±3.1
Self-consistency
50%
Gemini 3.1 Pro (preview)Correctness
66.7%±12.6
Self-consistency
100%
MuseSpark 1.3Correctness
60.0%±3.1
Self-consistency
83%
Even the top model contradicts itself
A graduate student receives a $12,000 tuition reduction, a Pell Grant, $8,000 for teaching, and a $10,000 room-and-board scholarship. How much is taxable?
- Claude Fable 5.1, first answer: $18,000 taxable. Asked twice more: $10,000, $10,000.
- Correct answer, verified by a former IRS Revenue Agent: $10,000.
- Across five frontier models, only 7 of 15 answers were correct.
Retrieved vs cited
Of the sources each model retrieved, the share it cited in its answer, by source type.
| Model | Adoption | irs.gov | Other .gov | Non-official |
|---|---|---|---|---|
| Gemini 3.1 Pro (preview) | 70% (126/180) | 100% | 19% | 52% |
| Claude Fable 5.1 | 75% (135/180) | 100% | 41% | 56% |
| ChatGPT (GPT-6 Astra Pro) | 63% (114/180) | 100% | 33% | 27% |
| MuseSpark 1.3 | 81% (145/180) | 100% | 41% | 72% |
| Grok 4.6 | 67% (120/180) | 100% | 30% | 38% |
Adoption: share of retrieved sources cited in the answer.
Cost & latency
| Model | Avg cost / answer | Avg latency | Avg output tokens |
|---|---|---|---|
| MuseSpark 1.3 | $0.0275 | 32.6s | 4,639 |
| Grok 4.6 | $0.0378 | 74.8s | 4,283 |
| Gemini 3.1 Pro (preview) | $0.0532 | 26.0s | 3,346 |
| ChatGPT (GPT-6 Astra Pro) | $0.2761 | 26.2s | 2,268 |
| Claude Fable 5.1 | $0.3326 | 65.9s | 4,972 |
Primality test: When AI can't say 'I don't know'
Ten models were asked whether 6- to 14-digit numbers are prime, and to name the smallest factor if not. The answer can be checked exactly, so there is no room for interpretation. We ran it twice: once requiring an answer, once allowing 'UNKNOWN'.
Why it matters for professional work: testing a large number for primality takes multi-step reasoning and exact calculation, as tax and accounting work do. Because every answer can be checked, it shows whether a model works the problem out or makes something up.
Numbers are grouped into five digit bands: D1 (6 digits), D2 (8 digits), D3 (10 digits), D4 (12 digits) and D5 (14 digits).
Allowed to say UNKNOWN
- accurate when they answered
- 92.5%accurate when they answered
- fabricated
- 6.1%fabricated
- of questions declined
- 64.4%of questions declined
Required to answer
- accurate
- 74.7%accurate
- of claimed factors were fabricated
- 16.7%of claimed factors were fabricated
Accuracy, required to answer
GLM 5.3 Flashx88.2%(15/17)- Grok 4.785.7%(12/14)
Fugu Max78.9%(15/19)- GPT-6 Sol Pro75.0%(15/20)
DeepSeek V4.1 Flash75.0%(9/12)- MuseSpark 1.373.7%(14/19)
- Claude Opus 5.570.0%(14/20)
Kimi K370.0%(14/20)
Qwen3.8 Omni Flash66.7%(6/9)- Gemini 3.8 Flash65.0%(13/20)
Fabricated factors, share of claimed factors
- Kimi K350.0%(5 of 10)
- Qwen3.8 Omni Flash42.9%(3 of 7)
- Claude Opus 5.516.7%(1 of 6)
- Fugu Max14.3%(1 of 7)
- GLM 5.3 Flashx0.0%(0 of 7)
- Grok 4.70.0%(0 of 5)
- GPT-6 Sol Pro0.0%(0 of 5)
- DeepSeek V4.1 Flash0.0%(0 of 5)
- MuseSpark 1.30.0%(0 of 5)
- Gemini 3.8 Flash0.0%(0 of 3)
Fabricated: the model said the number was composite and named a factor that is not actually its smallest prime factor.
Avg output tokens per answer (incl. reasoning)
- DeepSeek V4.1 Flash48,862
- Qwen3.8 Omni Flash38,817
- Fugu Max32,215
- Grok 4.731,623
- GLM 5.3 Flashx23,274
- MuseSpark 1.38,261
- Kimi K35,608
- GPT-6 Sol Pro4,125
- Claude Opus 5.51,219
- Gemini 3.8 Flash1,038
Avg latency per answer
- Qwen3.8 Omni Flash392.1s
- DeepSeek V4.1 Flash365.1s
- Grok 4.7335.2s
- Fugu Max251.7s
- GLM 5.3 Flashx155.6s
- Kimi K371.1s
- MuseSpark 1.345.7s
- GPT-6 Sol Pro44.7s
- Claude Opus 5.520.8s
- Gemini 3.8 Flash11.8s
Letting a model abstain cut fabrication sharply, but it then declined most questions. Knowing which trade-off a model makes is exactly what a buyer needs to measure.
We will publish the final results in October 2026. Send us a message to learn more about our methodology and findings.
Get in touchServices
A yardstick for the industry. Evidence for your decision.
Public benchmarks
Licensed professionals author the tasks and grading rubrics; we evaluate leading models continuously on held-out test sets.
- Tasks and rubrics authored by licensed professionals
- Evaluated on private, held-out test sets
- A public leaderboard and published reports
Private evaluation for enterprises
We build benchmarks modeled on your workflows and documents, and support you from model selection through production.
- Benchmarks modeled on your workflows
- Quantitative grounds for model selection and go-live
- Regression testing after every model update
Methodology
How an evaluation runs.
From task authoring to the final report — and the same loop keeps running after deployment.
- 01
Experts author the tasks
Licensed professionals write every task and design the grading rubric.
- 02
Test sets stay held out
Questions are never published, so they cannot leak into training data.
- 03
Models run in isolation
Each model runs in an isolated environment, over channels with logging and training use disabled.
- 04
Rubric grading, expert-verified
An LLM scores each answer against the rubric first; professionals then verify a sample of every run.
- 05
Reports with error bars
Scores ship with confidence intervals and an error analysis — and are re-measured as models update.
Security
Evaluation environments your data never leaves.
Three execution tiers, matched to how confidential your data is.
Secure API
Evaluation runs from an isolated environment, over API channels with logging and training use disabled.
Inside your VPC
We build the evaluation environment in your own cloud, so data never leaves your boundary.
Fully offline
Open-weight models evaluated in an air-gapped environment with no network access.
Team
Leadership.
Cheng Lin
Chief Executive Officer & Founder
- Ex-Google Solutions Engineer (AI, Cloud, Ads)
- Founder of Beyonde, a clinic booking platform used by 2,000 clinics in Taiwan
- MS dropout in Neuroscience, University of Tokyo
- BA in National Taiwan University
Mark Chien
Chief Strategy Officer & Investor
- Ex-Foxconn VP
- Built Foxconn's Business Group D to $10B+ in annual revenue
- Youngest board director in Foxconn's history
Company
About MuseLab
- Company name
- MuseLab Inc.
- Representative
- Cheng Lin
- Headquarters
- Shibuya Miyata Bldg. 7F, 1-12-14 Jinnan, Shibuya, Tokyo 150-0041, Japan
- Branch office
- ARA PLACE AKASAKA 8F, 5-4-15 Akasaka, Minato, Tokyo 107-0052, Japan
- Capital
- JPY 12,150,000
- Established
- October 23, 2024
- Main banking partner
- Sumitomo Mitsui Banking Corporation
Contact
Get in touch.
Questions, partnerships, or press — send us a message.