The State Bar of California admitted that 23 questions on the February 2025 bar exam were created with artificial intelligence, and not by lawyers.
The bar now says it will ask the California Supreme Court to adjust scores after the test was also marred by technical meltdowns.
This was a high-stakes licensing exam; the gatekeeper to becoming a lawyer. But it was riddled with AI-written questions, system crashes, and student lawsuits.
- 171 total scored multiple-choice questions
- 100 were from Kaplan
- 48 came from an old first-year exam
- 23 were AI-assisted, developed by ACS Ventures
- $8.25M: Kaplan’s contract
- $22M: State Bar’s budget deficit
The backlash
Legal educators are outraged.
“Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable,” said Mary Basick, assistant dean at UC Irvine Law.
“It’s a staggering admission,” added Katie Moran, a law professor at USF. “The same company that used AI also approved its own questions.”
Test-takers in February reported:
- Platform crashes
- Inability to save essays
- Copy-paste errors
- Nonsensical or typo-riddled questions
Meazure Learning, the testing platform provider, is now facing lawsuits from students.
The bigger focus
AI is already changing the legal world. But lack of oversight and cost-cutting turned this experiment into a disaster. Dean Andrew Perlman of Suffolk Law says AI could help in test creation, but only if experts carefully vet the output.
“In the future, we’ll worry about the competence of lawyers who don’t use AI,” Perlman predicted.
What’s next:
- Supreme Court review of score adjustments
- A May 5 meeting on further remedies
- No decision yet on returning to traditional exams
The legal profession just got a reality check. AI might be the future, but it’s not ready to grade the bar.
