California Bar Exam’s AI Controversy and Its Implications
In a significant development regarding the February 2025 bar exam, the State Bar of California has come under scrutiny after acknowledging that some of the multiple-choice questions were crafted with the assistance of artificial intelligence. This revelation follows widespread complaints from examinees about various technical difficulties and irregularities during the exam process.
Admission of AI Usage Raises Concerns
The State Bar’s announcement included a plan to petition the California Supreme Court for adjustments to the scores of those who participated in the controversial exam. However, the Bar has refrained from recognizing substantial issues concerning the quality of multiple-choice questions. Notably, some questions were reportedly recycled from first-year law school exams, raising important questions about their appropriateness for the licensing process.
Expert Reactions
Mary Basick, an assistant dean at UC Irvine Law School, expressed her astonishment, asserting, “Having the questions drafted by non-lawyers using artificial intelligence is just unbelievable.” Following the exam, many candidates noted that some questions appeared to be generated by AI, undermining the integrity of the examination process. Basick indicated that using AI for question creation creates a conflict of interest, as the psychometricians overseeing the exam’s validity also contributed to developing the questions.
Katie Moran, an associate professor at the University of San Francisco School of Law, echoed these sentiments, stressing the implications of employing a non-lawyer to draft exam questions that are then evaluated for quality and relevance by the same entity.
Details of Question Development
According to the State Bar, a majority of the multiple-choice questions—171 in total—were sourced from Kaplan Exam Services, a firm hired for cost-effective question preparation. Of these, 100 questions originated from Kaplan, 48 from a first-year law examination, and 23 were created using AI assistance from ACS Ventures, the Bar’s independent psychometrician.
State Bar’s Defense of Question Quality
Leah Wilson, the State Bar’s executive director, claimed confidence in the validity of the questions to “accurately and fairly assess the legal competence of test-takers.” Despite this, critics have highlighted numerous issues, including reported technical glitches during the exam, such as disconnections and screen errors, alongside complaints regarding the quality of certain questions.
Future Implications and Calls for Transparency
In light of these controversies, some experts are urging the State Bar to embrace transparency by releasing all questions used in the exam and reconsidering the shift away from the National Conference of Bar Examiners’ Multistate Bar Examination for future assessments. As the Bar proceeds with its current testing model, some faculty members are skeptical about the integrity of the question development process and the potential impact on aspiring lawyers’ readiness to practice.
Next Steps
The Committee of Bar Examiners is scheduled to convene to discuss potential adjustments regarding the February exam. However, state officials have indicated that reverting to the previous bar examination structure remains unlikely, particularly given the preferences expressed by a significant portion of examinees for remote testing options.
As the situation unfolds, it becomes increasingly vital for the State Bar of California to navigate these challenges judiciously, balancing the pursuit of innovation with the necessity of maintaining rigorous standards in the licensing process for future attorneys.