Originally published in Live Science, May 31, 2024.
Last year, claims that OpenAI’s GPT-4 model beat 90% of trainee lawyers on the bar exam generated a flurry of media hype. But these claims were likely overstated, a new study suggests.
GPT-4 didn’t actually score in the top 10% on the bar exam after all, new research suggests.
OpenAI, the company behind the large language model (LLM) that powers its chatbot ChatGPT, made the claim in March last year, and the announcement sent shock waves around the web and the legal profession.
Now, a new study has revealed that the much-hyped 90th-percentile figure was actually skewed toward repeat test-takers who had already failed the exam one or more times — a much lower-scoring group than those who generally take the test. The researcher published his findings March 30 in the journal Artificial Intelligence and Law.
To continue reading this article, click here.
