Machine Learning Times
EXCLUSIVE HIGHLIGHTS
Jev AI Will Make You Money – But Exactly How Much?
  Originally published in Forbes As I covered recently, Jev...
Jev Fever: Predictive AI Is Superhot Again
  Originally published in Forbes The more AI changes, the...
GenAI Owns the Hype, but Predictive AI Is Thriving
  EDITOR’S NOTE: This article is adapted from the...
The Data Disconnect: A Key Challenge for Machine Learning Deployment
  EDITOR’S NOTE: This article is excerpted from The...
SHARE THIS:
  • Jun 1, 2024
  • Comments Off on GPT-4 didn’t ace the bar exam after all, MIT research suggests — it didn’t even break the 70th percentile
  • Industry News, Left-hand
  • 5408 Views

2 years ago
GPT-4 didn’t ace the bar exam after all, MIT research suggests — it didn’t even break the 70th percentile

 

Originally published in Live Science, May 31, 2024.

Last year, claims that OpenAI’s GPT-4 model beat 90% of trainee lawyers on the bar exam generated a flurry of media hype. But these claims were likely overstated, a new study suggests.

GPT-4 didn’t actually score in the top 10% on the bar exam after all, new research suggests.

OpenAI, the company behind the large language model (LLM) that powers its chatbot ChatGPT, made the claim in March last year, and the announcement sent shock waves around the web and the legal profession.

Now, a new study has revealed that the much-hyped 90th-percentile figure was actually skewed toward repeat test-takers who had already failed the exam one or more times — a much lower-scoring group than those who generally take the test. The researcher published his findings March 30 in the journal Artificial Intelligence and Law.

To continue reading this article, click here.

Comments are closed.