Machine Learning Times
Machine Learning Times
Announcing Eric Siegel’s New Book: The AI Playbook
  Dear Reader, I’m excited to announce the forthcoming,...
Predictive Analytics for the Call Center
 So, you just received your shiny new smart watch....
MLW Preview Video: Gulrez Khan, Data Science Lead at PayPal
 In anticipation of his upcoming keynote presentation at Predictive...
MLW Preview Video: Devanshi Vyas, Co-Founder at Censius
 In anticipation of her upcoming presentation at Deep Learning...

2 months ago
ChatGPT’s Performance Is Slipping, New Study Says

Originally published in Decrypt, July 19, 2023.

UC Berkeley researchers found that ChatGPT has not improved over time, and in fact, may have gotten worse.

ChatGPT exploded onto the scene late last year, dazzling people with its human-like conversational abilities, and the release of latest version prompted a  crypto rally and calls for a pause in development. But according to a new study, the leading AI bot’s skills may actually be on the decline.

Researchers at Stanford and UC Berkeley systematically analyzed different versions of ChatGPT from March and June 2022. They developed rigorous benchmarks to evaluate the model’s competency in math, coding, and visual reasoning tasks. The results of ChatGPT’s performance over time were not good.

The tests revealed a startling drop-off in performance between versions. On a math challenge of determining prime numbers, ChatGPT solved 488 out of 500 questions correctly in March, an accuracy of 97.6%. However, in June, ChatGPT only managed to get 12 questions right, plunging to 2.4% accuracy.

To continue reading this article, click here.

2 thoughts on “ChatGPT’s Performance Is Slipping, New Study Says

Leave a Reply