Machine Learning Times
EXCLUSIVE HIGHLIGHTS
For Managing Business Uncertainty, Predictive AI Eclipses GenAI
  Originally published in Forbes The future is the ultimate...
AI Business Value Is Not an Oxymoron: How Predictive AI Delivers Real ROI for Enterprises
  Originally published in AI Realized Now “Shouldn’t a great...
How To Un-Botch Predictive AI: Business Metrics
  Originally published in Forbes Predictive AI offers tremendous potential...
2 More Ways To Hybridize Predictive AI And Generative AI
  Originally published in Forbes Predictive AI and generative AI...
SHARE THIS:

3 years ago
ChatGPT’s Performance Is Slipping, New Study Says

 
Originally published in Decrypt, July 19, 2023.

UC Berkeley researchers found that ChatGPT has not improved over time, and in fact, may have gotten worse.

ChatGPT exploded onto the scene late last year, dazzling people with its human-like conversational abilities, and the release of latest version prompted a  crypto rally and calls for a pause in development. But according to a new study, the leading AI bot’s skills may actually be on the decline.

Researchers at Stanford and UC Berkeley systematically analyzed different versions of ChatGPT from March and June 2022. They developed rigorous benchmarks to evaluate the model’s competency in math, coding, and visual reasoning tasks. The results of ChatGPT’s performance over time were not good.

The tests revealed a startling drop-off in performance between versions. On a math challenge of determining prime numbers, ChatGPT solved 488 out of 500 questions correctly in March, an accuracy of 97.6%. However, in June, ChatGPT only managed to get 12 questions right, plunging to 2.4% accuracy.

To continue reading this article, click here.

7 thoughts on “ChatGPT’s Performance Is Slipping, New Study Says

  1. As the backrooms game continue to capture the imagination of online communities, game developers, and players alike, the digital exploration of these mysterious spaces serves as a testament to the enduring power of internet-driven storytelling and shared experiences.

     
  2. The recent study highlighting the decline in ChatGPT’s performance is indeed concerning. A drop from 97.6% to 2.4% accuracy in prime number identification between March and June 2023 is significant. This suggests that continuous monitoring and evaluation of AI models are crucial to maintain their reliability.

    For those seeking consistent and dependable AI interactions, platforms like gptde.de offer access to ChatGPT in German without the need for registration. Such platforms can be valuable resources for users requiring stable AI assistance.