Machine Learning Times
Machine Learning Times
Elon Musk Predicts Artificial General Intelligence In 2 Years. Here’s Why That’s Hype
 Originally published in Forbes, April 10, 2024 When OpenAI’s...
Survey: Machine Learning Projects Still Routinely Fail to Deploy
 Originally published in KDnuggets. Eric Siegel highlights the chronic...
Three Best Practices for Unilever’s Global Analytics Initiatives
    This article from Morgan Vawter, Global Vice...
Getting Machine Learning Projects from Idea to Execution
 Originally published in Harvard Business Review Machine learning might...

1 month ago
Large language models use a surprisingly simple mechanism to retrieve some stored knowledge


Originally published in MIT News, March 25, 2024

Researchers demonstrate a technique that can be used to probe a model to see what it knows about new subjects.

Large language models, such as those that power popular artificial intelligence chatbots like ChatGPT, are incredibly complex. Even though these models are being used as tools in many areas, such as customer support, code generation, and language translation, scientists still don’t fully grasp how they work.

They found a surprising result: Large language models (LLMs) often use a very simple linear function to recover and decode stored facts. Moreover, the model uses the same decoding function for similar types of facts. Linear functions, equations with only two variables and no exponents, capture the straightforward, straight-line relationship between two variables.

The researchers showed that, by identifying linear functions for different facts, they can probe the model to see what it knows about new subjects, and where within the model that knowledge is stored.

To continue reading this article, click here.

4 thoughts on “Large language models use a surprisingly simple mechanism to retrieve some stored knowledge

  1. By identifying these linear functions, scientists can probe LLMs to understand Pokerogue what they know about new subjects and pinpoint where this knowledge is stored, shedding light on the inner workings of these complex AI systems.


Leave a Reply