Introduction to Large Language Models

But how is it possible that by just predicting the next word AI is able to write essays and answer questions? Ilya Sutskever, Chief Scientist at OpenAI, conceptually explained it in one of his interviews. Imagine you’re reading a detective story, and on the last page of the book, there is a sentence: “And the name of the killer is ___”. In order to make a good prediction of this next word, you really need to take into account all the information contained in the book. Thus, by better and better predicting the next word an LLM essentially compresses the information about the World into some model.

This is a lossy compression (that is, some of the information in the text it analyses is lost when the model is created). Otherwise, it would require memorizing the whole Internet, and would take significantly more computation and lead to longer prediction times. Due to this lossy nature of compression, LLMs are known to completely make up thing facts that do not exist in the real world, or just provide wrong answers from time to time. This phenomenon is known as hallucinations. It hasn’t been fully solved in any of the modern LLMs.

How was AI created?

The progress in AI was achieved through decades of research, going back to the 1960s and 1970s. But it only became possible when computer hardware evolved to a stage of being quite powerful and yet relatively cheap. Even today, the major bottleneck in Large Language Models progress is the limited supply of GPUs (graphics processing units). GPUs were originally designed for rendering graphics for video games, but their design also happens to be perfect for the type of operations that AI relies upon, so they are now heavily used in this area,

In machine learning, practitioners usually try to find a balance between a model memorizing the data, and generalizing well for unseen data. Sometimes a model just memorizes the data that it saw in the past, but performs poorly on the test or unseen data, this is called overfitting. On the other hand, when there is not enough data, a model doesn’t learn all the essential underlying characteristics, this is called underfitting. It was observed that with deep neural networks, when the amount of data available increases, but also the number of network parameters or neurons increases, the model doesn’t reach this overfitting stage, and performs well in general.

It was hypothesized that the more parameters the model has, the better it will perform. This direction of development was taken by OpenAI with its famous GPT models. With every new model generation you can see a significant increase of the number of model parameters. It was also observed that, with an increasing number of parameters, models often obtain new capabilities such as arithmetic operations, basic math, language translation, etc. On the other hand, latest LLMs already processed most of the publicly available Internet. In order to feed in more data, researchers have turned to other sources of information such as videos, podcasts, pdfs, sensor data etc.

Another breakthrough achieved by OpenAI, which led to the creation of ChatGPT, is adding the RLHF stage after the general model training. RLHF stands for Reinforcement learning from human feedback. The base language model is just some text essence or statistics extracted from the language. In order for it to perform useful tasks, and produce valuable output, you will need to provide many examples of “good responses” for particular prompts. During this stage many human raters rate the answers of the LLM to different types of prompts, thus giving it a sense of what a human would expect to be a good and appropriate answer.

LLM Security

Although all LLMs go through a so-called alignment stage, and should not provide answers to sensitive questions, people constantly come up with new ways to jailbreak LLMs or circumvent their security guardrails. In one famous example, a person was able to get a recipe for napalm from LLM by telling it an imaginary story about their grandmother.