Part 2: How LLMs Operate

Q: What the name “ChatGPT” tells us about LLMs

The first LLM that became widely known to the public was ChatGPT. Its name is an abbreviation that already hints at some of its key properties. Here’s what each part of the name stands for: Chat: This refers to how we interact with the model: through conversation. LLMs must be specifically trained to support this chat functionality, enabling them to respond in a natural, dialogue-based manner. G: Stands for “Generative,” meaning the model can create content. In the case of LLMs, that content is text. P: P stands for “Pretrained”. The term “Pretrained” refers to a model that has already been trained on a large dataset before you use it for your specific task. T: Stands for “Transformer,” which refers to the architecture of the model. This architecture is used by all major LLMs and is particularly effective at capturing relationships between words and across segments of text.

Markus Hofer
6 days ago
5 min read

Series: What Every CEO Needs to Know About Generative AI

While most executives are not deeply involved in technical details, it is still important to have a basic understanding of how Large Language Models (LLMs) work. This knowledge helps build intuition about the opportunities and risks of the technology and enables better decision-making. For that reason, I will present the most essential insights into how these models function.

In 2022, OpenAI released ChatGPT to the public. ChatGPT is what we call a Large Language Model (LLM). An LLM is built from artificial neural networks, which are systems originally designed as a model of the human brain. This is why we use the term “model” in AI for neural network–based programs that mimic human-like capabilities.

What the name “ChatGPT” tells us about LLMs

The first LLM that became widely known to the public was ChatGPT. Its name is an abbreviation that already hints at some of its key properties. Here’s what each part of the name stands for:

Chat	This refers to how we interact with the model: through conversation. LLMs must be specifically trained to support this chat functionality, enabling them to respond in a natural, dialogue-based manner.
G	Stands for “Generative,” meaning the model can create content. In the case of LLMs, that content is text.
P	P stands for “Pretrained”. The term “Pretrained” refers to a model that has already been trained on a large dataset before you use it for your specific task. This means that you don’t need to train the model, but also implies that the model is fixed and you cannot influence its inner working.
T	Stands for “Transformer,” which refers to the architecture of the model. This architecture is used by all major LLMs and is particularly effective at capturing relationships between words and across segments of text. Thanks to its pretraining, the model can for example disambiguate words like crane (bird) and crane (machine) based on context.

From the above, several key implications with direct business consequences become clear:

Limited ControlYou cannot change the model’s internal logic[1]. Influence is limited to input prompts. Furthermore, the model could inherit biases and inaccuracies from the dataset used for training. For compliance or brand tone, guardrails and monitoring are essentia
No up-to-date or Real-Time KnowledgeLLMs contain information only up to a specific cutoff date, which is the point when training was completed. This means they may not reflect recent events, regulations or market changes. For time-sensitive domains, you need strategies to provide current information by integrating external data sources (internet, documents, data bases, etc.).
No Learning During UseThe model does not improve through interaction. Each prompt is independent unless you build external systems to store context.

These properties and the resulting implications shape how LLM-based applications need to be designed and how LLMs can be deployed effectively and responsibly in your organization.

Inside the Black Box: Key Components That Shape LLM Behavior

Most explanations of LLMs are either overly complex or overly simplified. Below is a diagram highlighting a few essential components that help explain the basic behavior of a model like ChatGPT. It doesn’t dive deep into the inner workings of the neural network, as that level of detail isn’t relevant here.

Next, we’ll look at the main components of an LLM and explain what happens at each stage of its processing pipeline. In the diagram, functional blocks that perform the processing are marked with blue dots, while the data they handle or produce are marked with green dots.

	When you enter a prompt, the Tokenizer splits the input text into small pieces called tokens. Each token is then mapped to a numeric identifier (Token ID) from the model’s vocabulary. The output 🅐 is now just a sequence of token IDs and the input text is discarded. Example: “Who was William Tell?” {15546, 574, 12656, 25672, 30}.
	The set of token IDs {15546, 574, 12656, 25672, 30} is fed into a large neural network called the Transformer Model. Based on billions of pages of training data, the model predicts the next token ID. The result is a probability distribution 🅑 for possible next tokens.
	A filter selects the token IDs with the highest probabilities from the distribution, producing a set of candidate token IDs 🅒.
	A random selection algorithm picks one token ID 🅓 from the candidate set. This chosen token is added to the input sequence 🅐, and the Transformer Model processes the extended sequence to predict the next token. This step introduces non-determinism, meaning outputs can vary even for the same prompt.
	The selected token ID is converted back into text by the Detokenizer. As the process repeats, the output text is composed piece by piece.

The components shown in the diagram have important implications that anyone using LLMs should understand. These characteristics make LLMs fundamentally different from traditional software and must be considered when designing, testing, and deploying LLM-based applications.

Here are some of the most important implications resulting from the architecture and inner workings of an LLM:

a) LLMs Emulate Text Without Understanding It

The first step in processing is handled by the Tokenizer, which breaks the input text into pieces and replaces each piece with a number. From that moment on, the original text is discarded and the model works exclusively with numbers.

This is a crucial insight: LLMs do not “understand” language or think like humans. They emulate human-created text by predicting the next token based on patterns learned from vast amounts of training data and mathematical algorithms.

An LLM is not a therapist, not a personal advisor, and not a strategy consultant: it is a machine. Always take its output with a grain of salt and verify its plausibility.

b) LLMs Are Non-Deterministic

Because random processes are involved (see the “Random Picker” in the diagram), LLMs are non-deterministic. The same prompt can produce different outputs each time. This variability also affects quality: sometimes responses are excellent, other times they can be wrong or even inappropriate.

What this means for LLM based applications:

Testing is harder: You cannot rely on fixed outputs for the same input. Test strategies must account for variability and include multiple runs.
Guardrails are essential: Implement filters, validation layers, and human review for critical use cases to catch errors or inappropriate content.
Quality monitoring: Track performance over time and set thresholds for acceptable variation.

c) Hallucinations Are Normal: Why Incorrect Output Is Built In

Generating incorrect or fabricated output, which is often called hallucination, is not an anomaly. It is standard behavior, rooted in the architecture of LLMs. These models predict the next token based on probabilities, not facts and they have no built-in mechanism to verify truth.

What this means for you:

Expect variability: Even well-phrased prompts can produce wrong or misleading answers.
Design for risk: Treat LLM outputs as suggestions, not authoritative answers. Build workflows that verify plausibility before acting on them.

Hallucination is not a bug—it’s a result of how LLMs work. Understanding this helps set realistic expectations and avoid costly mistakes.

Above, we explored the behavior of LLMs and the implications of their inner workings. In the upcoming articles, we will examine the challenges these behaviors create and what they mean for designing and deploying LLM-based applications.

https://www.linkedin.com/pulse/what-every-ceo-needs-know-generative-ai-part-2-how-mjpaf

[1] Some but not all models can be fine-tuned and allow very limited modification of the inner logic.