Part 1: What LLMs Actually Do (First Principles)

The Core Truth: Token Prediction

LLMs don't think, reason, or understand. They predict the next token.

What's a token?

• A piece of text (word, part of word, or character)
• Example: "Hello world" → ["Hello", " world"] (2 tokens)
• Example: "strawberry" → might be ["straw", "berry"] or just ["strawberry"]

What the model does:

Given: "The capital of France is"
Model predicts: "Paris" (most likely next token based on training)

That's it. Every single thing an LLM does is just repeating this process over and over.

Part 2: What is "Thinking" or "Reasoning"? (Spoiler: It's Not Real Thinking)

The Marketing vs Reality

Marketing says: "The model thinks through the problem step by step"

Reality: The model generates more tokens that look like thinking before giving the final answer.

Example: Simple Math

Question: "What is 547 + 832?"

Without "thinking" (direct prediction):

Input: "What is 547 + 832?"
↓
Model predicts: "1379"