How AI Works
Understand the foundations of modern AI — from large language models to tokens and context windows. Learn what AI can and cannot do.
5 sections · about 25 minutes
What Is a Large Language Model?
A Large Language Model (LLM) is an AI system trained on vast amounts of text data to understand and generate human language. Models like Claude, GPT, and Gemini are all LLMs. They learn patterns in language — grammar, facts, reasoning styles, and even coding conventions — by processing billions of documents.
Key Insight
LLMs don't "know" things the way humans do. They've learned statistical patterns about how words and ideas relate to each other. This is powerful but fundamentally different from human understanding.
Think of an LLM like an incredibly well-read assistant that has processed millions of books, articles, and conversations. It can draw on all of that to help you, but it's working from patterns, not personal experience or real-time information.
- Training: LLMs are trained on large datasets of text from the internet, books, code, and other sources
- Parameters: Models have billions of adjustable parameters that encode learned patterns
- Fine-tuning: After initial training, models are refined for specific tasks like following instructions or being helpful
- RLHF: Reinforcement Learning from Human Feedback helps models align with human preferences and values
Check yourself
What is the primary way LLMs generate responses?
Tokens, Context Windows, and Why They Matter
When you type a message to an AI, your text is broken into tokens — small chunks of text, roughly 3-4 characters each. The word "understanding" might be split into "under" + "standing". Every AI model has a context window — the maximum number of tokens it can process in a single conversation.
Rule of Thumb
1 token ≈ ¾ of a word in English. So 100,000 tokens ≈ 75,000 words ≈ roughly a 300-page book.
Context windows matter because they determine how much information the AI can "see" at once. Claude's context window is 200,000 tokens — enough to process entire codebases or lengthy documents. But even with large windows, the AI's attention is not equally distributed across all tokens.
- Token limit: Each API call has a maximum input + output token count
- Cost: You pay per token (input and output), so efficiency matters
- Attention: Information at the beginning and end of the context tends to be weighted more heavily
- Context engineering: The practice of carefully managing what information goes into the context window
Check yourself
Approximately how many words fit in a 100,000-token context window?
How AI Generates Text
AI generates text one token at a time through next-token prediction. Given everything that came before, the model predicts what token is most likely to come next. It does this thousands of times to produce a complete response.
This process is probabilistic — the model assigns probabilities to many possible next tokens and samples from that distribution. Parameters like temperature control how random this sampling is: low temperature produces more predictable text, high temperature produces more creative (but potentially less accurate) text.
This Explains Hallucinations
Because the model generates text based on what "sounds right" statistically, it can produce confident-sounding text that is factually incorrect. The model doesn't have a separate fact-checking system — it's all pattern matching.
- Your prompt is tokenized and fed into the model
- The model processes all tokens through its neural network
- It generates a probability distribution over possible next tokens
- A token is sampled from this distribution
- Steps 2-4 repeat until the response is complete
Capabilities and Hard Limits
Modern AI models are remarkably capable at many tasks, but they have fundamental limitations that every user should understand. Knowing these boundaries helps you use AI effectively and avoid costly mistakes.
What AI Does Well
- Writing and editing: Drafting, summarizing, translating, and refining text
- Code generation: Writing, reviewing, and debugging code across many languages
- Analysis: Breaking down complex topics, comparing options, identifying patterns
- Brainstorming: Generating ideas, exploring angles, creative problem-solving
- Explanation: Making complex concepts accessible at any level
- Structured tasks: Following templates, formatting data, organizing information
Hard Limits
- No real-time information: Training data has a cutoff date — the model doesn't know about events after that
- No true reasoning: While models can simulate reasoning, they can make logical errors that a human wouldn't
- Hallucination risk: Models can state incorrect information with high confidence
- No persistent memory: Each conversation starts fresh (unless using special features like Projects)
- No internet access: Unless connected to tools/MCP servers, models can't browse the web
- No emotional understanding: Models simulate empathy but don't feel emotions
Check yourself
Which of the following is a fundamental limitation of current LLMs?
Overview of Major AI Models
The AI landscape includes several major model families. Understanding their differences helps you choose the right tool for each task.
Claude (Anthropic)
Claude is built by Anthropic with a focus on safety, helpfulness, and honesty. The current model family includes Opus 4.6 (deepest reasoning), Sonnet 4.6 (best balance of speed and capability), and Haiku 4.5 (fastest and most cost-effective). Claude excels at long-form analysis, coding, and following nuanced instructions.
GPT (OpenAI)
OpenAI's GPT family includes GPT-4o and o1/o3 reasoning models. GPT-4o is a strong general-purpose model, while the o-series models focus on extended reasoning for complex problems. Widely adopted with a large plugin ecosystem.
Gemini (Google)
Google's Gemini models are natively multimodal — designed to process text, images, video, and audio together. They integrate deeply with Google's ecosystem (Search, Workspace, Android).
Choosing a Model
There's no single "best" model. The right choice depends on your task, budget, and requirements. We tend to reach for Claude because of its instruction-following and safety behaviour, but the right model is the one that fits the task in front of you.
Prompt Engineering