If you’ve been around AI for any length of time, you’ve come across the idea of prompts, the way you give instructions to an LLM like ChatGPT. As it turns out, there are things you can do to better craft your prompts to improve the quality of results. That craft is known as “prompt engineering.” But here’s what’s fascinating: as context windows have exploded from 4K to over 1M tokens in the last two years, something more powerful has emerged, Context Engineering.
What is a Context Window?
Think of it as a big sheet of paper that you pass to the LLM. You can write a certain number of words on it. Those words can be “write me a limerick about pickleball” or “here’s a 5,000-word essay, give me a 100-word summary” along with the entire essay itself.
Context windows have a limit, usually expressed as a number of “tokens.” A token is roughly ¾ of an English word or about four characters. “ChatGPT” is two tokens: “Chat” and “GPT.” This matters because billing, latency, and memory all scale with token count.
Key limitation: LLMs only know what they were trained on and what’s provided to them in the context window.
How Context Windows Are Used
Conversational Memory
You may have noticed that ChatGPT and Claude have good short-term memory. That’s why you can ask follow-up questions and they know what you’re talking about. Behind the scenes, they’re actually passing your prior prompts and outputs into the context window. It’s like the movie Memento, where the main character forgets everything and has to write notes on his body to remember who he is.
RAG: Teaching AI on Demand
Retrieval-Augmented Generation (RAG) finds documents related to your prompt and passes them into the context window, “teaching” the LLM precisely the things that will help it answer your question.
Tool Calling: Extending AI’s Capabilities
Tool calling tells the LLM: “Hey, pretend like you have access to these tools for web search, stock price lookups, weather forecasts, whatever.” LLMs can then invoke one or more tools to help process a prompt.
Here’s a quirky detail: when we let the LLM know what tools it has available, it doesn’t actually call the tools itself. It just takes context in and produces output. The app then invokes the tool on the LLM’s behalf and puts the results back in the context window. It’s a clever workaround.
With a million-token context window, you can feed an AI an entire codebase, a complete business plan, or months of customer support conversations, and it can reason across all of that information simultaneously.
The Shift from Prompt to Context Engineering
Remember the prompt engineering craze of 2023? Headlines pushed $300k+ salaries for “Prompt Engineers” at places like Anthropic. Many jobs went viral because they required no formal CS degree, just the ability to speak fluent “LLM.”
While some prompt engineers were hired, it didn’t become the massive job category many predicted. The reason? Everyone became a prompt engineer. And now, something even more important is emerging.
The Simple Explanation:
- Prompt engineering was like learning to ask really good questions
- Context engineering is like being a librarian who decides what books someone has access to before they even start reading
What Context Engineers Actually Do
- Curate: Decide which documents, memories, or APIs matter for each specific task
- Structure: Layer system messages → tools → retrieved data → user prompt in optimal order
- Compress: Summarize or chunk information to stay under token limits while preserving what matters
- Evaluate: Measure accuracy and watch for “context dilution” where irrelevant info distracts the model
More context means richer documents and longer conversations, but cost and latency rise roughly linearly with window length. This leaves significant room for discovering best practices, many of which are still emerging.
Prompt vs. Context Engineering
| Aspect | Prompt Engineering | Context Engineering |
|---|---|---|
| Focus | How you ask | What AI has access to |
| Skill | Crafting effective instructions | Information architecture and data strategy |
| Scope | Single interaction | Entire system design |
| Analogy | Asking good questions | Curating the right library |
| Value | Optimizing sentences | Optimizing knowledge |
Why This Matters
Context engineering requires thinking about information architecture, data strategy, and user experience in ways prompt engineering never did. The companies who master this will have a massive competitive advantage.
Key insight: Prompting tells the model how to think, but context engineering gives the model the training and tools to get the job done.
Stop thinking about AI as a chatbot and start thinking about it as a reasoning engine with access to the right context and tools. The prompt engineering era taught us to talk to AI. The context engineering era is teaching us to think with AI.
Disclaimer
The information provided on this page is for educational purposes only. Concepts and terminology in the AI field evolve rapidly. We recommend staying current with developments from leading AI research organizations and practitioners. AI Crash Course assumes no responsibility for outcomes resulting from the application of these concepts.
