Context Windows

When you chat with AI tools like ChatGPT, Gemini, Claude, or other Large Language Models (LLMs), it often feels like the AI remembers everything you've said. You can ask follow-up questions, refer to earlier messages, and continue a conversation naturally.

But have you ever noticed that after a very long conversation, the AI may forget something you mentioned much earlier?

This happens because every AI model has a Context Window.

Understanding context windows is important because they affect how much information an AI can remember during a conversation, how large your documents can be, and how effectively you can work with AI.

What Is a Context Window?

A Context Window is the maximum amount of information an AI model can process at one time.

Think of it as the AI's working memory for the current conversation.

The context window includes:

• Your current prompt
• Previous messages in the conversation
• Documents you upload
• Instructions you've provided
• The AI's own responses

All of this information is measured in tokens, not words.

Once the context window becomes full, the model cannot keep adding unlimited information. Depending on the system, it may stop accepting more input, shorten responses, or gradually remove older conversation from its working memory.

An Easy Analogy

Imagine writing notes on a whiteboard.

At first, the board is empty, so you can write lots of information.

As you continue writing, the whiteboard fills up.

Eventually, there is no more space. To write something new, you must erase part of what was written earlier.

A context window works in a similar way.

The AI has a limited amount of space to hold the information needed for the current conversation.

What Uses the Context Window?

Many people think only their question counts toward the context window, but that's not true.

The following all use space:

• Your instructions
• Previous conversation
• Uploaded documents
• Code snippets
• Images converted into text descriptions
• The AI's replies
• Your latest question

Everything together must fit inside the model's context window.

Why Is Context Important?

Context allows AI to understand what you are talking about.

Imagine this conversation:

You:

My favorite programming language is Python.

Later you ask:

Can you suggest some good projects?

Because the earlier message is still inside the context window, the AI understands that you mean Python projects.

Without that context, the AI might ask which programming language you are referring to.

This ability makes conversations feel natural and continuous.

What Happens When the Context Window Is Full?

Suppose you've been chatting with an AI for a long time.

Eventually, the context window reaches its maximum capacity.

At that point, several things may happen:

• Older messages may no longer be considered.
• The AI may forget details mentioned earlier.
• Responses may become less consistent.
• You may need to remind the AI about important information.

This is one reason why very long conversations sometimes require repeating key details.

Context Windows and Tokens

As you learned in the previous lesson, AI works with tokens instead of words.

A context window is measured in the number of tokens the model can process.

For example, the context window includes:

• Input tokens, such as your messages
• Output tokens, such as the AI's responses

If your conversation uses more tokens than the model can handle, older content may be removed from the active context.

The exact size of the context window depends on the AI model you are using. Different models support different maximum token limits, and these limits continue to increase as newer models are released.

Why Larger Context Windows Are Helpful

A larger context window allows AI to work with more information at once.

This is especially useful for tasks like:

• Reading long PDF documents
• Summarizing research papers
• Analyzing contracts
• Reviewing source code
• Understanding long conversations
• Comparing multiple documents

For example, instead of uploading one chapter of a book at a time, a model with a larger context window may be able to process the entire book or a substantial portion of it in one request.

Context Windows for Developers

When developers build AI-powered applications, they also need to think about context.

For example, if a chatbot is helping users troubleshoot technical issues, developers often include:

• System instructions
• User profile information
• Previous conversation
• Documentation
• Current user question

All of this information consumes part of the context window.

Developers must carefully choose what information is most relevant so the model can generate accurate responses without exceeding its limits.

A Simple Python Example

The following example sends a prompt to an AI model.

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-4.1",
    input="""
You are a helpful programming tutor.

The user's favorite language is Python.

Suggest three beginner-friendly Python projects.
"""
)

print(response.output_text)

In this example:

• The system instruction
• The user's preference
• The final question

all become part of the model's context window.

The AI uses this combined context to generate a more relevant answer.

Best Practices for Using Context Windows

To get better results from AI:

• Keep prompts clear and focused.
• Avoid unnecessary repetition.
• Summarize long conversations when needed.
• Split very large tasks into smaller parts.
• Repeat important information if the conversation becomes very long.

These habits help the AI stay focused on the most relevant information.