I wanted to automate browsing websites and extracting useful information instead of manually stepping through pages.

To make the architecture concrete, I will use a simple example throughout this post:

Open a news homepage, automatically choose three major articles, visit them, and save their content.

The system runs locally on a Windows PC with an RTX 5080 16 GB GPU. I use a Qwen3.5-9B Q6_K multimodal model through LM Studio.

At a high level:

  • The LLM handles natural-language understanding and semantic decisions.
  • Python + a browser automation library (Playwright) handles deterministic web interaction.
  • WebContentAgent connects the model to those capabilities and manages the task.

Architecture

flowchart TD
    U["User"]

    subgraph LS["LM Studio"]
        UI["Chat AI UI"]
        M["Loaded Model Runtime<br/>Qwen3.5-9B<br/>OpenAI-Compatible API"]
    end

    A["WebContentAgent<br/>Local Tool Service"]

    B["Browser<br/>Chromium"]

    S["Persistent Task State<br/>Checkpoint / Manifest"]

    U -->|"① Natural-language instruction"| UI
    UI -->|"② Inference request"| M
    M -->|"③ Text or Tool Call"| UI
    UI -->|"④ Local RPC via MCP"| A
    A -->|"⑤ Browser automation"| B
    A -->|"⑥ Local HTTP inference"| M
    A -->|"⑦ Persist task state"| S

Below is the request flow step by step.

1. User → Chat UI

I start from the LM Studio chat interface with a natural-language request:

Open this news website.
Choose three major news articles and save them.

The Chat UI acts as the controller. It maintains the conversation, available tool definitions, and previous tool results, then builds an inference request for the local model.

2. Chat UI → Local Model

LM Studio loads the Qwen3.5 model onto the local GPU and runs it using its local inference runtime.

Conceptually, the runtime contains:

Model weights
+
Inference engine
+
GPU execution
+
Local inference API

LM Studio exposes an OpenAI-compatible endpoint:

http://127.0.0.1:1234/v1

The Chat UI sends the current messages and tool definitions to this endpoint.

The input can contain text, images, conversation history, and tool schemas. The model then performs inference and generates output tokens.

3. Model → Text or Tool Call

The model may simply return text:

I found three relevant articles.

Or it may decide that an external capability is required and generate a structured Tool Call:

{
  "tool": "start_collection",
  "arguments": {
    "url": "https://news.example.com"
  }
}

LM Studio parses this generated output into a structured tool invocation.

This is an important distinction: the model runtime performs inference, while the Chat controller interprets the generated Tool Call and decides how to execute it.

4. LM Studio → WebContentAgent

WebContentAgent runs as a separate local Python service and exposes a small set of high-level tools, for example:

discover_site(...)
start_collection(...)
get_run_status(...)
resume_run(...)

LM Studio communicates with this service through MCP — Model Context Protocol.

For this architecture, MCP can be thought of as a standardized RPC interface for AI tools.

The flow is:

Model generates Tool Call
        ↓
LM Studio parses it
        ↓
MCP request
        ↓
WebContentAgent tool endpoint

This is a local inter-process RPC rather than an HTTP call to the model.

The distinction between the two interfaces is useful:

LLM inference:
    OpenAI-compatible HTTP API

Agent tools:
    MCP / local RPC

5. What WebContentAgent Does

Once WebContentAgent receives the request, it orchestrates the actual task.

For deterministic web operations, it uses Python and Playwright to control Chromium.

For example, opening a homepage and extracting all links is straightforward:

await page.goto(url)

Suppose the homepage contains 40 candidate links.

Finding those 40 links is a deterministic programming problem. There is no reason to ask an LLM to inspect every page element one by one.

But the next question is semantic:

Which three of these links are the main editorial articles?

At this point, WebContentAgent calls the same local model again.

This call does not go through the Chat UI. The Agent directly sends an HTTP inference request to:

http://127.0.0.1:1234/v1

For example:

Here are 40 candidate links from a news homepage.

Choose the three main editorial articles.
Return structured JSON.

The model may return:

{
  "selected": [2, 7, 11]
}

The Agent then continues deterministically:

Open article 2
Extract content
Save result

Open article 7
Extract content
Save result

Open article 11
Extract content
Save result

The LLM is only used where semantic reasoning is useful.

One Model, Two Roles

The same loaded model serves two different roles.

Controller role

Chat UI
   ↓
Local LLM

The model interprets the user’s request and decides which Agent tool to invoke.

Worker role

WebContentAgent
   ↓
Local HTTP API
   ↓
Local LLM

The model solves smaller semantic tasks encountered during execution, such as article selection or ambiguous image classification.

These calls share the same model weights loaded on the GPU, but they do not share conversation state.

Each inference request has its own:

Prompt
Context
Images
KV cache

The Worker also does not receive the MCP tool definitions, so an internal semantic inference does not recursively launch another Agent.

Long-Running Agents and Context Overflow

One of the most useful design lessons came from an actual failure.

An early version started a long-running task and then allowed the Chat controller to repeatedly query its status:

start_collection
       ↓
get_run_status
       ↓
get_run_status
       ↓
get_run_status
       ↓
...

In one run, the Chat called get_run_status seven times in roughly eight seconds and eventually failed with:

Context size has been exceeded.

The important part is that an Agent’s Chat context contains much more than visible conversation text.

It also contains:

System prompt
Conversation history
Tool schemas
Tool-call arguments
Tool results

Every repeated tool invocation adds more information to the transcript.

So context usage looks more like:

Messages
+
Tool definitions
+
Tool-call history
+
Tool results

Increasing the Context Window

The first improvement was straightforward.

The model had originally been running with a 16K context window. I increased the validated configuration to 32K.

That provided more headroom, but it did not solve the architectural problem.

An indefinitely growing Chat will eventually fill 32K just as it fills 16K.

Do Not Use Chat as a Scheduler

The more important fix was to change the control flow.

Instead of:

Chat
 ↓
Start task
 ↓
Poll
 ↓
Poll
 ↓
Poll
 ↓
Wait until finished

the system now behaves like this:

Chat
 ↓
Start task
 ↓
Return run-id
 ↓
End Chat turn

The background task continues independently.

Only when the user later asks:

How is the task going?

does the controller issue a single status request.

In other words:

The Chat is the control plane, not the task scheduler.

Durable Task State

A long-running Agent also needs to remember things such as:

Which pages have completed
Which page is currently running
Retry counts
Failures
What should run next

Keeping this information in the LLM context would be both expensive and fragile.

Instead, WebContentAgent writes a few small, durable records to disk:

  • A checkpoint records the latest confirmed progress, such as the last completed page, the current retry count, and what should run next.
  • A manifest lists the pages in the run and the status or saved output associated with each one.
  • An event log records important transitions and failures in time order, making it possible to understand what happened before an interruption.

Together, these records let the Agent reconstruct its application state without relying on the Chat transcript or the model’s context.

This means a task can survive:

Chat closure
Worker restart
Model reload
Process failure

and resume from its last reliable checkpoint.

This leads to an important separation:

LLM context
    → reasoning state

Persistent storage
    → application state

Two Different Kinds of Retry

There is another subtle context issue.

A repeated Chat-level tool call:

Tool Call
 → Tool Result
 → Tool Call
 → Tool Result

directly grows the Chat transcript.

A Worker inference retry is different:

Prompt
 ↓
Invalid JSON
 ↓
Validation error
 ↓
Retry inference

This grows only that Worker’s current inference context.

WebContentAgent therefore manages a separate inference budget for Worker requests and checks that the request, expected output, image budget, and safety margin all fit inside the model context before sending the HTTP request.

So there are really two separate context problems:

Chat Context
    Controller conversation + tools

Worker Context
    One bounded inference request

Treating them separately makes long-running Agents much easier to reason about.

End-to-End Example

The original news example now looks like this:

User
 ↓
LM Studio Chat UI
 ↓
Local LLM
 ↓
Tool Call
 ↓
WebContentAgent
 ↓
Browser extracts 40 candidate links
 ↓
Agent calls Local LLM
 ↓
LLM selects 3 articles
 ↓
Browser visits the 3 pages
 ↓
Python processes the content
 ↓
Checkpoint + results

The model participates only where semantic reasoning is valuable.

Everything else remains normal software.

Appendix: A Few Practical Bugs

A few other issues showed up while running the system for real.

Browser sessions are independent

A page being logged in inside my normal browser does not mean the Agent’s isolated Chromium profile is logged in. The Agent has its own browser session and must validate that the expected content is actually visible there.

Windows path length is real

Deep directory hierarchies combined with long page titles and image filenames eventually hit Windows path-length limitations. The fix required budgeting filename lengths against their final paths rather than only their temporary staging paths.

A background process is not automatically a service

Starting a Worker process makes long-running tasks independent from a Chat turn, but it does not provide Windows-service-level lifetime guarantees. Durable checkpoints make interruption recoverable even when the process itself disappears.

Conclusion

The resulting Agent system is not just an LLM with browser access.

It is a composition of:

Local LLM
+
Chat controller
+
Tool RPC
+
Agent service
+
Browser automation
+
Persistent task state

The LLM handles semantic decisions, deterministic software performs the execution, and persistent state keeps long-running work recoverable.