This guide exists because "just run a local LLM" hides a surprising number of moving parts. Each chapter below breaks one piece down — diagram first, plain-English explanation second.
Five layers, each depending on the one below it. A request from a person travels down through a front end, a model manager, the model itself, and finally the physical hardware doing the math.
A browser-based chat interface. It doesn't run any AI itself — it sends your messages to Ollama and displays what comes back, styled like a familiar chat app.
Ollama downloads models (in layers — see Engineering Notebook, Note 004), keeps them on disk, loads them into memory on demand, and exposes a local API that front ends like Open WebUI (or Claude Code, or a custom script) can talk to.
One Ollama instance can manage several models at once, swapping them in and out of memory as different tasks call for them.
A Large Language Model is a neural network trained on huge amounts of text to predict, one piece at a time, what comes next. Ask it a question and it's generating its answer token by token, guided by everything it learned during training.
Not all LLMs are trained for the same job. Coder models (like Qwen2.5-Coder) are tuned for writing and reasoning about code, and may lean toward structured, tool-call-style output even in casual chat (see Note 005). Conversational models are tuned for natural back-and-forth dialogue instead.
Your text is broken into tokens → sent to the model → the model predicts the next token, then the next, then the next → tokens are streamed back and decoded into readable text. All of this happens locally, on the hardware in the room with you.
CPU and GPU (or NPU) do the math; RAM holds the model while it's running; SSD stores the model files between runs. More RAM and a stronger GPU/NPU let you run bigger, smarter models at usable speed.
Same table shows up on the Hardware page too — it's the answer key for half the questions in this chapter.
See Hardware for the full breakdown of what the lab is actually running on.
Models are described by parameter count (like "14B" = 14 billion parameters) — roughly, bigger means smarter but slower and hungrier for RAM. Quantization shrinks a model's memory footprint (e.g. 4-bit) at a small cost to precision, making larger models usable on modest hardware.
Agents sit a layer above plain chat — they can read and write files, run commands, and carry out multi-step tasks rather than just answering questions. Claude Code is the agent building and documenting this very lab.
Put it all together and you get the lab's actual setup: two boxes, two operators, connected over the internet.
Two boxes, one shared network — that's the entire lab today, no data center required.
The interim target adds each operator's everyday computer into the picture too, so the lab can eventually be reached from more than just the dedicated mini PCs:
The "someday" version — same two lab boxes, plus a way in from the everyday computers already on our desks.
See the full write-up on the Architecture page.
Mobile Access — LAN and Tailscale — reaching Open WebUI from an iPhone on Wi-Fi, and from anywhere over cellular.
Read →Every remote-access and reliability test, numbered and dated — the evidence behind Parts 2–4, with a mobile-friendly summary and the downloadable spreadsheet.
Open →