← Part 4: Always-On Server  ·  AI Lab Portal Home

This page is a live document. Findings current as of 2026-07-26 — Tests 20 & 21 remain unchanged; the displayed-timer question this page's own methodology raised has since been root-caused as an upstream Open WebUI bug, not a lab issue — see Test 24.

Part 4 fixed whether Lab PC 2 stays reachable after extended idle. This page answers a different complaint that surfaced along the way: even when reachable, a phone query through Open WebUI takes about a minute while the model itself reports only ~30 seconds of thinking. Where does the rest go?

Core distinction

Part 4 = the server stays reachable.

Part 5 = the response stops feeling slow, and the reason why is measured, not guessed.

Earlier drafts of Part 4 blamed the cold/warm gap on Ollama reloading the model from SSD. That theory was never measured before it was written down — this page is the record of measuring it, and of the correction that followed.

The Two Paths #

To find out where the missing seconds were going, a small diagnostic bench (ollama-timing-bench.html) was built to call Ollama's /api/generate endpoint directly from the iPhone over Tailscale — the same network path as the chat UI, but with Open WebUI removed from the equation entirely.

Diagram comparing the Chat Path through Open WebUI at about 44 seconds against the Measurement Path bypassing Open WebUI at about 17 seconds
Chat Path (through Open WebUI) ≈44 s vs. Measurement Path (bypassing Open WebUI) ≈17 s — both warm, single runs (Test 20).

The gap between the two panels — about 27 seconds — is what pointed the investigation at Open WebUI rather than at Ollama or the network. Ollama's API already reports load_duration, prompt_eval_duration, and eval_duration, but Open WebUI's chat interface never surfaces them, which is exactly why a separate instrument was needed to see them at all.

What the Bench Measured — Test 20 #

Three runs of the standard prompt ("Answer in one sentence only: why do cats purr?"), from the iPhone, July 10. Run 1 immediately followed a forced model unload to guarantee a true cold start:

RunWallModel loadPrompt evalGenerationNetworkTok/s
20a — cold21.8 s4.5 s0.2 s16.9 s0.0 s14.5
20b — warm16.6 s0.2 s0.1 s16.3 s0.1 s14.5
20c — warm17.2 s0.2 s0.1 s16.8 s0.1 s14.5

The assumption was wrong

Cold model load costs 4.5 seconds (Test 20a) — not the 77–90 seconds earlier drafts attributed to it. Generation time is identical cold or warm (16.3–16.9 s, Tests 20a–c), and throughput holds at 14.5 tokens/sec on every run. Network transit over Tailscale from an iPhone on cellular costs roughly a tenth of a second.

Ollama's total work on this prompt is ≈17 s warm, ≈22 s cold (Test 20). The same prompt through Open WebUI ran ≈44 s warm and ≈121 s cold (Part 4, Tests 19/20). The overhead lives in Open WebUI, not in Ollama and not in Tailscale.

Test 20 Result: Cold model load measured at 4.5 s — the ~77 s cold/warm gap does not come from model reload

Where the Overhead Actually Goes — Test 21 #

Test 20 showed where the missing time wasn't. Test 21 tested the leading suspect: Open WebUI fires its own automatic model calls per exchange — chat title generation, tag generation, and follow-up-question generation — each one a full qwen3:8b inference that never appears in the chat UI's "thought for N seconds" label.

Diagram showing Open WebUI's background Title, Tags, and Follow-Up calls stacking on top of the chat response when features are enabled, versus a single clean path when they are disabled
Background features enabled — three separate full-model calls (Title, Tags, Follow-Up) stack on top of every exchange — vs. disabled, a single clean path. Median of 6 runs, HP Edge (Test 21).

Each of those three features is its own complete LLM call — one user question quietly becomes four separate inference runs on the same single-threaded local model, all queued behind each other. That queuing also explains the "Load failed" timeout seen back in Test 17a: the first real request was waiting behind background calls, not behind a slow disk.

The Toggle Test — Result #

Method: HP desktop, Edge, one browser, standard prompt, a fresh chat for every run, model kept warm throughout. Baseline with Title, Tags, and Follow-Up all switched on, then all three switched off (Admin → Settings → Interface).

ConditionThinkingTotal (median)OverheadVariance
Features ON (Title + Tags + Follow-Up, all on qwen3:8b)11–13 s~58 s18–86 swild
Features OFF~11 s~13 s~2 sgone

The decisive detail is the shape, not just the average

Thinking time stayed rock-steady at ~12 seconds in both conditions — the model itself never varied. All of the overhead, and all of the run-to-run variance, lived in the background calls. Turning them off collapsed the total to the Ollama floor (~13 s, matching Test 20's warm measurement) and removed the swing entirely.

Honest caveat: the first 1–2 runs immediately after saving the setting were still slow (39 s, 52 s) — likely a queued background task draining, or the setting propagating — before the effect fully settled. Later runs were consistently ~13 s.

Test 21 Result: Disabling Title/Tags/Follow-Up collapsed the chat path from ~58 s median to ~13 s median, and eliminated the run-to-run variance

Result and Tuned Path #

Confirmed: the tuned chat path runs ≈13 s (median, Test 21), down from ≈58 s (median, Test 21) — roughly a 45-second reduction, and the latency is now predictable instead of swinging between 18 and 86 seconds. The fix adopted was to leave Title, Tags, and Follow-Up switched off, at the cost of losing those three conveniences.

Not yet tested — a deferred Task Model alternative: pointing Open WebUI's Task Model (Admin → Interface → Task Model) at a small model, such as qwen3:0.6b, instead of the full qwen3:8b, could in principle keep automatic titles/tags/follow-ups while cutting their cost. This is an untested idea, not a recommendation — it has never been run and has no measurement behind it. (Test 22, as actually run, became the RAG known-answer baseline — see the Test Log.)

Reconciling the Two Figures' Numbers #

Figure 1 (above, Test 20) uses 44 s / 17 s — single runs, iPhone, Ollama bench vs. chat UI. Figure 2 (above, Test 21) uses 58 s / 13 s — medians of six runs, HP desktop on Edge, chat UI only, comparing features on vs. off. These are two different measurement methods answering two different questions, not a contradiction: Test 20 isolates Ollama's own time; Test 21 isolates the effect of one specific fix. Each figure's caption names its own test — read the numbers against the test that produced them, not against each other.

Lessons Learned

  1. Measure at the layer you suspect — Ollama had been reporting load_duration all along; the number was one API call away, hidden only because the chat front end never displayed it.
  2. An unmeasured cause is a guess, however confident. The cold/warm gap was attributed to model reload for days; measured directly, reload costs 4.5 seconds, and the missing ~77 seconds belonged to Open WebUI instead.
  3. The overhead was never the model and never the network — it was the front end doing invisible extra inference. A steady thinking-time next to a wildly varying total is the signature of background tasks queuing against a single-threaded local model.

What's Next

Back to Part 4

Always-On Server — the Modern Standby fix and the tests that proved reachability survives extended idle.

Read →

Reference: Test Log

Every remote-access and reliability test, numbered and dated — including Tests 20 and 21 behind this page.

Open →

← Part 4: Always-On Server  ·  Test Log  ·  AI Lab Portal Home

Where the Seconds Go: Live document — findings current as of 2026-07-26 (Tests 20 & 21 unchanged) · displayed-timer question root-caused as an upstream Open WebUI bug, see Test 24