Prose Assistant Strawman Design Approach
Possible Initial Modes
- Conversation / Sidebar Mode — AG talks with the assistant about the story, characters, ideas, problems, or alternatives. This material does not automatically become manuscript text.
- Narrative Mode — Gemma is writing or revising actual prose intended for the manuscript.
- Character Mode — AG interacts from the perspective of, or about, a particular character. The system loads the relevant character information.
- Bulk / Review Mode — Larger sections of manuscript can be submitted for analysis, continuity checking, summarizing, or revision.
- Continue Mode — Gemma continues directly from the current manuscript position using the immediately relevant story state.
Instead of putting everything into one ever-growing prompt, maintain several categories of information:
- Permanent instructions would contain Allen’s writing rules, style preferences, and instructions for how the Prose Assistant should behave.
- Story knowledge would contain relatively stable information such as characters, locations, relationships, background, world rules, and important established facts.
- Current story state would contain what is happening now: current chapter, scene, location, characters present, unresolved issues, and recent events.
- Working text would contain only the manuscript material needed for the present task — perhaps the last several paragraphs or the current scene rather than the entire book.
The selected mode determines which of these pieces Gemma receives.
AG should be able to tell the Prose Assistant what he is doing without constructing prompts manually.
For example:
- Sidebar
- Narrative
- Character: Sarah
- Bulk Review
- Continue
Eventually these could be buttons or menu choices, but initially simple commands are probably better because they let us test the behavior before spending time on the interface.
The first prototype should answer one question:
Can Gemma reliably behave differently depending on the selected mode?
For example, we could run a 20–30 turn session where AG moves between Sidebar and Narrative, asks for one paragraph, discusses it, returns to Narrative, and uses Continue.
At this stage we deliberately avoid building an elaborate long-term memory system.
The initial prototype should also help determine what development environment is required. Where possible, early experiments should be performed within Open WebUI by modifying the Prose Assistant’s instructions and testing Gemma directly. If the desired behavior cannot be implemented or reliably controlled within Open WebUI, the prototype can then move to a code-assisted approach using Debbie (Claude Code).
Once Gemma follows the mode protocol reliably, the application can begin maintaining the story state automatically.
For example, after a scene is completed, the system might update:
- characters involved
- current location
- important events
- newly established facts
- unresolved plot points
- current manuscript position
That compact state could then accompany future Narrative or Continue requests without resending hundreds of pages of old conversation.
This is probably the most important architectural point in Jason’s proposal.
Gemma should not be responsible for remembering everything.
The Prose Assistant application should decide:
“For this request, what does Gemma actually need to know?”
That means the application assembles the prompt from the appropriate instructions, story information, current state, and manuscript excerpt based on the selected mode.
This gives us a path around the context-window problem instead of trying to make the context window infinitely larger.
The project at this stage is about choosing between having Gemma maintain the protocol/mode state within Open WebUI, versus developing an application that maintains the protocol/mode state and uses Gemma as the prose engine.
There are 6 classes of decisions to be made
Option A — Open WebUI configuration
Smaller project and closer to what we already have, but Gemma bears responsibility for maintaining protocol/mode state. We would need to demonstrate that Gemma can do that reliably.
Option B — Prose Assistant application
More development work, but the application owns mode/state/context management and Gemma becomes primarily the prose engine. This also gives us much more control over exactly what enters Gemma's context window.
The Prose Assistant must not be Gemma-specific or Open-WebUI-specific. The language model should be a replaceable component.