Personal AI assistant
Local prototype ยท In development
This project explores reliable conversation execution: preserving state across retries and interruptions, keeping context bounded, validating streamed responses and checking budgets before generation.
The engineering focus is the execution lifecycle around the model. Idempotent replay gives repeat requests a stored result, while explicit interruption handling avoids silently starting another generation.
A conversation is more than a model call
A chat interface has to handle more than a successful answer. A client can retry a request, disconnect during a stream or return after the process restarts. Conversation history also grows beyond what belongs in a single prompt.
The design treats each generation as a persistent execution with its own identity and outcome, rather than tying its lifecycle to an open connection.
Execution, context and streaming decisions
Each new turn is checked against an enabled model profile, assembled context and budget admission before generation starts. Reusing a request identifier with the same content and profile replays the existing execution. Changing either returns a conflict, so a retry can't silently become a different request.
The persistence contract stores output snapshots before delivery and retains terminal outcomes for later reads. Disconnecting leaves generation running; an explicit Stop persists the stopped state and requests upstream cancellation.
The streaming path checks the response format, completion marker and output bounds before reporting completion. Receiving text alone isn't enough to mark an execution successful.
Context selection keeps recent turns and uses deterministic extraction to summarize older material. The summary is marked as lossy conversation data, never policy. The entire assembled prompt must fit the configured character bound; an oversized latest turn is rejected rather than silently shortened.
From request to stored outcome
- Request
- Context selection
- Budget admission
- Model stream
- Execution state
These stages show the main responsibilities. Persistent execution state connects admission, output delivery and later replay, so delivery isn't the only record of what happened.
A fresh request goes through context and budget checks. A repeat request reads the existing execution instead of repeating admission or inference.
Trade-offs
Bounded context and lost detail
Extracting older turns leaves room for recent context without another model call, but fragments can miss details or relationships. The raw conversation remains stored; the prompt contains only a bounded view of it. A character limit makes prompt size explicit, but it isn't an exact token count.
Conservative interruption handling
After a restart, running executions are marked interrupted rather than automatically rerun. This favors avoiding duplicate generation over automatic recovery: even unstarted work can become interrupted, and a usage reservation can remain held. Starting a new generation requires an explicit new request identifier.
Project scope
This is a local, single-owner assistant in development, not a deployed service. The current chat contract keeps retrieval providers, tools and uploads disabled so the work stays focused on conversation execution.
Development testing uses local storage and runtime components with a test-only provider to exercise the execution lifecycle.