The voice that survived
The first HALO was deleted. I kept the voice stack: a smaller cache, conditioning loaded once, and audio that starts before the sentence is finished.
Engineering note · Archived prototype, August–September 2026
The first version of HALO was deleted. I kept the voice stack, because that was the part that worked.
This is a note on that earlier prototype, built across August and September 2026. It eventually ran locally on a Raspberry Pi 5. It is a record of the voice system I kept, rather than a description of the current agent's runtime.
The interesting work happened around the model: deciding what to keep in memory, when playback could begin, and which commands should never reach a language model at all.
Stop paying for work you already did
The text-to-speech path used Pocket TTS through a custom ONNX loop. Its exported key-value cache had a fixed capacity much larger than many of the sentences being spoken. Each generation step copied that capacity, even though attention only needed the populated portion.
I made the cache input length symbolic and supplied a cache sized for the sentence. The archived checks recorded identical output after that change. The useful distinction was between a model's available capacity and the work a particular request needed. A fixed allocation can quietly turn capacity into a recurring cost.
The selected voice's conditioning state was another piece of repeated work. It could be loaded once and reused. Re-encoding it for each utterance added waiting without changing what the utterance needed to know.
Neither decision required a more capable model. Both required looking closely at the loop that ran the existing one.
Let the first sound leave
A voice can generate quickly and still feel slow if its interface holds all the audio until the sentence is complete.
The custom loop began playback after the first two audio frames. Generation and audio decoding ran on separate threads. The decoder consumed the frames already waiting, while playback continued through a single audio process. The speaking pipeline could prepare the next text chunk while the current one played.
This made the boundary between stages important. The output was a stream with an evolving queue, not a finished file passed from one function to another. Batching helped the decoder keep up, but the amount of buffered work also mattered when speech had to stop.
The hardware added another boundary: the model produced audio at one sample rate and the speaker accepted another. Resampling needed to carry its state across blocks. A streaming system still has to preserve continuity at each join; making the chunks smaller does not make those joins disappear.
Mute is control flow
The earlier agent once interpreted a mute request as something to answer. That is an especially unhelpful failure for a speaking machine.
I put the mute command in deterministic code. Recognizing the command should change the audio state directly, without asking a language model to decide whether silence is an appropriate response.
The same principle shaped recovery. A reconnected speakerphone was discovered again. A fallback voice covered model startup and failures before sound began. Playback could be terminated if its process stopped accepting audio. These paths were ordinary parts of operating the system, not interesting things for the agent to improvise.
There was also a physical constraint: a speakerphone hears itself. The prototype mostly separated listening from speaking, with a louder interruption able to stop playback and become the next turn. That compromise belonged to the room and hardware as much as to the software.
What was worth keeping
The archived notes contain timings from that particular setup. I am leaving them out here: the decisions are useful without presenting an old local measurement as a general performance claim.
What survived was a set of boundaries I could reason about. Voice conditioning belonged outside the per-utterance loop. Cache work followed the sentence. The first playable audio did not wait for the last frame. Mute and recovery stayed in code.
I would want the same questions answered in the next version: what work repeats, what can begin sooner, what happens when a stage stalls, and who actually has authority to stop the sound?
Those questions are less glamorous than choosing a model. They are also the part I kept.