Inside murmur: prefetching, interruptions, and audio scheduling

Preparing ahead can reduce the wait between segments. The first batch still has to be generated after startup. An interruption also needs a fresh generation request: the old program can keep playing while that request runs, but the reply still takes time. These are three separate latency measurements.

The local program owns the schedule

The main application is TypeScript running on Node.js. Its terminal UI runs in a separate process using Bun and OpenTUI. Model inference and production speech synthesis depend on external services.

Figure 1 groups the components by responsibility: blue for local orchestration, yellow for content preparation, purple for context, and green for audio. The arrows show interactions; preparation tasks can run concurrently.

Figure 1. The Director controls when content airs; AudioEngine controls audio state. Music preparation also calls the model to select songs. That work is grouped inside Music here.

The model submits results through tools

Brain uses the Claude Agent SDK. Each task gets its own tool set. The harness disables the CLAUDE.md files, skills, and arbitrary MCP tools from the listener’s everyday coding environment so those settings do not affect the radio.[2]

For example, the model submits a batch of segments through emit_talk_beats. The program validates the tool arguments against a schema and reads the text, without having to extract a result from free-form prose. When the listener asks for another song, the model calls a music-switching tool. A callback owned by the Director performs the action.

Song introductions follow the same separation of responsibilities. The Director starts the source and waits for the engine to confirm that actual audio has been queued. Only then does it update “now playing,” record the song, and play the introduction. This avoids introducing a song whose source never started. The check reaches as far as audio scheduling; it cannot prove that sound came out of the speakers.[3]

What goes into each generation request

Before each request, the program assembles a bounded context package:

• The persona describes the host’s identity. Once created, everyday conversations do not automatically rewrite it.
• The listener profile holds selected, consolidated long-term information. Executed one-off control instructions are not directly learned as preferences.
• Recent conversation keeps the current thread going. Earlier material can be retrieved when needed.
• Broadcast history records songs, topics, and time-based anchors to help avoid repetition.

These live in local Markdown and JSONL files. Profile consolidation runs in the background without blocking playback. If it fails, the previous profile and processing checkpoint remain intact.[4]

Selected context is sent to the inference service, and text to be spoken is sent to the speech service. Data still leaves the machine.

The terminal UI only displays state and receives input. It exchanges newline-delimited JSON with the engine over a Unix socket, with schema validation at both ends. Without Bun, the application can fall back to a plain-text interface using the same scheduler.

Preparing the next segment during playback

Suppose preparation begins with R seconds left until the planned handoff, and the work takes P seconds. Ignoring playback startup overhead and failure retries, the extra wait at the handoff is:

Extra wait at the handoff: W = max(0, P - R) 

Reducing P helps. Starting earlier also helps, because preparation can overlap the current segment’s playback.

Figure 2 assumes a 30-second segment and 12 seconds of preparation for the next one. Both numbers are illustrative.

Figure 2. Prepare B while A plays, then play B after A ends. Diagram lengths do not represent elapsed time.

The Director maintains two kinds of prefetch.

The talk buffer has a target depth of two segments. Each entry holds text and a speech-synthesis Promise that has already started. After consuming an entry, the Director refills the buffer in the background, with at most one refill task in flight. Being in the queue does not mean synthesis is complete; playback may still have to wait for the Promise.

Music prefetch has one slot. Search, selection, and source resolution run in the background. If the pick is not ready at a planned music boundary, the Director plays another talk segment and checks again at the next boundary.

A deeper queue could absorb larger variations in preparation time. It would also spend more on generation, and content farther back in the queue could become stale before airing. The current depth has not been established as a global optimum.

Prefetched content can become stale

When refilling the talk buffer, the model needs to know what is already queued. Otherwise, it may write about the same topic again. We include queued text in the generation context so new segments follow it. Broadcast history is still updated only when the content airs.

The transition out of a song needs separate handling. If the next talk segment was written before the song started, its context does not include that song. Playing it several minutes later can abruptly return the listener to an earlier topic.

After a song starts, the system generates a short closing link, or coda, with the current song in context. It can play over the song’s ending or go to the front of the queue after the song finishes.

What happens to the queue when the listener interrupts

Suppose the radio is playing A, with B and C buffered. The listener types, “Let’s stop talking about this.”

B and C no longer fit. Appending the reply to the queue would make the listener sit through two more segments on the subject they just asked to leave.

murmur handles it in this order:

1. Clear the old talk queue and invalidate its in-flight refill work.
2. Keep the current audio playing while generating and synthesizing the reply.
3. Once the reply audio is ready, stop any remaining old voice playback and start the reply.
4. Refill the queue using the updated conversation context.

If another line arrives while the reply is being prepared, it is merged into the reply, and the superseded preparation task is invalidated. An ordinary interruption leaves the song playing. The mixer lowers its volume when the voice comes in.

Clearing the queue alone is not enough. An old request could return a few seconds later and put obsolete text back into it.

The code uses an incrementing version number, epoch, to decide whether a result is still valid. The talk-refill path can be simplified to:

const startedIn = talkEpoch const beats = await generateTalks() if (startedIn !== talkEpoch) return // Context changed; discard this batch. enqueue(beats) 

An interruption increments talkEpoch. An old task may keep running, but its result is discarded when the versions no longer match. Interactive tasks use similar checks to prevent later tool calls from a superseded task from changing state.

This only blocks results that have not yet taken effect. Completed actions are not rolled back, and model requests already sent may continue consuming resources.

murmur tries to wait until the reply audio is ready before switching, so the old voice may continue for a while after the listener presses Enter. That reduces silence, but the time until the reply is heard still depends on generation and synthesis. Tests need to measure silence and reply latency separately.

Mixing voice and music

murmur uses node-web-audio-api to build one audio graph for voice, the main song, and the background bed. The engine schedules gain changes ahead of time on the audio clock, without relying on JavaScript timers to change volume during playback.[5]

Figure 3. Solid arrows carry audio; the dotted arrow shows control. Voice changes the song channel’s gain without pausing the song.

The current settings lower the main song to a linear gain of 0.3 over about 0.3 seconds, then restore it over 2.5 seconds after the voice ends. A gain of 0.3 is an amplitude ratio. It does not mean “30% as loud.”

The drop needs to be quick enough to keep the song from covering the voice. A slower recovery avoids a sudden jump in volume immediately after a sentence ends. These values were adjusted by listening; they are not universal.

The background bed stays steady during speech. It only crossfades when the main song enters or leaves. Making it rise and fall with every sentence would produce audible pumping.

Speech synthesis currently waits for the service to return a complete audio clip before playback starts. Knowing the clip’s duration makes it easier to schedule music recovery, handle interruptions, and join clips. It also means the first line and each reply must wait for the whole clip to be ready.

Long sources such as songs are decoded and queued onto the playback timeline in chunks. The whole song does not have to be loaded first.

Measurements

Across two historical runs using the real model, real music-selection tools, and the production speech service, all 13 boundaries that consumed prefetched talk entered playback in the same logged second as talk.buffer warm. These included two transitions from music back to talk. The program still kept its default two-second pause between segments.[1]

Prefetch covered the generation wait at those boundaries. The logs only have one-second resolution, so they do not establish zero latency. There are also too few samples to calculate a meaningful long-run P95.

There is a separate before-and-after record for the first song:

The experiments used separate data directories, preset personas, cached background beds, and a fixed “listener present” signal. The changes included starting music selection earlier, simplifying search, and limiting the selection context. The improvement in the table reflects those changes together. These numbers come from small-sample engineering records in the repository; I did not rerun the measurements for this article.

After optimization, the song was ready roughly 30 seconds before it played in both runs. It still had to wait for the third segment boundary. The scheduling rule at the time required two talk segments first, so making search faster would not have brought the song forward in those runs.

Cold-start waiting remains. Historical measurements put the first audible voice at roughly 29 to 39 seconds. Prefetch did not cover that first batch.

Music discovery also varies considerably. In another real log, five selections took roughly 82 to 192 seconds each. Talk can continue during selection, but the listener may wait a long time for the next song.

Natural transitions still need listening tests. Even with a buffer hit, ready audio, and correctly scheduled gain changes, a sentence can fail to follow the previous content. An interruption can land at an uncomfortable moment.

We use model-free tests for scheduling and invalidation rules, and offline audio rendering for gain changes and handoffs. Latency, source failures, and continuity need runs against real services, followed by actually listening to the result.

Implementation and measurement sources

The implementation described here was checked against d6c3619. Performance figures come from existing engineering records and were not remeasured for this article.

1. Historical measurements, run conditions, and reproduction entry points.
2. Model isolation and task tools, interruption tasks.
3. Scheduling, prefetch, version invalidation, and audio-start confirmation.
4. Local memory storage, background profile consolidation.
5. Audio graph and mixing parameters, complete-clip speech synthesis.

Project repository: wine-fall/murmur.