Showing the Work: Progress Streaming for Catalog-Backed Chat

When a catalog-backed chatbot takes more than a second to answer, silence reads as failure. The fix is not making retrieval faster. It is streaming which stage is running so the user knows the system is working.

This post describes a design we are committing to for HoverBot catalog skills: six named progress stages, a capability-gated SSE transport that degrades to a plain JSON call, an absolute wall-clock deadline, AbortSignal cancellation, and five explicit stream outcomes. The design is in open PR #3 and open PR #6, and is not live in production yet. Alexander Khomenko authored the implementation in commit a6d8ac3c across hoverbot-api, hoverbot-config-ui, and hoverbot-widget.

Why Silence Fails Before Search Does

Catalog retrieval is a multi-step pipeline. The orchestrator interprets the user's message, resolves product references from prior turns, calls a search backend, validates returned SKUs, ranks candidates, and only then assembles a response. Each step can add hundreds of milliseconds to several seconds depending on catalog size, query complexity, and backend load.

Users do not experience that as a pipeline. They experience a chat bubble with a typing indicator that never changes. Nielsen Norman Group's response-time research identifies one second as the threshold where flow breaks and ten seconds as the point where attention is lost. A catalog query that finishes in four seconds is fast enough to be correct and slow enough to feel broken if the UI says nothing.

The instinct is to optimize latency. That is worth doing, but it does not solve the perception problem. Even a well-tuned retrieval can spike when a user asks a comparison question across three product lines with constraint filters. You cannot guarantee sub-second answers for every catalog turn. You can guarantee the user sees which stage is running.

This connects to the broader knowledge-retrieval picture in knowledge management for AI chatbots: retrieval quality depends on what you fetch, but retrieval UX depends on whether the user waits with context or waits in the dark.

The Six Stages

Progress updates are typed against a fixed stage list. No free-form status strings from the backend; the adapter and widget agree on six values:

export const CATALOG_PROGRESS_STAGES = [
  'interpreting',
  'resolving_reference',
  'retrieving',
  'validating',
  'ranking',
  'building_response'
] as const;

export interface CatalogProgressUpdate {
  stage: CatalogProgressStage;
  detail?: string;
  elapsedMs: number;
}

Each stage maps to a user-facing message in the chat controller:

• interpreting: Understanding your request. The orchestrator parses intent, constraints, and search mode before touching the catalog.
• resolving_reference: Resolving the products you mentioned. Handles ordinals ("the second one"), pronouns, and context handoff from prior turns.
• retrieving: Searching the catalog. The HTTP adapter calls the search backend. This is usually the longest stage.
• validating: Checking product information. Confirms returned SKUs exist, are in scope for the tenant, and match the query constraints.
• ranking: Ranking the best matches. Reorders candidates by relevance, availability, or business rules before presentation.
• building_response: Preparing the results. Assembles the final message, product cards, or clarification prompt the user will see.

The optional detail field carries adapter-specific context (for example, a category name) without expanding the stage vocabulary. The elapsedMs field is wall-clock time since the search started, useful for logging and for deciding when to show a "still working" fallback message.

Not every query runs every stage. A first-turn category browse may skip resolving_reference. A cache hit might flash through retrieving in under 50ms. That is fine. The stages describe what is happening when it happens, not a mandatory sequence with equal duration.

Capability-Gated Transport

Progress streaming is opt-in at three levels. Adapter config exposes progressStreaming?: 'auto' | 'off'. The default in config-ui is 'off'. Tenants turn it on explicitly.

When set to 'auto', the HTTP search adapter checks four conditions before opening an SSE stream:

1. Config says progressStreaming: 'auto'.
2. The caller sets supportsProgressStreaming: true.
3. The caller provides an onProgress callback.
4. The catalog backend's /health endpoint advertises capabilities.progressStreaming with version: 1, transport: 'sse', and a stages array.

If any check fails, the adapter logs the transport selection and falls back to a standard POST that returns JSON when complete. No error, no broken widget, no second integration path for non-streaming clients.

The capability probe is cached for 60 seconds per base URL so every catalog turn does not pay an extra health round-trip. The transport itself uses Server-Sent Events as defined in the WHATWG HTML specification: a long-lived HTTP response where the server pushes event: and data: frames. SSE fits progress updates because they are server-to-client, unidirectional, and small. The chat API already uses SSE for answer streaming on compatible clients; catalog progress rides the same pattern.

Why degrade instead of requiring SSE everywhere? Embedded widgets run on third-party sites with varied network stacks, corporate proxies, and older mobile WebViews. Some cannot hold an SSE connection reliably. Forcing SSE would break those clients or require maintaining two widget builds. Capability gating lets streaming clients get stage updates and everyone else get the same final answer through JSON.

The Deadline and Cancellation

Streaming progress does not remove the need for timeouts. It makes timeouts legible.

Adapter config includes timeouts.absoluteMs: a wall-clock deadline for the entire catalog operation, progress stream included. Separate connect and read timeouts still apply per HTTP hop, but the absolute deadline caps total user-visible wait regardless of how many stage transitions occur.

Cancellation flows through the standard AbortController / AbortSignal interface. The widget creates an AbortController per request, passes its signal to the chat API, and wires the typing indicator's cancel action to abort(). When the signal fires, the adapter stops reading the SSE stream and throws a transport error with outcome cancelled.

That last part matters. A stream can end five ways, and conflating them loses debuggability:

export type CatalogStreamOutcome =
  | 'cancelled'
  | 'closed_without_result'
  | 'error'
  | 'malformed'
  | 'timeout';

• cancelled: User or client aborted via AbortSignal.
• timeout: Absolute or read deadline exceeded.
• error: Network failure or non-2xx response mid-stream.
• malformed: SSE frame parsed but stage name not in the allowed list.
• closed_without_result: Stream ended cleanly but no search result arrived.

These outcomes are not user-facing copy. They are the classification layer for logs, metrics, and deciding whether to retry. A cancelled outcome after the user closes the widget should not increment the same error counter as a malformed frame from a misconfigured backend. Transport failures throw CatalogSearchTransportError with the outcome attached so callers cannot accidentally treat a dead stream as an empty result set.

HTTP semantics for long-lived responses are governed by RFC 9110. The practical implication for us: the client must handle connection drops, the server must not assume the client read every event, and both sides need a defined terminal state. Explicit outcomes are that terminal state for catalog progress.

What the Widget Does With It

The widget does not render a progress bar. It updates the typing indicator text:

onProgress: progress => {
  if (progress && typeof progress.message === 'string') {
    this.updateTypingIndicator(progress.message);
  }
}

The API maps each stage to a short sentence ("Searching the catalog…", "Ranking the best matches…") before the event reaches the widget. The widget only displays the string. It does not know about stage enums or elapsed milliseconds.

That is deliberate. A progress bar implies measurable completion. Catalog retrieval does not have a stable denominator. Is retrieving 40% of the work? It depends on the query. A bar that jumps from 30% to 90% in one frame is worse than a label that says what is happening now. Nielsen Norman Group's guidance on progress indicators distinguishes determinate bars (known duration) from indeterminate indicators (unknown duration). Catalog search is indeterminate. Stage names are the honest representation.

The widget also keeps a fallback timer. If no progress event arrives within 17 seconds, the typing indicator switches to "This search is taking a little longer. I'm still working on it…" That covers backends that support streaming but emit sparse updates, and clients where capability gating fell back to JSON mid-flight.

For customer-facing deployments, this sits alongside the automation patterns in customer service automation in 2026: automate the lookup, but keep the human-visible loop honest when the lookup takes time.

What We Gave Up

Progress streaming adds complexity across 20 files. The adapter now maintains two transport paths (SSE and JSON), a capability cache, SSE frame parsing, and outcome classification. Every new catalog backend must advertise progress capabilities in its health endpoint or streaming silently degrades. That is the intended behaviour, but it means backend teams have a contract to implement.

Fast stages look silly. When validating finishes in 12ms, the user may see "Checking product information…" flash for a single frame. We considered suppressing stages below a minimum display time and rejected it. Artificial delays lie about system speed. A flash is honest; a forced 500ms pause is theater.

Default is off. Tenants must enable progressStreaming: 'auto' in catalog skill config. We did not ship it as the default because not every catalog backend supports SSE progress yet, and we would rather have tenants opt in once their backend is ready than have streaming fail open on every turn.

Observability gets harder before it gets easier. Five outcome types means five buckets in dashboards instead of one "search failed" counter. The payoff is that on-call can distinguish user cancels from backend timeouts without reading stack traces.

What Ships Next

When PR #3 and PR #6 merge, catalog skills with progressStreaming: 'auto', a streaming-capable widget, and a backend that advertises SSE progress will show stage updates during retrieval. Everything else continues to work as a plain JSON search with a static typing indicator.

The design does not make catalog search faster. It makes the wait interpretable. For a chat interface backed by a live product catalog, that is the difference between "broken" and "working on it."