Streaming Responses to React

Ask the ShopKart agent a question today and the customer stares at a "Thinking..." bubble for several seconds, then the whole answer appears at once. Functional, but it feels slow - especially when the agent ran a couple of tools first. The fix is streaming: send the reply token by token so the customer watches it type out in real time, exactly like ChatGPT or Claude.

Under the hood this is Server-Sent Events (SSE): a one-way stream from server to browser over a single long-lived HTTP connection. Spring AI exposes streaming as a reactive Flux, and the browser consumes it. Let us wire it up.

Blocking vs streaming

The .call() you used so far blocks until the model finishes, then hands back the full string. The streaming counterpart, .stream(), hands back a Flux that emits chunks as the model produces them. Spring maps that Flux onto an SSE response, and the browser reads each chunk the instant it arrives.

The streaming endpoint

To return a reactive stream, the controller method returns Flux and declares it produces text/event-stream. Add a streaming variant of the chat endpoint. Spring AI's streaming works whether or not tools are involved - the same ChatClient with your ShopKartTools still runs the agent loop, then streams the final answer.

package com.shopkart.controller;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.*;
import reactor.core.publisher.Flux;

@RestController
@RequestMapping("/api/chat")
public class ChatStreamController {

    private final ChatClient chatClient;

    public ChatStreamController(ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    @GetMapping(value = "/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    public Flux<String> stream(@RequestParam String message) {
        return chatClient.prompt()
                .user(message)
                .stream()
                .content();   // Flux<String> - one emission per token chunk
    }
}

The only differences from the blocking endpoint: .stream() instead of .call(), a Flux return type, and the text/event-stream content type. Spring handles the SSE framing - each emitted chunk is written to the wire as an SSE data: event automatically.

We use @GetMapping here because the browser's built-in EventSource only issues GET requests. For simple single-turn messages that is fine. (If you need to POST a full conversation and still stream, you use the fetch streaming API with a ReadableStream reader instead of EventSource - shown at the end.)

A note on WebFlux

Returning a Flux from a controller requires the reactive stack on the classpath. If your app is Spring MVC (the spring-boot-starter-web we used), Spring can still stream a Flux for SSE endpoints, but adding spring-boot-starter-webflux gives you the full reactive toolkit and is the common choice for streaming AI endpoints. Spring AI's .stream() returns a Project Reactor Flux, so the pieces fit together naturally.

Consuming the stream in React

The browser's EventSource API opens an SSE connection and fires an event per chunk. In the ChatWidget, add a streaming send that creates an empty assistant bubble and grows it as tokens arrive.

function sendMessageStreaming(text) {
  // Show the user's message and an empty assistant bubble to fill in
  setMessages((prev) => [
    ...prev,
    { role: 'user', content: text },
    { role: 'assistant', content: '' },
  ]);

  const url = `${API}/chat/stream?message=${encodeURIComponent(text)}`;
  const source = new EventSource(url);

  source.onmessage = (event) => {
    // Append this chunk to the last (assistant) message
    setMessages((prev) => {
      const next = [...prev];
      const last = next[next.length - 1];
      next[next.length - 1] = { ...last, content: last.content + event.data };
      return next;
    });
  };

  source.onerror = () => {
    // SSE closes with an error event when the stream ends; tidy up.
    source.close();
    setSending(false);
  };
}

Each onmessage appends event.data to the last message, so the assistant bubble visibly types itself out. That immutable update - spread the array, spread the last message, replace it - is important: mutating state in place would not trigger a re-render.

Wiring it into the form

Swap the widget's submit handler to call the streaming version:

async function handleSubmit(e) {
  e.preventDefault();
  const text = input.trim();
  if (!text || sending) return;
  setInput('');
  setSending(true);
  sendMessageStreaming(text);
}

Start both servers, ask "recommend running shoes under 3500 rupees", and the reply now types out live. If the agent ran a searchProducts tool first, there is a brief pause while the loop runs, then the final answer streams - the customer sees motion instead of a frozen spinner.

Closing the stream cleanly

When POST and streaming both matter

async function streamViaFetch(messages) {
  const res = await fetch(`${API}/chat/stream`, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ messages }),
  });
  const reader = res.body.getReader();
  const decoder = new TextDecoder();

  while (true) {
    const { value, done } = await reader.read();
    if (done) break;
    const chunk = decoder.decode(value, { stream: true });
    // append chunk to the assistant message, same as onmessage above
    appendToAssistant(chunk);
  }
}

This pairs with a @PostMapping streaming endpoint that accepts the conversation. Same streaming feel, but you control the request method and body. Use EventSource for simple cases and fetch streaming when you need POST semantics.

Key takeaways

• Streaming uses Server-Sent Events: a single long-lived connection over which the server pushes tokens as they are generated.
• Swap .call() for .stream().content() to get a Flux, and return it from a controller method that produces text/event-stream.
• In React, EventSource fires onmessage per chunk; append each event.data to the last assistant message with an immutable update to make it type out live.
• EventSource is GET-only; use the fetch streaming API with a ReadableStream reader when you need to POST a full conversation.
• Close the stream on completion or error, and disable proxy buffering in production so streaming actually reaches the browser.

Next up: Capstone: The AI Shopping Assistant - assemble every piece into the finished ShopKart assistant.