Talking to LLMs with ChatClient

A ShopKart shopper opens the chat widget and types: "Suggest a good gift under 2000 rupees for a friend who loves cricket." Somewhere behind that box, an HTTP request needs to reach a language model, carry the shopper's message, get a reply, and return it. In Spring AI, the object that orchestrates that round trip is the ChatClient. In this lesson you will build a real endpoint that does exactly this.

The ChatClient.Builder

When you added the spring-ai-starter-model-openai starter, Spring Boot auto-configured a ChatClient.Builder bean. You inject that builder and call .build() to get a ChatClient. The idiomatic pattern is to build it once in your bean's constructor.

package com.shopkart.assistant;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.stereotype.Service;

@Service
public class AssistantService {

    private final ChatClient chatClient;

    // Spring injects the auto-configured builder
    public AssistantService(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    public String ask(String question) {
        return chatClient.prompt()
                .user(question)
                .call()
                .content();
    }
}

Read the fluent chain out loud: build a prompt(), set the user() message, call() the model synchronously, take the content(). That last method returns a plain String — the assistant's reply. That is the entire happy path.

Why inject the builder instead of the finished client? Because the builder lets you attach defaults (a system prompt, model options, advisors) that apply to every call from that client. We will use that heavily starting in the next lesson. For now, a bare builder.build() is enough.

Anatomy of the fluent call

Each link in the chain has a job. Here is the request flow from your controller to the model and back.

A working /assistant/ask endpoint

Let us expose the service over HTTP. This is an ordinary Spring MVC controller — nothing about AI changes how you write controllers.

package com.shopkart.assistant;

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class AssistantController {

    private final AssistantService assistantService;

    public AssistantController(AssistantService assistantService) {
        this.assistantService = assistantService;
    }

    @GetMapping("/assistant/ask")
    public AssistantReply ask(@RequestParam String question) {
        String answer = assistantService.ask(question);
        return new AssistantReply(question, answer);
    }

    public record AssistantReply(String question, String answer) {}
}

Run the app and hit it:

curl "http://localhost:8080/assistant/ask?question=Suggest%20a%20cricket%20gift%20under%20Rs%202000"

{
  "question": "Suggest a cricket gift under Rs 2000",
  "answer": "For a cricket-loving friend under Rs 2000, consider a good-quality tennis-ball cricket bat, a pair of batting gloves, or a branded cricket cap..."
}

That is a real, working AI endpoint in about 40 lines of code. Notice the model answered from its general knowledge — it does not yet know ShopKart's actual catalog. Grounding it in real inventory is the job of lessons 5 and 6.

Accepting a POST body

Query params are fine for a demo, but real chat messages can be long and contain special characters. Use a POST with a JSON body in practice.

@PostMapping("/assistant/chat")
public AssistantReply chat(@RequestBody ChatRequest request) {
    String answer = assistantService.ask(request.message());
    return new AssistantReply(request.message(), answer);
}

public record ChatRequest(String message) {}

curl -X POST http://localhost:8080/assistant/chat \
  -H "Content-Type: application/json" \
  -d '{"message":"Do you sell wireless earbuds?"}'

Reading response metadata

Sometimes you need more than the text — how many tokens were used (for cost tracking), which model answered, why it stopped. Swap content() for chatResponse().

import org.springframework.ai.chat.model.ChatResponse;

public AssistantReply askWithUsage(String question) {
    ChatResponse response = chatClient.prompt()
            .user(question)
            .call()
            .chatResponse();

    String answer = response.getResult().getOutput().getText();

    var usage = response.getMetadata().getUsage();
    long promptTokens = usage.getPromptTokens();
    long generationTokens = usage.getCompletionTokens();
    long totalTokens = usage.getTotalTokens();

    // log these; token counts drive your cost dashboards
    return new AssistantReply(question, answer);
}

We come back to token usage and cost control in the final lesson. For now, know that the information is one method call away.

Streaming: don't make the shopper wait

Language models generate text token by token. If a reply is three sentences long, waiting for the whole thing before showing anything feels slow. Streaming pushes each chunk to the browser as it is produced — the same "typing" effect you see in consumer chat apps.

Replace call() with stream(), and content() now returns a reactive Flux.

import reactor.core.publisher.Flux;

public Flux<String> askStreaming(String question) {
    return chatClient.prompt()
            .user(question)
            .stream()
            .content();     // Flux<String> — emits partial chunks
}

Expose it as a Server-Sent Events endpoint so the browser receives chunks live:

import org.springframework.http.MediaType;

@GetMapping(value = "/assistant/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
public Flux<String> stream(@RequestParam String question) {
    return assistantService.askStreaming(question);
}

curl -N "http://localhost:8080/assistant/stream?question=Recommend%20a%20good%20office%20chair"

With -N (no buffering), curl prints fragments as they arrive: "For", " a", " comfortable", " office", " chair"... Streaming does not change what the model says, only how the words reach the user. Use call() when you need the full string (for example to parse it, which we do next lesson); use stream() for interactive chat UIs.

Key takeaways

• Inject the auto-configured ChatClient.Builder, call .build() once, and reuse the ChatClient.
• The core chain is prompt().user(text).call().content(), returning a String.
• ChatClient lives behind ordinary Spring MVC controllers — no special plumbing.
• Use call().chatResponse() to read token usage, model id, and finish reason.
• Use stream().content() to get a Flux for a live typing effect in chat UIs.
• The model so far answers from general knowledge only; grounding it in the ShopKart catalog comes later.

Next up: Prompt Templates & System Prompts — giving the assistant a ShopKart persona and guardrails.