Wiring the LLM in with Spring AI

Your ShopKart backend serves products and takes orders. Your React storefront looks the part. But the chat widget is still faking its replies. Time to make it real: in this lesson you drop in Spring AI 1.0, create a ChatClient, expose POST /api/chat, and have an actual model answer the customer.

It is still a chatbot at this point - it can talk about shopping but cannot search the catalog or place an order. That upgrade to a true agent comes next lesson. Getting a clean conversation working first keeps the moving parts small.

Add Spring AI to the project

Spring AI is provider-agnostic - the same ChatClient code works whether the model behind it is OpenAI or Anthropic. We default to OpenAI. Add the BOM for version management and the OpenAI model starter.

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.ai</groupId>
      <artifactId>spring-ai-bom</artifactId>
      <version>1.0.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<dependencies>
  <dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-openai</artifactId>
  </dependency>
</dependencies>

Then configure the key and model in application.properties. Read the API key from an environment variable - never commit it.

spring.ai.openai.api-key=${OPENAI_API_KEY}
spring.ai.openai.chat.options.model=gpt-4o-mini
spring.ai.openai.chat.options.temperature=0.3

To use Anthropic Claude instead, you would swap the starter to spring-ai-starter-model-anthropic, set spring.ai.anthropic.api-key, and choose a Claude model - the ChatClient code below does not change at all. That provider-independence is the whole point of Spring AI.

Create the ChatClient bean

Spring AI auto-configures a ChatModel from your properties. You wrap it in a ChatClient, which is the fluent, developer-facing API. Configure a system prompt once on the builder so every conversation starts with ShopKart's personality and rules.

package com.shopkart.config;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.model.ChatModel;
import org.springframework.context.annotation.*;

@Configuration
public class ChatConfig {

    private static final String SYSTEM_PROMPT = """
        You are the ShopKart shopping assistant for an Indian e-commerce store.
        - Be concise, friendly, and helpful.
        - All prices are in Indian Rupees (INR). Always show the rupee symbol.
        - Only discuss shopping, products, and orders. Politely decline off-topic requests.
        - If you do not know something about the catalog yet, say so honestly.
        """;

    @Bean
    public ChatClient chatClient(ChatModel chatModel) {
        return ChatClient.builder(chatModel)
                .defaultSystem(SYSTEM_PROMPT)
                .build();
    }
}

The ChatClient.builder(chatModel) fluent API mirrors Spring's RestClient and WebClient, so it feels familiar. defaultSystem(...) sets a system message applied to every prompt made through this client - the ideal place for tone, guardrails, and domain rules.

The chat endpoint

Now expose POST /api/chat. The React widget will send the user's message; the endpoint calls the model and returns its reply. Start with the simplest possible version.

package com.shopkart.controller;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.*;

record ChatRequest(String message) {}
record ChatReply(String reply) {}

@RestController
@RequestMapping("/api/chat")
public class ChatController {

    private final ChatClient chatClient;

    public ChatController(ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    @PostMapping
    public ChatReply chat(@RequestBody ChatRequest request) {
        String answer = chatClient.prompt()
                .user(request.message())
                .call()
                .content();
        return new ChatReply(answer);
    }
}

The core of Spring AI in three fluent calls: .prompt() starts a request, .user(...) adds the user's message, .call() sends it synchronously, and .content() extracts the text. The system prompt from your bean is applied automatically.

Keeping conversation memory

The version above forgets everything after each message - every request is a fresh conversation. For a shopping assistant that is a problem: the customer says "show me earbuds", then "which of those has the best battery?" and the model has no idea what "those" means.

The cleanest fix is to let the frontend own the history and send the whole conversation each turn. Accept the full message list and replay it into the prompt.

package com.shopkart.controller;

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.messages.*;
import org.springframework.web.bind.annotation.*;

import java.util.*;

record ChatMessage(String role, String content) {}
record ConversationRequest(List<ChatMessage> messages) {}
record ChatReply(String reply) {}

@RestController
@RequestMapping("/api/chat")
public class ChatController {

    private final ChatClient chatClient;

    public ChatController(ChatClient chatClient) {
        this.chatClient = chatClient;
    }

    @PostMapping
    public ChatReply chat(@RequestBody ConversationRequest request) {
        List<Message> history = new ArrayList<>();
        for (ChatMessage m : request.messages()) {
            if ("user".equals(m.role())) {
                history.add(new UserMessage(m.content()));
            } else if ("assistant".equals(m.role())) {
                history.add(new AssistantMessage(m.content()));
            }
        }

        String answer = chatClient.prompt()
                .messages(history)
                .call()
                .content();

        return new ChatReply(answer);
    }
}

The conversation round-trip

Here is what one message now triggers across the stack.

Connect the React widget

Back in ChatWidget.jsx, replace the mockReply stub with a real call. Send the full message array (minus the seed greeting is optional; sending it is harmless) and append the reply.

async function sendMessage(e) {
  e.preventDefault();
  const text = input.trim();
  if (!text || sending) return;

  const nextMessages = [...messages, { role: 'user', content: text }];
  setMessages(nextMessages);
  setInput('');
  setSending(true);

  try {
    const res = await fetch(`${API}/chat`, {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ messages: nextMessages }),
    });
    if (!res.ok) throw new Error(`HTTP ${res.status}`);
    const data = await res.json();
    setMessages((prev) => [...prev, { role: 'assistant', content: data.reply }]);
  } catch (err) {
    setMessages((prev) => [
      ...prev,
      { role: 'assistant', content: 'Sorry, I could not reach the assistant. Try again.' },
    ]);
  } finally {
    setSending(false);
  }
}

Because the widget already stored messages as { role, content }, the request body is just { messages: nextMessages }. No translation, no mapping - the design decision from lesson 3 pays off exactly here. Start both servers, type "What kinds of products do you sell?", and you get a real answer from the model.

Key takeaways

• Add spring-ai-bom:1.0.0 and spring-ai-starter-model-openai; configure the key and model in application.properties from an environment variable.
• The ChatClient fluent API - .prompt().user(...).call().content() - is the heart of Spring AI, and it is provider-agnostic (swap the starter for Anthropic Claude with no code change).
• Set tone and guardrails once with defaultSystem(...) on the ChatClient bean.
• Send the full { role, content } message array from the frontend so the model has conversation memory for follow-up questions.
• This is still a chatbot - it talks but cannot act. Tools change that next.

Next up: AI Agents with Tool Calling - give the model @Tool functions bound to your real repositories and watch it reason, act, and observe.