Spring AI — AI Engineering for Java/Spring

What it is

Spring AI is a framework that brings AI engineering capabilities to Java and Spring Boot applications. It provides Spring-friendly APIs for chat completions, embeddings, retrieval-augmented generation (RAG), vector database integration, function calling, and structured output parsing. It supports major AI providers including OpenAI, Anthropic Claude, Google Gemini, Mistral, Ollama, and Azure OpenAI.

Spring AI is for Java developers and enterprise teams who want to add AI capabilities to existing Spring Boot applications. If your backend is Java and you need to integrate LLMs, build RAG pipelines, or add AI-powered features, Spring AI provides familiar Spring patterns (auto-configuration, dependency injection, properties) for AI integration.

How it saves time or tokens

Spring AI abstracts provider-specific APIs behind a common interface. Switch from OpenAI to Anthropic by changing a configuration property, not your code. The auto-configuration handles client setup, retry logic, and rate limiting. RAG support with built-in vector store integrations (PGVector, Chroma, Pinecone, Milvus) means you wire up retrieval without writing embedding pipeline code from scratch. Spring Boot starters reduce AI integration to adding a dependency and setting an API key.

How to use

1. Add the Spring AI starter to your pom.xml or build.gradle.
2. Configure your AI provider API key in application.properties.
3. Inject the ChatClient or EmbeddingClient beans and use them in your services.

Example

import org.springframework.ai.chat.client.ChatClient;
import org.springframework.web.bind.annotation.*;

@RestController
public class AiController {

    private final ChatClient chatClient;

    public AiController(ChatClient.Builder builder) {
        this.chatClient = builder.build();
    }

    @GetMapping("/ask")
    public String ask(@RequestParam String question) {
        return chatClient.prompt()
            .user(question)
            .call()
            .content();
    }

    @GetMapping("/structured")
    public Movie recommend(@RequestParam String genre) {
        return chatClient.prompt()
            .user("Recommend a " + genre + " movie")
            .call()
            .entity(Movie.class);
    }
}

record Movie(String title, int year, String director) {}

Related on TokRepo

• AI tools for coding -- developer tools for AI integration
• AI tools for API -- API integration frameworks

Common pitfalls

• Hardcoding the AI provider instead of using the abstraction layer. Spring AI's value is provider portability. Program against the ChatClient interface so you can switch providers by changing configuration.
• Not configuring retry and rate limiting. AI provider APIs have rate limits. Use Spring AI's built-in retry configuration to handle transient failures gracefully.
• Ignoring structured output parsing. Instead of parsing LLM responses manually, use .entity(MyClass.class) to get typed Java objects. This reduces parsing bugs and makes AI responses type-safe.