LangChain4j — LLM Integration for Java

What it is

LangChain4j is an open-source Java framework that brings LLM integration patterns to the Java ecosystem. It provides a unified API across 20+ LLM providers (OpenAI, Anthropic, Google, Ollama, and others) and 30+ vector store backends. The framework supports RAG pipelines, tool use, MCP protocol, structured output, and Spring Boot auto-configuration.

The project targets Java and Kotlin developers building AI-powered enterprise applications who need the same capabilities that Python developers get from LangChain, but in a Java-native form factor.

How it saves time or tokens

LangChain4j eliminates the need to write provider-specific HTTP clients for each LLM API. Switching from OpenAI to Anthropic requires changing one configuration line, not rewriting API integration code. The declarative AI Services interface lets developers define AI capabilities as Java interfaces with annotations, reducing boilerplate to near-zero. RAG pipelines are assembled from composable components rather than built from scratch.

How to use

1. Add the Maven dependency:

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j</artifactId>
    <version>1.0.0</version>
</dependency>

1. Create a chat model and send a message:

ChatLanguageModel model = OpenAiChatModel.withApiKey("sk-...");
String answer = model.chat("Hello!");
System.out.println(answer);

1. Define an AI Service interface for structured interaction:

interface Assistant {
    @SystemMessage("You are a helpful coding assistant.")
    String chat(String userMessage);
}

Assistant assistant = AiServices.create(Assistant.class, model);
String response = assistant.chat("Explain Java records.");

Example

// RAG pipeline with in-memory embedding store
import dev.langchain4j.data.document.Document;
import dev.langchain4j.store.embedding.inmemory.InMemoryEmbeddingStore;

EmbeddingModel embeddingModel = OpenAiEmbeddingModel.withApiKey("sk-...");
InMemoryEmbeddingStore<TextSegment> store = new InMemoryEmbeddingStore<>();

// Ingest documents
Document doc = Document.from("LangChain4j supports 20+ LLM providers.");
EmbeddingStoreIngestor.ingest(doc, embeddingModel, store);

// Query with RAG
ContentRetriever retriever = EmbeddingStoreContentRetriever.from(store);
Assistant ragAssistant = AiServices.builder(Assistant.class)
    .chatLanguageModel(model)
    .contentRetriever(retriever)
    .build();
String answer = ragAssistant.chat("How many providers?");

Related on TokRepo

• RAG tools directory -- Retrieval-augmented generation frameworks and tools
• AI tools for coding -- Developer tools for AI application development

Common pitfalls

• Version 1.0.0 introduced breaking API changes from the 0.x series; migration guides are available in the documentation
• The AI Services interface uses runtime proxy generation; ensure your build tool does not strip reflection metadata
• Spring Boot auto-configuration requires the separate langchain4j-spring-boot-starter dependency, not just the core library