Jsonformer — Guaranteed Structured JSON from Language Models

Introduction

Jsonformer takes a different approach to structured output from LLMs. Instead of hoping the model produces valid JSON and then parsing it, Jsonformer controls the generation process: it fills in the fixed JSON structure (braces, colons, commas) itself and only asks the LLM to generate the actual content values. This guarantees syntactically valid JSON every time.

What Jsonformer Does

• Generates JSON output that conforms to a provided JSON schema
• Handles string, number, integer, boolean, array, and nested object types
• Controls the LLM generation process to ensure structural validity
• Works with any Hugging Face causal language model
• Eliminates the need for retry loops and JSON repair logic

Architecture Overview

Jsonformer walks the JSON schema tree recursively. At each node, it determines the expected type. For structural tokens (braces, brackets, colons, commas, keys), Jsonformer writes them directly to the output buffer without invoking the LLM. For value nodes (strings, numbers, booleans), it prompts the LLM to generate the appropriate content, constraining the token space to valid options. String values use a stop-token strategy to terminate at the closing quote. Numeric values are parsed from the model output and validated against the schema. This approach avoids the common failure mode of unclosed brackets or mismatched types.

Self-Hosting & Configuration

• Install with pip install jsonformer alongside transformers and torch
• Pass any Hugging Face causal LM and tokenizer to the Jsonformer constructor
• Define the target structure as a standard JSON schema dictionary
• Adjust max_string_token_length to control maximum string field length
• Use temperature and other generation parameters for value diversity

Key Features

• Schema-constrained generation produces valid JSON on every call
• LLM only generates value content, reducing token waste on structural syntax
• Supports nested objects and arrays for complex data structures
• Works with local models (no API dependency)
• Lightweight wrapper with minimal dependencies beyond Hugging Face Transformers

Comparison with Similar Tools

• Outlines — grammar-based constrained generation supporting regex and JSON schema; more general but heavier setup
• Guidance (Microsoft) — template-based LLM control with interleaved generation and logic; broader scope, more complex API
• Instructor — structured output via function calling for API-based LLMs (OpenAI, Anthropic); Jsonformer targets local models
• LMQL — query language for LLMs with type constraints; more expressive but requires learning a new syntax
• llama.cpp grammars — GBNF grammar support built into llama.cpp; lower-level and specific to GGUF models

FAQ

Q: Does Jsonformer work with API-based models like GPT-4? A: No. Jsonformer requires direct access to the model's token generation process, so it only works with locally loaded Hugging Face models.

Q: Can Jsonformer handle deeply nested schemas? A: Yes. Jsonformer recursively walks the schema tree and handles nested objects and arrays to arbitrary depth.

Q: How does Jsonformer handle the LLM generating invalid content for a number field? A: Jsonformer constrains the token vocabulary during number generation and parses the result. If parsing fails, it falls back to a default value.

Q: Is Jsonformer actively maintained? A: Jsonformer has a focused scope and stable API. While commits are infrequent, the library works reliably with current Hugging Face Transformers versions.

Sources

• https://github.com/1rgs/jsonformer
• https://pypi.org/project/jsonformer/