# Jsonformer — Guaranteed Structured JSON from Language Models

> Jsonformer is a Python library that generates valid JSON from local Hugging Face language models by constraining token generation to follow a provided JSON schema, eliminating malformed output.

## Install

Save as a script file and run:

# Jsonformer — Guaranteed Structured JSON from Language Models

## Quick Use
```bash
pip install jsonformer
python -c "
from jsonformer import Jsonformer
from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained('databricks/dolly-v2-3b')
tokenizer = AutoTokenizer.from_pretrained('databricks/dolly-v2-3b')
schema = {'type': 'object', 'properties': {'name': {'type': 'string'}, 'age': {'type': 'number'}}}
jf = Jsonformer(model, tokenizer, schema, 'Generate a person')
result = jf()
print(result)
"
```

## Introduction
Jsonformer takes a different approach to structured output from LLMs. Instead of hoping the model produces valid JSON and then parsing it, Jsonformer controls the generation process: it fills in the fixed JSON structure (braces, colons, commas) itself and only asks the LLM to generate the actual content values. This guarantees syntactically valid JSON every time.

## What Jsonformer Does
- Generates JSON output that conforms to a provided JSON schema
- Handles string, number, integer, boolean, array, and nested object types
- Controls the LLM generation process to ensure structural validity
- Works with any Hugging Face causal language model
- Eliminates the need for retry loops and JSON repair logic

## Architecture Overview
Jsonformer walks the JSON schema tree recursively. At each node, it determines the expected type. For structural tokens (braces, brackets, colons, commas, keys), Jsonformer writes them directly to the output buffer without invoking the LLM. For value nodes (strings, numbers, booleans), it prompts the LLM to generate the appropriate content, constraining the token space to valid options. String values use a stop-token strategy to terminate at the closing quote. Numeric values are parsed from the model output and validated against the schema. This approach avoids the common failure mode of unclosed brackets or mismatched types.

## Self-Hosting & Configuration
- Install with `pip install jsonformer` alongside `transformers` and `torch`
- Pass any Hugging Face causal LM and tokenizer to the Jsonformer constructor
- Define the target structure as a standard JSON schema dictionary
- Adjust `max_string_token_length` to control maximum string field length
- Use `temperature` and other generation parameters for value diversity

## Key Features
- Schema-constrained generation produces valid JSON on every call
- LLM only generates value content, reducing token waste on structural syntax
- Supports nested objects and arrays for complex data structures
- Works with local models (no API dependency)
- Lightweight wrapper with minimal dependencies beyond Hugging Face Transformers

## Comparison with Similar Tools
- **Outlines** — grammar-based constrained generation supporting regex and JSON schema; more general but heavier setup
- **Guidance (Microsoft)** — template-based LLM control with interleaved generation and logic; broader scope, more complex API
- **Instructor** — structured output via function calling for API-based LLMs (OpenAI, Anthropic); Jsonformer targets local models
- **LMQL** — query language for LLMs with type constraints; more expressive but requires learning a new syntax
- **llama.cpp grammars** — GBNF grammar support built into llama.cpp; lower-level and specific to GGUF models

## FAQ
**Q: Does Jsonformer work with API-based models like GPT-4?**
A: No. Jsonformer requires direct access to the model's token generation process, so it only works with locally loaded Hugging Face models.

**Q: Can Jsonformer handle deeply nested schemas?**
A: Yes. Jsonformer recursively walks the schema tree and handles nested objects and arrays to arbitrary depth.

**Q: How does Jsonformer handle the LLM generating invalid content for a number field?**
A: Jsonformer constrains the token vocabulary during number generation and parses the result. If parsing fails, it falls back to a default value.

**Q: Is Jsonformer actively maintained?**
A: Jsonformer has a focused scope and stable API. While commits are infrequent, the library works reliably with current Hugging Face Transformers versions.

## Sources
- https://github.com/1rgs/jsonformer
- https://pypi.org/project/jsonformer/

---
Source: https://tokrepo.com/en/workflows/asset-7e16d729
Author: Script Depot