Jina Reader — Convert Any URL to LLM-Ready Text

What it is

Jina Reader is a free API service that converts any web URL into clean, LLM-friendly markdown. You prepend r.jina.ai/ to any URL and get back the page content as structured markdown, stripped of ads, navigation, and boilerplate. It handles JavaScript-rendered pages, PDFs, and even images (via OCR).

It serves developers building RAG pipelines, AI agents that need to read web content, or anyone who wants to feed web pages to an LLM without dealing with HTML parsing and content extraction.

How it saves time or tokens

Raw HTML is bloated with navigation, scripts, ads, and formatting tags that waste LLM tokens. Jina Reader extracts just the content and returns clean markdown, typically reducing input size by 70-90% compared to raw HTML. This means lower token costs and better LLM comprehension since the model processes only relevant text.

How to use

1. The simplest method -- prepend the URL:

curl https://r.jina.ai/https://example.com

1. Use in Python:

import requests

url = 'https://r.jina.ai/https://docs.python.org/3/tutorial/'
response = requests.get(url)
markdown_content = response.text
print(markdown_content[:500])

1. Use in an AI agent pipeline:

def read_url(url: str) -> str:
    resp = requests.get(f'https://r.jina.ai/{url}')
    return resp.text

# Feed the result to your LLM as context
context = read_url('https://docs.anthropic.com/en/docs/welcome')

Example

The API handles the rendering and extraction automatically. No configuration needed for most use cases.

Related on TokRepo

• AI tools for web-scraping -- web scraping tools for AI
• AI tools for RAG -- RAG pipeline components and tools

Common pitfalls

• Some websites block automated requests. Jina Reader handles most sites but very aggressive anti-bot protection may prevent content extraction.
• Very long pages may be truncated. For large documents, check the response length and consider splitting across multiple requests if needed.
• The free tier has rate limits. For high-volume production use, check Jina's pricing for higher rate limits.