Tool-Use Agent Bootcamp

What's in this pack

This is the bootcamp a working dev would walk if they'd never wired a function call before and wanted to land on a production agent that picks the right tool, returns valid JSON, and is covered by evals — not a wishlist of 30 frameworks. Every pick here has a healthy GitHub repo, real docs, and earns its place in the chain. The order matters: each tool teaches the next.

You can do all ten in a weekend if you already know Python or TypeScript. By Sunday night you'll have a small agent that takes a natural-language request, picks one of several real tools (search, email, GitHub), returns a typed result, and ships with a regression eval that runs in CI.

Install in this order

1. Fireworks JSON Mode + Function Calling on Open Models — start here. Cheapest way to see a real function-call round-trip work on an OSS model without spending a dollar on OpenAI yet. You feed a schema, the model returns valid JSON, you parse it. Internalize this loop before adding any framework.
2. Groq Tool Use — Llama 3.3 at 280 tok/s — same idea, but at speeds that make iteration painless. Run the same prompts you wrote against Fireworks; watch how tool selection changes with a smarter model. This is also your fallback provider once you go to prod.
3. Structured Outputs — Force LLMs to Return Valid JSON — the conceptual primer. JSON-mode and function-calling are both special cases of constrained generation. Read this before reaching for a library, or you'll cargo-cult.
4. Instructor — Typed Structured Outputs for LLMs — the Python ergonomics layer. You define a Pydantic model, Instructor handles the schema, retries, and validation. Drop-in replacement for raw response_format calls. After this you should never write a JSON-schema dict by hand again.
5. Outlines — Structured Outputs with Any Model — Instructor's cousin for the OSS world. Where Instructor wraps providers' built-in JSON modes, Outlines does constrained decoding locally (logits masking). Pick whichever fits your stack; both are worth knowing.
6. Composio — 250+ Tool Integrations for AI Agents — once you trust structured outputs, you need tools to call. Composio ships 250+ pre-built integrations (Gmail, GitHub, Slack, Notion, Linear, Stripe) with auth handled. Skip writing your own send_email wrapper for the third time.
7. PydanticAI — Type-Safe AI Agent Framework — first real agent loop. Lightweight, type-safe, Python-native. Take the Pydantic models from step 4, the Composio tools from step 6, and PydanticAI orchestrates the call/retry/handoff. Small surface area; few footguns.
8. OpenAI Agents SDK — Multi-Agent Systems in Python — the OpenAI-blessed alternative. Better if you're staying on OpenAI/Azure and want handoffs, guardrails, and built-in tracing. Less type-strict than PydanticAI but more batteries-included.
9. LangGraph — Stateful AI Agents as Graphs — graduate to this when a single agent loop isn't enough. Stateful, branching workflows; explicit state machine; checkpoints. Heavier dependency, steeper learning curve, but the right answer for multi-step research, approval flows, and human-in-the-loop.
10. Promptfoo — Test & Red-Team LLM Apps — the closing eval. Every tool-using agent regresses silently when a provider quietly updates a model. Promptfoo runs your tool-use test suite in CI, asserts on JSON schemas, and red-teams for prompt injection. Don't ship without it.

How they fit together

[Fireworks JSON Mode] ──┐
                         ├──► raw constrained-generation primer
[Groq Tool Use] ─────────┘
         │
         ▼
[Structured Outputs guide] ──► conceptual frame
         │
         ▼
[Instructor] ◄──► [Outlines]   ── pick the lib that matches your stack
         │
         ▼
[Composio] ──► pre-built tool catalog (Gmail, GitHub, Slack…)
         │
         ▼
[PydanticAI] ──or── [OpenAI Agents SDK]   ── first real agent loop
         │
         ▼
[LangGraph]   ── graduate to graph state when you outgrow a single loop
         │
         ▼
[Promptfoo]   ── CI evals for tool selection + JSON validity

The four-link spine Structured Outputs → Instructor/Outlines → Composio → PydanticAI is the dividing line. Below it, you're "poking at JSON mode". Above it, you're building an agent. Don't skip Promptfoo at the top — every production agent silently breaks the day a model is updated, and only an eval suite catches it.

Tradeoffs you'll hit

• Instructor vs Outlines — Instructor leans on the provider's native JSON/tool mode (OpenAI, Anthropic, Gemini), which is fast and high-quality. Outlines does its own constrained decoding, which works on any local model but is slower. Use Instructor for OpenAI/Anthropic; Outlines for vLLM/Ollama.
• PydanticAI vs OpenAI Agents SDK — PydanticAI is provider-agnostic, type-strict, lightweight. The OpenAI SDK has handoffs, guardrails, and tracing built-in but is best when you stay inside the OpenAI ecosystem. New devs starting today: try PydanticAI first.
• Composio vs hand-rolled tools — Composio costs a SaaS dependency and a tiny bit of latency. In return it kills the entire "write the OAuth flow for Gmail again" tax. Hand-roll only for tools that don't exist in Composio or for cost-sensitive high-volume calls.
• LangGraph too early — beginners often jump straight to LangGraph because it looks impressive. Don't. Single-loop agents (PydanticAI / OpenAI Agents SDK) cover 80% of cases. Only reach for LangGraph when you have explicit human-in-the-loop, branching, or checkpointing needs.

Common pitfalls

• No schema, no agent. If your tool's input isn't typed (Pydantic / Zod / JSON Schema), the model will hallucinate fields. Always define inputs before wiring tools.
• Hidden tool count. Adding a 30th tool silently degrades selection. Most production agents top out around 8-12 tools per agent; above that, route to specialist sub-agents.
• Forgetting retry-on-validation. Models occasionally emit JSON that parses but fails your business validation. Instructor handles this automatically; raw response_format does not. Don't ship without a retry layer.
• Eval-by-vibes. "It worked when I tried it" is not a CI gate. Set up Promptfoo from day one — even 10 cases beats nothing — and add a case every time you find a real-world failure.
• Provider lock-in via tool schema. OpenAI's tool format and Anthropic's tools block are subtly different. Use Instructor / PydanticAI / OpenAI Agents SDK to abstract; never inline the raw provider JSON in your app code.