IA Local-First — Tus Datos Nunca Salen del Portátil

What's in this pack

This is the rig you build when you've decided your journal, your client recordings, and your half-written code are not going into someone else's training set. Every tool here is open-source, actively maintained, and runs with no outbound network call required once the models are downloaded.

The motivation is rarely just privacy in the abstract. It's three concrete things stacked: (1) the monthly token bill that scales with how curious you are, (2) terms of service that change, and (3) the dawning realization that you've been pasting your entire inbox into a chat window owned by a company that openly indexes it. A local stack fixes all three permanently.

This pack is not the same as our self-hosted-ai pack — that one is for shipping a SaaS on your own metal (Tabby, Onyx, LibreChat, n8n). This one is for individuals who want a private AI on a personal machine, including non-developer tools like meeting transcription and a notes app.

Install in this order

1. Ollama — model runner. Start here. Single command (curl -fsSL ollama.com/install.sh | sh), pulls models with ollama pull llama3.1, exposes an OpenAI-compatible API on localhost:11434. Everything downstream points at this.
2. GPT4All — alternative model runner with a GUI. If you don't live in a terminal, install this instead of (or alongside) Ollama. Same job, friendlier surface for non-devs.
3. Open WebUI — the local ChatGPT replacement. Talks to Ollama out of the box, supports multi-turn chat, RAG over uploaded files, web search plugins. This is where 80% of "I just want to ask the AI something" happens.
4. Continue — local coding assistant for VS Code and JetBrains. Configure it to call your local Ollama model instead of Copilot's servers. Inline edits, chat, refactor — all on-device. Slower than Copilot, but your private repo never leaves the machine.
5. Khoj — AI second brain. Indexes your Markdown notes, PDFs, org-mode, even Notion exports, then lets you chat with them via local LLM. This is the RAG layer for your life, not your codebase.
6. Faster Whisper — speech-to-text. 4x faster than vanilla Whisper, runs on CPU or GPU, OpenAI Whisper accuracy. Drop audio in, get a transcript out. Foundation for the next tool.
7. Meetily — privacy-first meeting assistant. Records, transcribes via Whisper locally, summarizes via your local LLM. Zoom/Meet recordings never touch a cloud.
8. ComfyUI — local image generation via Stable Diffusion. Node-based, fast on Apple Silicon and CUDA, runs SDXL / Flux / SD3 models pulled from Hugging Face. No prompt logging, no content policy, no usage cap.
9. Joplin — privacy-focused note app with optional end-to-end encryption. Where you keep the source material your local AI reads. Markdown, plugins, syncs between devices via your own storage.

How they fit together

        ┌─────────────────────────────────────┐
        │   Your laptop (no outbound calls)   │
        └─────────────────────────────────────┘
                       │
  ┌────────────────────┴────────────────────┐
  │                                          │
Ollama / GPT4All  ◄──── OpenAI-compatible API ────┐
  (model runner)                                  │
  │                                                │
  ├─► Open WebUI  ─── chat in browser              │
  │                                                │
  ├─► Continue    ─── code in VS Code              │
  │                                                │
  ├─► Khoj        ─── chat with your notes ◄── Joplin
  │                                                │
  └─► Meetily     ─── meeting summary ◄── Faster Whisper
                                                   │
ComfyUI ── standalone (its own model runtime) ─────┘

The trick is that all six client tools (Open WebUI, Continue, Khoj, Meetily, plus anything else you wire up) point at the single Ollama endpoint. You download a model once. Every app reuses it. Disk and RAM are the budgets to watch, not API quota.

Tradeoffs you'll hit

• Cloud quality vs local quality — Be honest: GPT-5 / Claude 4.5 still beat any 8B-quant local model at frontier reasoning, long-context, and code generation on unfamiliar codebases. Local wins on privacy, latency for short prompts, cost at volume, and offline use. The right mental model is "local for 80% of daily work, cloud for the hard 20%" — not "local replaces cloud".
• Apple Silicon vs NVIDIA — Apple Silicon M2/M3/M4 with 32 GB+ RAM runs 13B models comfortably via Metal/MPS. NVIDIA with 16 GB+ VRAM is faster on bigger models but louder, hotter, more expensive. Most of this pack runs well on a $2K Mac; ComfyUI and 70B models start asking for a real GPU.
• Quantized vs full precision — Most Ollama models default to Q4_K_M (4-bit quantization). You lose maybe 2-3% accuracy for 4x less RAM. Always start quantized. Only go full precision if you can measure a quality gap that matters to you.

Common pitfalls

• RAM blow-ups — running Open WebUI + Continue + Khoj simultaneously, each holding a model in memory, will OOM a 16 GB machine. Configure Ollama with OLLAMA_MAX_LOADED_MODELS=1 and let it page models in and out.
• Model files are huge — Llama 3.1 70B is 40 GB on disk. Plan storage before you ollama pull everything that looks interesting. Keep a kill list.
• MPS vs CUDA confusion — most install guides assume NVIDIA. On Apple Silicon, check for the -metal or mps variant of each tool. ComfyUI in particular needs the right Python wheel.
• "Actually I do need cloud for X" — be at peace with it. Routing your frontier-difficulty queries to Claude/GPT through a privacy-aware client (LibreChat with logging off, or just the API with Bearer and no organization ID) is a sane hybrid.
• Voice assistant ambition — Meetily + Faster Whisper handle batch transcription beautifully. Real-time conversational voice (sub-500ms latency, interruption) is still genuinely hard locally. Don't promise that to yourself in week one.