[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"pack-detail-log-analysis-search-es":3,"seo:pack:log-analysis-search:es":99},{"code":4,"message":5,"data":6},200,"操作成功",{"pack":7},{"slug":8,"icon":9,"tone":10,"status":11,"status_label":12,"title":13,"description":14,"items":15,"install_cmd":98},"log-analysis-search","📜","#374151","new","Nuevo · esta semana","Pack de Análisis y Búsqueda de Logs","Diez selecciones para el ingeniero que lee logs a las 3 a.m. — loggers estructurados, stack de envío y almacenamiento (Fluent Bit → Loki \u002F Elasticsearch \u002F ClickHouse), tail SQL local con lnav, agrupación de errores con Sentry, y servidores MCP para que tu agente IA consulte trazas y alertas directamente.",[16,28,35,43,51,58,67,74,81,91],{"id":17,"uuid":18,"slug":19,"title":20,"description":21,"author_name":22,"view_count":23,"vote_count":24,"lang_type":25,"type":26,"type_label":27},2134,"17e7e031-40e4-11f1-9bc6-00163e2b0d79","winston-versatile-logging-library-node-js-17e7e031","winston — Versatile Logging Library for Node.js","winston is the most popular logging library for Node.js, offering multiple transports, structured JSON output, and configurable log levels for production applications.","Script Depot",82,0,"en","skill","Skill",{"id":29,"uuid":30,"slug":31,"title":32,"description":33,"author_name":22,"view_count":34,"vote_count":24,"lang_type":25,"type":26,"type_label":27},1261,"6922366e-37b5-11f1-9bc6-00163e2b0d79","loguru-python-logging-made-stupidly-simple-6922366e","Loguru — Python Logging Made Stupidly Simple","Loguru replaces Python logging boilerplate with a single import. No handlers, no formatters, no config files — just logger.info(). It adds colorized output, structured context, file rotation, and exception diagnosis out of the box.",84,{"id":36,"uuid":37,"slug":38,"title":39,"description":40,"author_name":41,"view_count":42,"vote_count":24,"lang_type":25,"type":26,"type_label":27},1404,"18438936-38e7-11f1-9bc6-00163e2b0d79","fluent-bit-lightweight-high-performance-log-metrics-18438936","Fluent Bit — Lightweight High-Performance Log and Metrics Processor","Fluent Bit is a fast, lightweight telemetry agent from the Fluentd family. It collects logs, metrics and traces from any source, processes them with filters, and forwards them to dozens of backends.","AI Open Source",109,{"id":44,"uuid":45,"slug":46,"title":47,"description":48,"author_name":49,"view_count":50,"vote_count":24,"lang_type":25,"type":26,"type_label":27},958,"92fa7c1f-352f-11f1-9bc6-00163e2b0d79","grafana-loki-prometheus-inspired-log-aggregation-system-92fa7c1f","Grafana Loki — Prometheus-Inspired Log Aggregation System","Loki is a horizontally scalable, multi-tenant log aggregation system by Grafana Labs. Unlike other log systems, Loki indexes metadata about logs, not log content itself.","Grafana Labs",209,{"id":52,"uuid":53,"slug":54,"title":55,"description":56,"author_name":22,"view_count":57,"vote_count":24,"lang_type":25,"type":26,"type_label":27},1201,"8cbbd0e8-3734-11f1-9bc6-00163e2b0d79","elasticsearch-distributed-search-analytics-engine-8cbbd0e8","Elasticsearch — Distributed Search and Analytics Engine","Elasticsearch is the most popular search and analytics engine. It provides near-real-time full-text search, structured search, analytics, and logging across petabytes of data — powering search for Wikipedia, GitHub, Stack Overflow, and millions of applications.",166,{"id":59,"uuid":60,"slug":61,"title":62,"description":63,"author_name":41,"view_count":64,"vote_count":24,"lang_type":25,"type":65,"type_label":66},965,"2fce985b-3535-11f1-9bc6-00163e2b0d79","clickhouse-open-source-real-time-analytics-database-2fce985b","ClickHouse — Open Source Real-Time Analytics Database","ClickHouse is a lightning-fast, open-source column-oriented database for real-time analytics. Query billions of rows in milliseconds with SQL. Used by Cloudflare, Uber, eBay.",114,"config","Config",{"id":68,"uuid":69,"slug":70,"title":71,"description":72,"author_name":22,"view_count":73,"vote_count":24,"lang_type":25,"type":26,"type_label":27},1375,"4493f997-38c4-11f1-9bc6-00163e2b0d79","lnav-logfile-navigator-sql-live-tailing-4493f997","lnav — The Logfile Navigator with SQL and Live Tailing","lnav is an advanced log file viewer that understands dozens of log formats, provides SQL queries against log records, live-tails rotating files, and timestamps-merges multiple logs into one view.",113,{"id":75,"uuid":76,"slug":77,"title":78,"description":79,"author_name":41,"view_count":80,"vote_count":24,"lang_type":25,"type":26,"type_label":27},945,"ece57add-34d8-11f1-9bc6-00163e2b0d79","sentry-open-source-error-tracking-performance-monitoring-ece57add","Sentry — Open Source Error Tracking & Performance Monitoring","Sentry is the developer-first error tracking and performance monitoring platform. Capture exceptions, trace performance issues, and debug production errors across all languages.",173,{"id":82,"uuid":83,"slug":84,"title":85,"description":86,"author_name":87,"view_count":88,"vote_count":24,"lang_type":25,"type":89,"type_label":90},3608,"818380f9-674d-5217-88ab-f393ff99a247","signoz-mcp-server-query-traces-logs-alerts","SigNoz MCP Server — Query Traces, Logs & Alerts","SigNoz MCP Server connects MCP clients to your SigNoz instance: query traces\u002Flogs, inspect alerts, and automate observability workflows using an API key.","MCP Hub",85,"mcp","MCP",{"id":92,"uuid":93,"slug":94,"title":95,"description":96,"author_name":87,"view_count":97,"vote_count":24,"lang_type":25,"type":89,"type_label":90},3286,"284265e6-a9c0-5b2f-b769-60966256e908","clickhouse-mcp-read-only-defaults-drop-protection","ClickHouse MCP — Read-Only Defaults + Drop Protection","ClickHouse MCP connects MCP clients to ClickHouse or embedded chDB with read-only defaults, optional writes, and double opt-in for DROP\u002FTRUNCATE safety.",18,"tokrepo install pack\u002Flog-analysis-search",{"pageType":100,"pageKey":8,"locale":25,"title":101,"metaDescription":102,"h1":103,"tldr":104,"bodyMarkdown":105,"faq":106,"schema":122,"internalLinks":128,"citations":141,"wordCount":154,"generatedAt":155},"pack","Log Analysis + Search Pack — 10 Open-Source Tools for 3 a.m. Debugging","winston, Loguru, Fluent Bit, Loki, Elasticsearch, ClickHouse, lnav, Sentry, SigNoz MCP, ClickHouse MCP — an opinionated pipeline that turns raw stdout into structured, searchable, AI-queryable logs. Install in order.","Log Analysis + Search Pack — The Stack You Actually Need at 3 a.m.","Ten picks in install order: structured loggers first (winston, Loguru), then ship-and-store (Fluent Bit → Loki \u002F Elasticsearch \u002F ClickHouse), then read tools (lnav for local, Sentry for grouping), then MCP servers so an AI agent can answer 'what broke at 02:47?' without you grepping anything.","## What this pack solves\n\nIt's 3 a.m. The pager says 5xx rate jumped. You SSH in, `tail -f` a file that's already rotated, grep for an exception that's actually three exceptions sharing a substring, and 40 minutes later you've narrowed it to \"something in checkout.\" That's the problem this pack kills.\n\nThe goal isn't observability theatre — no fifteen dashboards no one opens. The goal is: **structured logs go in one end, a question comes out the other**, and the question can be asked by you, a teammate, or an AI agent with MCP access.\n\nEvery pick here is **open-source or has a self-hostable open-source core**. The full pipeline runs on a single mid-size VM up to about 50 GB\u002Fday of log volume; past that, you split Loki\u002FClickHouse onto their own boxes. No vendor lock-in, no per-GB pricing surprises.\n\n## Install in this order\n\n1. **winston (Node)** or **Loguru (Python)** — start with structured logging in your app. JSON output, one log line per event, every line has `timestamp`, `level`, `service`, `trace_id`. If your logs aren't structured at the source, every downstream tool is fighting your formatter instead of doing its job.\n2. **Fluent Bit** — the shipper. Tails files \u002F journald \u002F Docker logs, parses JSON, adds host labels, batches, retries, ships to your store. Tiny C binary, ~5 MB RSS, runs as a sidecar or DaemonSet. The non-negotiable middle layer.\n3. **Grafana Loki** — the store, default pick. Indexes labels (not full text), uses object storage, cheap to run. Best when you ship structured JSON and search by `service=checkout level=error`. LogQL feels like PromQL — five minutes to learn if you know Prometheus.\n4. **Elasticsearch** — alternative store when you need full-text search across log message bodies, not just labels. Heavier (JVM, more disk) but unbeatable when the question is \"find every log mentioning `OrderId=abc-123` anywhere\". Pair with Kibana for the UI.\n5. **ClickHouse** — alternative store when you have **a lot** of logs (>100 GB\u002Fday) and want SQL. Columnar, eats compressed JSON for breakfast, queries that take Elasticsearch 30 seconds run in 1 second. The right pick at scale.\n6. **lnav** — the local terminal log navigator. SQL queries against log files directly, live tailing, format auto-detection, syntax-highlighted error highlighting. The tool you reach for when SSH'd to one box and the centralized store isn't relevant. Single binary, no daemon.\n7. **Sentry** — error grouping + alerting. Different from Loki\u002FES\u002FCH — those store all logs; Sentry catches **exceptions and stack traces**, groups duplicates intelligently, sends an alert when a new error appears or volume spikes. Self-hostable.\n8. **SigNoz MCP Server** — Model Context Protocol bridge. Lets Claude \u002F ChatGPT \u002F Cursor query SigNoz's traces, logs, and alerts conversationally. \"What's the slowest endpoint in the last hour?\" → real answer from real data, not hallucinated.\n9. **ClickHouse MCP** — the safer MCP pick when your store is ClickHouse. Read-only by default, drop-table protection, parameterized queries. Hand it to an agent without panicking that it'll `DROP DATABASE production`.\n\n## How the pipeline fits together\n\n```\n[ your app ]\n     │\n     ▼  winston \u002F Loguru   (structured JSON to stdout)\n     │\n[ Fluent Bit ]   (parses, labels, batches)\n     │\n     ├──▶ Loki           ← cheap, label-indexed\n     ├──▶ Elasticsearch  ← full-text-heavy queries\n     └──▶ ClickHouse     ← high-volume SQL analytics\n     │\n     ├──▶ Sentry         ← errors only, grouped + alerted\n     │\n     ▼  read paths:\n        - lnav            (local file, no daemon)\n        - Grafana         (Loki UI)\n        - Kibana          (ES UI)\n        - SigNoz MCP      (AI agent → traces\u002Flogs\u002Falerts)\n        - ClickHouse MCP  (AI agent → SQL, read-only)\n```\n\nThe critical insight: **pick one store, not all three**. Loki is the right default for 80% of teams. Move to Elasticsearch only if full-text search across message bodies is a daily need. Move to ClickHouse only when log volume + query latency push you off Loki. The pack lists all three because the right answer depends on your traffic shape — not because you should install all three.\n\n## Tradeoffs you'll hit\n\n- **Loki vs Elasticsearch vs ClickHouse** — Loki is cheapest to run and easiest to operate, but its full-text search is genuinely weak (substring matches across millions of lines are slow). Elasticsearch is the opposite: heavy to run, brilliant at \"find this string anywhere.\" ClickHouse is the SQL nuclear option — incredibly fast at aggregations but you write SQL, not LogQL\u002FKQL. Pick the one whose tradeoff matches your usual question.\n- **winston vs Loguru vs pino vs zap** — winston is the Node default but pino is faster (and the pino ecosystem has caught up). Loguru is the Python default but `structlog` is more flexible if you have complex context binding. This pack picks the defaults; switch later if you hit a real limit.\n- **Sentry vs the log store** — Sentry overlaps with your log store on error capture. Worth running both: Sentry for the \"new error appeared, page on-call\" loop; the log store for the \"reconstruct the request sequence\" loop. They're different jobs.\n- **MCP server vs custom agent tools** — MCP standardizes how agents call your tools, so any MCP-aware client (Claude Desktop, Cursor, ChatGPT custom GPTs) can use the same SigNoz\u002FClickHouse access. Custom OpenAI function-calling is more flexible per-agent but doesn't port. MCP wins for any tool you'll expose to more than one agent runtime.\n\n## Common pitfalls\n\n- **Logging strings instead of structured fields** — `log.info(\"user \" + userId + \" failed\")` is unsearchable. `log.info({ event: \"login_failed\", userId })` is queryable in any of the stores. This is the single change that makes 80% of the rest of the stack worthwhile.\n- **Fluent Bit without flow control** — under burst load, Fluent Bit's tail input can OOM. Set `Mem_Buf_Limit` and enable file-based buffering before you discover this in production.\n- **Loki labels with high cardinality** — never label by `user_id`, `request_id`, `trace_id`. Loki's storage cost is linear in unique label-set count; one accidental high-cardinality label can 100x your bill. Keep labels to `service`, `env`, `level`, `host`.\n- **Sentry sample rate at 100%** — fine until your background job spams the same error 50k times in 10 minutes and you hit your quota. Use the SDK's `before_send` to deduplicate aggressive loops at the source.\n- **MCP server exposed to read-write by default** — every MCP server doc shows the read-write example first. For ClickHouse MCP specifically, the read-only mode (set in env) is the only safe default when an agent is on the other end. Audit the config.\n- **Indexing log messages as schema** — ES\u002FCH will let every JSON field become a column or mapping. Six months later you have 12,000 fields, half of them typos from one buggy service. Normalize event names and field names at the logger, not at the store.",[107,110,113,116,119],{"q":108,"a":109},"Do I really need all three of Loki, Elasticsearch, and ClickHouse?","No — pick one. The pack lists all three because the right answer depends on your shape. Loki is the default for ~80% of teams: cheap, label-indexed, easy to run. Pick Elasticsearch if your daily question is 'find this string anywhere in any message body' (it's much better at unstructured full-text). Pick ClickHouse when you cross ~100 GB\u002Fday or need real SQL analytics on logs. Running all three is fine for a comparison week, painful as a permanent state.",{"q":111,"a":112},"Where does AI fit in this stack — is the SigNoz MCP just a chat UI?","It's more than a chat UI. The MCP server exposes traces, logs, and alerts as tools an AI agent can call autonomously. Practical examples: a Claude agent triages a Sentry alert by querying SigNoz for the trace, pulls the corresponding logs from Loki, and writes a one-paragraph incident summary into your ticket — all from one prompt. The ClickHouse MCP plays the same role for SQL-style log analytics, with read-only enforced so the agent can't drop a table.",{"q":114,"a":115},"Why winston\u002FLoguru instead of just printing JSON manually?","Three reasons. First: structured fields are added by API, not string concatenation, so they're consistent across the codebase. Second: log levels, sampling, and transports (file \u002F stdout \u002F network) are decoupled from call sites. Third: ecosystems — winston has 100+ transports, Loguru integrates with FastAPI\u002FDjango out of the box. You could roll your own with `json.dumps`, but you'll re-invent these features within a month.",{"q":117,"a":118},"Is Sentry redundant if I already ship error logs to Loki?","No, the jobs differ. Loki\u002FES\u002FCH stores everything indiscriminately and answers 'show me the sequence around this request.' Sentry deduplicates exceptions by stack trace, groups them as 'issues,' tracks first-seen \u002F regression \u002F volume spike, and pages you when a new issue appears. Treat Sentry as your error inbox and your log store as the witness — both serve you, neither replaces the other.",{"q":120,"a":121},"Can this whole pack run on one VM, or do I need a Kubernetes cluster?","One mid-size VM (16 vCPU, 32 GB RAM, 500 GB SSD) handles up to ~20 GB\u002Fday of log volume with Loki + Fluent Bit + Sentry self-hosted comfortably. Past 50 GB\u002Fday, split Loki object storage off to S3-compatible storage and give ClickHouse\u002FElasticsearch their own nodes. You don't need Kubernetes for this — docker-compose is fine and arguably preferable below 50 GB\u002Fday. Add k8s when you have ops appetite to maintain it, not because the log pipeline requires it.",{"@context":123,"@type":124,"name":125,"description":126,"numberOfItems":127,"inLanguage":25},"https:\u002F\u002Fschema.org","ItemList","Log Analysis + Search Pack","Ten open-source picks in install order: structured loggers, log shippers, three store choices (Loki, Elasticsearch, ClickHouse), local SQL tailing with lnav, error grouping with Sentry, and MCP servers for AI agent queries.",10,[129,133,137],{"url":130,"anchor":131,"reason":132},"\u002Fen\u002Fai-tools-for\u002Fobservability","AI tools for observability","The MCP servers in this pack are the agent-facing side of observability",{"url":134,"anchor":135,"reason":136},"\u002Fen\u002Ftopics","Browse other topic packs","Backend toolkit, data engineer toolbox, and more curated packs",{"url":138,"anchor":139,"reason":140},"\u002Fen\u002Ffeatured","Featured assets on TokRepo","These ten tools sit alongside the broader curated catalog",[142,146,150],{"claim":143,"source_name":144,"source_url":145},"Grafana Loki indexes log labels rather than full text","Grafana Loki documentation","https:\u002F\u002Fgrafana.com\u002Fdocs\u002Floki\u002Flatest\u002F",{"claim":147,"source_name":148,"source_url":149},"ClickHouse is a columnar OLAP database used for log analytics at scale","ClickHouse documentation","https:\u002F\u002Fclickhouse.com\u002Fdocs",{"claim":151,"source_name":152,"source_url":153},"Model Context Protocol is an open standard for connecting tools to LLM agents","Model Context Protocol specification","https:\u002F\u002Fmodelcontextprotocol.io\u002F",920,"2026-05-22T00:00:00Z"]