# crw — Fast Web Scraping + Search MCP in Rust

> crw is a Rust web scraping/search tool with a Firecrawl-compatible API plus built-in MCP support for agents. Verified 87★; pushed 2026-05-14.

## Install

Copy the content below into your project:

## Quick Use

```bash
curl -fsSL https://raw.githubusercontent.com/us/crw/main/install.sh | CRW_BINARY=crw sh
crw setup
crw example.com
# Add to Claude Code (local) (per README):
claude mcp add crw -- npx crw-mcp
```

## Intro

crw is a Rust web scraping/search tool with a Firecrawl-compatible API plus built-in MCP support for agents. Verified 87★; pushed 2026-05-14.

**Best for:** Agent workflows that need reliable web-to-markdown scraping plus a local CLI or MCP server option

**Works with:** Rust CLI; optional local mode; integrates with MCP clients like Claude Code via `claude mcp add` (per README)

**Setup time:** 8-20 minutes

### Key facts (verified)

- GitHub: 87 stars · 5 forks · pushed 2026-05-14.
- License: AGPL-3.0 · owner avatar + repo URL verified via GitHub API.
- README-backed entrypoint: `crw setup`.

## Main

- Use local mode first to validate output quality and latency before you depend on it in a larger agent workflow.

- If you need API compatibility, keep the Firecrawl-style endpoint paths in your client code so switching providers stays easy.

- Treat rendering as an operational variable: allow fallback renderers and surface warnings to your agent logs.

- For production, add rate limiting and host allowlists so web access doesn’t become an unbounded crawler.

### README (excerpt)

<a name="readme-top"></a>
<p align="center">
  <a href="https://fastcrw.com">
    <img src="docs/logo.png" alt="fastCRW" height="120" />
  </a>
  <p align="center">The web scraper built for AI agents. Single binary. Zero config.</p>
  <p align="center">
    <a href="https://crates.io/crates/crw-server"><img src="https://img.shields.io/crates/v/crw-server.svg" alt="crates.io"></a>
    <a href="https://github.com/us/crw/actions"><img src="https://github.com/us/crw/workflows/CI/badge.svg" alt="CI"></a>
    <a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue.svg" alt="License"></a>
    <a href="https://github.com/us/crw/stargazers"><img src="https://img.shields.io/github/stars/us/crw?style=social" alt="GitHub Stars"></a>
    <a href="https://fastcrw.com"><img src="https://img.shields.io/badge/Managed%20Cloud-fastcrw.com-blueviolet" alt="fastcrw.com"></a>
  </p>
  <p align="center">
    <a href="https://twitter.com/fastcrw">
      <img src="https://img.shields.io/badge/Follow%20on%20X-000000?style=for-the-badge&logo=x&logoColor=white" alt="Follow on X" />
    </a>
    <a href="https://www.linkedin.com/company/fastcrw">
      <img src="https://img.shields.io/badge/Follow%20on%20LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white" alt="Follow on LinkedIn" />
    </a>
    <a href="https://discord.gg/kkFh2SC8">
      <img src="https://img.shields.io/badge/Join%20our%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Join our Discord" />
    </a>
  </p>
  <p align="center">
    <a href="https://www.producthunt.com/products/fastcrw?utm_source=badge-featured&utm_medium=badge&utm_campaign=badge-fastcrw" target="_blank" rel="noopener noreferrer"><img src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=1116966&theme=light&t=1775671073751" alt="fastCRW - Search + scrape live web results for AI agents | Product Hunt" width="250" height="54" /></a>
  </p>
  <p align="center">
    Works with: Claude Code · Cursor · Windsurf · Cline · Copilot · Continue.dev · Codex · Gemini CLI
  </p>
  <p align="center">
    <a href="#-quick-start">Quick Start</a> &bull;
    <a href="#-connect-to-ai-agents">AI Agents</a> &bull;


### Source-backed notes

- README includes a quick start install script via `curl ... install.sh | CRW_BINARY=crw sh` followed by `crw setup`.
- README states it provides a Firecrawl-compatible API and includes MCP integration examples for Claude Code via `npx crw-mcp`.
- README contains benchmark tables and resource claims; treat them as README-provided context, not independently verified metrics.

### FAQ

- **Is it only an API server?**: README describes both a CLI and MCP integration; you can start with the CLI locally.
- **How do I connect it to my agent?**: README shows adding it to Claude Code via `claude mcp add ... -- npx crw-mcp`.
- **How do I keep web access safe?**: Use allowlists, rate limits, and start with minimal scope in local mode.

## Source & Thanks

> Created by [us](https://github.com/us). Licensed under AGPL-3.0.
>
> [us/crw](https://github.com/us/crw) — ⭐ 87

Thanks to the upstream maintainers and contributors for publishing this work under an open license.

---

<!-- ZH -->

## Quick Use

```bash
curl -fsSL https://raw.githubusercontent.com/us/crw/main/install.sh | CRW_BINARY=crw sh
crw setup
crw example.com
# Add to Claude Code (local) (per README):
claude mcp add crw -- npx crw-mcp
```

## Intro

crw 是 Rust 写的抓取/搜索工具，提供 Firecrawl 兼容 API，并内置面向 agent 的 MCP（含 CLI 与本地模式）；已验证 87★，更新于 2026-05-14。

**Best for:** 需要稳定网页抓取/转 Markdown，并希望同时支持 CLI 与 MCP 方式接入的 agent 工作流

**Works with:** Rust CLI；可本地运行；README 提供通过 `claude mcp add` 接入 MCP 客户端的示例

**Setup time:** 8-20 minutes

### Key facts (verified)

- GitHub：87 stars · 5 forks；最近更新 2026-05-14。
- 许可证：AGPL-3.0；作者头像与仓库链接均已通过 GitHub API 复核。
- README 中可对照的入口：`crw setup`。

## Main

- 先用本地模式跑通输出质量与延迟，再把它接进更大的 agent 流水线里。

- 如果要兼容生态，客户端侧尽量保持 Firecrawl 风格的 endpoint 路径，未来切换成本更低。

- 把渲染策略当作运维变量：允许 failover，并把 warnings 明确写进日志供 agent/人排查。

- 生产环境建议加限流与 host allowlist，避免 web access 变成无边界爬虫。

### README (excerpt)

<a name="readme-top"></a>
<p align="center">
  <a href="https://fastcrw.com">
    <img src="docs/logo.png" alt="fastCRW" height="120" />
  </a>
  <p align="center">The web scraper built for AI agents. Single binary. Zero config.</p>
  <p align="center">
    <a href="https://crates.io/crates/crw-server"><img src="https://img.shields.io/crates/v/crw-server.svg" alt="crates.io"></a>
    <a href="https://github.com/us/crw/actions"><img src="https://github.com/us/crw/workflows/CI/badge.svg" alt="CI"></a>
    <a href="LICENSE"><img src="https://img.shields.io/badge/license-AGPL--3.0-blue.svg" alt="License"></a>
    <a href="https://github.com/us/crw/stargazers"><img src="https://img.shields.io/github/stars/us/crw?style=social" alt="GitHub Stars"></a>
    <a href="https://fastcrw.com"><img src="https://img.shields.io/badge/Managed%20Cloud-fastcrw.com-blueviolet" alt="fastcrw.com"></a>
  </p>
  <p align="center">
    <a href="https://twitter.com/fastcrw">
      <img src="https://img.shields.io/badge/Follow%20on%20X-000000?style=for-the-badge&logo=x&logoColor=white" alt="Follow on X" />
    </a>
    <a href="https://www.linkedin.com/company/fastcrw">
      <img src="https://img.shields.io/badge/Follow%20on%20LinkedIn-0077B5?style=for-the-badge&logo=linkedin&logoColor=white" alt="Follow on LinkedIn" />
    </a>
    <a href="https://discord.gg/kkFh2SC8">
      <img src="https://img.shields.io/badge/Join%20our%20Discord-5865F2?style=for-the-badge&logo=discord&logoColor=white" alt="Join our Discord" />
    </a>
  </p>
  <p align="center">
    <a href="https://www.producthunt.com/products/fastcrw?utm_source=badge-featured&utm_medium=badge&utm_campaign=badge-fastcrw" target="_blank" rel="noopener noreferrer"><img src="https://api.producthunt.com/widgets/embed-image/v1/featured.svg?post_id=1116966&theme=light&t=1775671073751" alt="fastCRW - Search + scrape live web results for AI agents | Product Hunt" width="250" height="54" /></a>
  </p>
  <p align="center">
    Works with: Claude Code · Cursor · Windsurf · Cline · Copilot · Continue.dev · Codex · Gemini CLI
  </p>
  <p align="center">
    <a href="#-quick-start">Quick Start</a> &bull;
    <a href="#-connect-to-ai-agents">AI Agents</a> &bull;


### Source-backed notes

- README 的 Quick Start 使用 `curl ... install.sh | CRW_BINARY=crw sh` 安装，并通过 `crw setup` 初始化。
- README 说明提供 Firecrawl 兼容 API，并给出通过 `npx crw-mcp` 接入 Claude Code 的 MCP 示例。
- README 包含 benchmark/资源占用等声明；这些属于 README 语境信息，并非本脚本额外验证的性能结论。

### FAQ

- **它只是 API server 吗？**：README 同时提供 CLI 与 MCP 接入方式；建议先用 CLI 本地验证。
- **怎么接入 agent？**：README 给出 `claude mcp add ... -- npx crw-mcp` 的示例用法。
- **如何更安全？**：用 allowlist/限流，并从最小可用范围开始逐步扩展。

## Source & Thanks

> Created by [us](https://github.com/us). Licensed under AGPL-3.0.
>
> [us/crw](https://github.com/us/crw) — ⭐ 87


---
Source: https://tokrepo.com/en/workflows/crw-fast-web-scraping-search-mcp-in-rust
Author: Script Depot