[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"pack-detail-ai-cost-optimization-stack-zh":3,"seo:pack:ai-cost-optimization-stack:zh":102},{"code":4,"message":5,"data":6},200,"操作成功",{"pack":7},{"slug":8,"icon":9,"tone":10,"status":11,"status_label":12,"title":13,"description":14,"items":15,"install_cmd":101},"ai-cost-optimization-stack","💰","#F59E0B","new","本周新建","AI 成本优化 — 省 token 的工程化栈","10 件套，给账单已经是真正成本项的 SaaS \u002F agent 团队 — LiteLLM \u002F OpenRouter \u002F Manifest router \u002F Portkey \u002F Helicone 缓存 \u002F Cloudflare AI Gateway \u002F LLMLingua 压缩 \u002F TokenCost 计算器 \u002F LiteLLM 成本看板 \u002F Fireworks 微调。五层：先观测、再缓存、再路由、再压缩、再微调。不损质量的前提下 10–50% 节省是常见数。",[16,28,38,46,53,61,69,79,86,93],{"id":17,"uuid":18,"slug":19,"title":20,"description":21,"author_name":22,"view_count":23,"vote_count":24,"lang_type":25,"type":26,"type_label":27},2789,"0f113965-1adc-4435-982b-fb613fa4d157","litellm-proxy-unified-gateway-for-100-llm-apis","LiteLLM Proxy — Unified Gateway for 100+ LLM APIs","LiteLLM Proxy maps 100+ LLM providers (Anthropic, OpenAI, Bedrock, Vertex) to one OpenAI-compatible endpoint. Auth, rate limit, cost track, fallbacks.","LiteLLM (BerriAI)",92,0,"en","agent","Agent",{"id":29,"uuid":30,"slug":31,"title":32,"description":33,"author_name":34,"view_count":35,"vote_count":24,"lang_type":25,"type":36,"type_label":37},2829,"7bb772b3-1ab0-4d27-a758-1cd9acc4f6ff","openrouter-unified-api-for-300-llms-with-auto-failover","OpenRouter — Unified API for 300+ LLMs with Auto Failover","OpenRouter is one OpenAI-compatible endpoint for 300+ LLMs across 60+ providers. Transparent pricing, no markup, automatic failover when a route is down.","OpenRouter",96,"skill","Skill",{"id":39,"uuid":40,"slug":41,"title":42,"description":43,"author_name":44,"view_count":45,"vote_count":24,"lang_type":25,"type":36,"type_label":37},863,"15266cba-33d7-11f1-9bc6-00163e2b0d79","manifest-smart-llm-router-cuts-costs-70-15266cba","Manifest — Smart LLM Router That Cuts Costs 70%","Intelligent LLM routing that scores requests across 23 dimensions in under 2ms. Routes to the cheapest capable model among 300+ options from 13+ providers. MIT, 4,200+ stars.","AI Open Source",185,{"id":47,"uuid":48,"slug":49,"title":50,"description":51,"author_name":44,"view_count":52,"vote_count":24,"lang_type":25,"type":36,"type_label":37},308,"585d3a26-0cca-47cb-ac88-2797a853367d","portkey-ai-gateway-route-250-llms-585d3a26","Portkey AI Gateway — Route to 250+ LLMs","Portkey AI Gateway routes to 250+ LLMs with sub-1ms latency, 40+ guardrails, retries, fallbacks, and caching. 11.1K+ stars. Apache 2.0.",143,{"id":54,"uuid":55,"slug":56,"title":57,"description":58,"author_name":59,"view_count":60,"vote_count":24,"lang_type":25,"type":36,"type_label":37},2817,"5d1acc2e-f42d-4fce-aec7-771506f858ae","helicone-cache-cut-llm-spend-with-drop-in-response-caching","Helicone Cache — Cut LLM Spend with Drop-In Response Caching","Helicone Cache short-circuits identical LLM requests at the proxy. Set Helicone-Cache-Enabled header, exact-match responses come back in ms at zero cost.","Helicone",112,{"id":62,"uuid":63,"slug":64,"title":65,"description":66,"author_name":67,"view_count":68,"vote_count":24,"lang_type":25,"type":36,"type_label":37},624,"b1962c77-9ecf-4a84-87b1-e7d4b677dabe","cloudflare-ai-gateway-llm-proxy-cache-analytics-b1962c77","Cloudflare AI Gateway — LLM Proxy, Cache & Analytics","Free proxy gateway for LLM API calls with caching, rate limiting, cost tracking, and fallback routing across providers. Reduce costs up to 95% with response caching. 7,000+ stars.","Cloudflare",173,{"id":70,"uuid":71,"slug":72,"title":73,"description":74,"author_name":75,"view_count":76,"vote_count":24,"lang_type":25,"type":77,"type_label":78},862,"1510da0c-33d7-11f1-9bc6-00163e2b0d79","llmlingua-compress-prompts-20x-minimal-loss-1510da0c","LLMLingua — Compress Prompts 20x with Minimal Loss","Microsoft research tool for prompt compression. Reduce token usage up to 20x while maintaining LLM performance. Solves lost-in-the-middle for RAG. MIT, 6,000+ stars.","Script Depot",249,"prompt","Prompt",{"id":80,"uuid":81,"slug":82,"title":83,"description":84,"author_name":22,"view_count":85,"vote_count":24,"lang_type":25,"type":36,"type_label":37},2791,"72b2e16c-71b4-4702-87ed-f6ea3ba99f69","litellm-cost-tracking-per-project-llm-spend-dashboard","LiteLLM Cost Tracking — Per-Project LLM Spend Dashboard","LiteLLM ships a built-in cost dashboard. Track LLM spend by project, user, model, tag. Hard budgets that block at the proxy. SOC2 \u002F SSO via Pro tier.",75,{"id":87,"uuid":88,"slug":89,"title":90,"description":91,"author_name":75,"view_count":92,"vote_count":24,"lang_type":25,"type":36,"type_label":37},859,"43b26691-33ce-11f1-9bc6-00163e2b0d79","tokencost-llm-price-calculator-400-models-43b26691","TokenCost — LLM Price Calculator for 400+ Models","Client-side token counting and USD cost estimation for 400+ LLMs. 3 lines of Python to track prompt and completion costs. Supports OpenAI, Anthropic, Mistral, AWS Bedrock. MIT, 2K+ stars.",181,{"id":94,"uuid":95,"slug":96,"title":97,"description":98,"author_name":99,"view_count":100,"vote_count":24,"lang_type":25,"type":36,"type_label":37},2850,"2f07f6a8-78ac-480a-b7a4-00282133dd4d","fireworks-fine-tuning-serverless-lora-on-llama-in-30-min","Fireworks Fine-Tuning — Serverless LoRA on Llama in 30 min","Fireworks runs serverless LoRA fine-tuning on Llama, Qwen, Mixtral. Upload JSONL, get a deployed fine-tune in 30 min on the same endpoint.","Fireworks AI",49,"tokrepo install pack\u002Fai-cost-optimization-stack",{"pageType":103,"pageKey":8,"locale":104,"title":13,"metaDescription":105,"h1":13,"tldr":106,"bodyMarkdown":107,"faq":108,"schema":124,"internalLinks":175,"citations":188,"wordCount":201,"generatedAt":202},"pack","zh","LiteLLM \u002F OpenRouter \u002F Manifest \u002F Portkey \u002F Helicone 缓存 \u002F Cloudflare AI Gateway \u002F LLMLingua \u002F TokenCost \u002F Fireworks 微调 — 五层栈，在不掉质量的前提下把 LLM 账单砍 10–50%。TokRepo 一键安装。","10 件套，给账单已经是真实成本项的 SaaS \u002F agent 团队。按顺序五层：观测（TokenCost \u002F LiteLLM 成本看板）、缓存（Helicone \u002F Cloudflare AI Gateway）、路由（LiteLLM \u002F OpenRouter \u002F Manifest \u002F Portkey）、压缩（LLMLingua）、微调（Fireworks LoRA）。按顺序走完的团队多数能省 10–50%，用户无感；上限只在高缓存命中的客服聊天、大批量分类这类场景出现，不适用于全新 agent 推理。","## 这个 pack 包含什么\n\n月度 OpenAI 账单跨过五位数那天，每个工程师突然都对缓存有了观点。这个 pack 是无聊但有用的有序剧本，真正能动账单：先观测再优化、先缓存再路由、先路由再压缩、先压缩再微调。多数把五层都走完的团队能省 10–50%，用户感觉不到 — 上限只在客服 \u002F FAQ（高缓存命中）和大批量分类（小模型微调）这类场景出现，不适用于全新的 agent 推理任务，那种要小心。\n\n| # | 资产 | 层 | 做什么 |\n|---|---|---|---|\n| 1 | LiteLLM Proxy | 路由 | 一个 OpenAI 兼容端点接 100+ 提供商、fallback 链、按 key 预算 |\n| 2 | OpenRouter 统一 API | 路由 | 托管网关接 300+ 模型、单 key、自动 failover |\n| 3 | Manifest 智能路由 | 路由 | 语义路由 — 先走便宜模型，置信度低再升级 |\n| 4 | Portkey AI Gateway | 路由 | 企业级网关、250+ LLM、虚拟 key、guardrails、缓存 |\n| 5 | Helicone 缓存 | 缓存 | 一个 proxy header 接入响应缓存；精确匹配 + 语义两种模式 |\n| 6 | Cloudflare AI Gateway | 缓存 | 边缘缓存 + 分析 + 重试 + 限流 — 有免费档 |\n| 7 | LLMLingua | 压缩 | prompt 压缩最高 20×，质量损失最小 |\n| 8 | LiteLLM 成本看板 | 观测 | 按项目 \u002F 用户 \u002F 模型追踪花费 + 硬预算阻断 |\n| 9 | TokenCost | 观测 | 400+ 模型的离线价格计算器 — 上线前先算清楚 |\n| 10 | Fireworks 无服务 LoRA | 微调 | Llama 上 30 分钟做 LoRA — 用窄场景替换前沿模型 |\n\n## 按这个顺序装 — 先观测、最后才微调\n\n```bash\n# 第一层：观测（动手改之前先做这层）\ntokrepo install tokencost                                  # 离线价格模型\ntokrepo install litellm-cost-tracking                      # 实时按项目看板\n\n# 第二层：缓存（重复场景下 ROI 最高、风险最低）\ntokrepo install helicone-cache                             # 一行接入响应缓存\n# 或: tokrepo install cloudflare-ai-gateway                # 边缘缓存 + 分析，免费档\n\n# 第三层：路由（先走便宜，安全升级）\ntokrepo install litellm-proxy                              # 自部署、BYOK\n# 或: tokrepo install openrouter-unified-api               # 托管、一个 key 接 300+ 模型\ntokrepo install manifest-smart-router                      # 上面再套语义路由\n\n# 第四层：压缩（前三层做出基线再上）\ntokrepo install llmlingua                                  # prompt 压缩 2–20×\n\n# 第五层：微调（最后的手段 — 真烧工程师时间）\ntokrepo install fireworks-fine-tune                        # 给那一个窄任务在 Llama 上做 LoRA\n```\n\n顺序比选型更重要。跳着来要么浪费工程师时间去微调一个原本 $0.50\u002FMTok 缓存就能替的模型，要么默默把质量压下去而你都归因不了 — 因为你从来没插桩过基线。不光鲜的真相：最大省钱来自第二层（缓存）和第三层（路由），不是看起来更高级的第五层。\n\n### 第一层 — 观测\n把 TokenCost 当库装上，每个 PR 都能在 staging 打印改动前后的 token 数。生产装 LiteLLM Proxy（开成本追踪）或 Portkey，每条调用都有按项目 \u002F 用户 \u002F 模型拆分的账本。在你能精确到两位有效数字回答「一次用户会话花多少钱」之前，不要进第二层。\n\n### 第二层 — 缓存\nHelicone 一个 proxy header 就能开精确匹配缓存，加 embedding 相似度还能开语义缓存。Cloudflare AI Gateway 在边缘做同样的事，有免费档、默认 24 小时缓存。客服机器人、FAQ、检索问答、幂等分类场景，30–70% 命中率是常见的。Agent 规划循环和创意生成不是，别硬上。\n\n### 第三层 — 路由\nLiteLLM Proxy 是自部署默认款 — 一个 OpenAI 兼容 URL 映射到 Anthropic \u002F Bedrock \u002F Vertex \u002F OpenAI，外加 fallback 链和按 key 预算。OpenRouter 是不想自部署时的托管版。在两者之上再加一层 Manifest，它给 prompt 分类、先丢便宜模型、置信度低才升前沿。Portkey 在网关层加企业功能（SSO \u002F 审计 \u002F 虚拟 key \u002F guardrails）。\n\n### 第四层 — 压缩\nLLMLingua 按 token 重要度评分把 prompt 压到 20×。质量损失完全看任务类型：摘要、抽取、分类基本无感；数学、代码生成、复杂推理就很危险。上线前必须对着 eval 套件 A\u002FB。把压缩比当预算看，不是目标。\n\n### 第五层 — 微调\n在 Llama 上跑 Fireworks 无服务 LoRA，大约 30 分钟训练就能把前沿模型替换在一个窄高频任务上。值得做的条件：≥1 万条标注或 LLM 生成的样本、任务稳定（不是 prompt 还在每周改）、单这个任务上前沿模型账单足够大、能 cover 工程时间。低量端点不要为了省 5% 去微调。\n\n## 它们怎么拼起来\n\n```\nclient app\n   │\n   ▼\n┌──────────────┐    cache hit  ┌────────────┐\n│ Helicone \u002F   │──────────────▶│ cached     │\n│ Cloudflare   │               │ response   │\n│ AI Gateway   │               └────────────┘\n└──────┬───────┘ cache miss\n       │\n       ▼\n┌──────────────────┐  classify  ┌─────────────────┐\n│ Manifest router  │───────────▶│ cheap model     │\n│（语义）          │            │ (Llama \u002F Haiku) │\n└──────┬───────────┘            └─────────────────┘\n       │ 置信度低 \u002F 升级\n       ▼\n┌─────────────────┐  可选      ┌──────────────┐\n│ LiteLLM \u002F       │───────────▶│ LLMLingua    │\n│ OpenRouter \u002F    │ 压缩        │ 先压一道     │\n│ Portkey 网关    │            └──────┬───────┘\n└──────┬──────────┘                   │\n       │                              │\n       ▼                              ▼\n  前沿模型 (Opus \u002F GPT-4 \u002F Gemini Ultra)\n       │\n       ▼\nLiteLLM 成本账本 + TokenCost 对账\n```\n\n## 取舍（实话部分）\n\n- **缓存命中 vs 新鲜度** — 客服机器人 70% 命中是赚到，股价助手 70% 命中是事故。TTL 按路由设、不要全局设。FAQ 默认 24 小时合适，时效任何敏感的都不行。\n- **路由延迟开销** — 语义路由会在真正调用前加 50–200 ms（embedding + 分类）。聊天里看不见，流式语音 agent 里看得见。前后都测端到端 p95。\n- **压缩的质量损失** — LLMLingua 5× 在摘要上几乎免费，20× 在抽取里开始丢事实。压缩上线必须配 held-out eval set，不能只看几条抽样。\n- **便宜模型路由错** — 把数学题路由到 Haiku 因为路由器以为是「简单问答」是一种静默回退，一周后用户抱怨「答错」才会浮上来。把路由决策跟 trace 一起记，每周复查置信度最低的那一档。\n- **微调锁定** — 在 Llama 3.1 70B 上的 LoRA 是你的、能搬走；闭源模型上的微调不是。基座挑稳了再开。\n- **观测本身也不免费** — 自部署 Langfuse \u002F LiteLLM 成本看板要基建；托管 Helicone \u002F Portkey 每调用一点费用。盈亏平衡通常在每月 100 万调用左右 — 这之下，免费档够用。\n\n## 常见踩坑\n\n- **没观测就优化** — 工程师「GPT-4 太贵」就开始微调模型，从没插桩过真正的花费 top 端点。十次有八次账单被一个功能占大头、不是整个产品。先装第一层。\n- **不小心把鉴权 \u002F 私密内容缓存了** — 只按 prompt 文本做 key 的语义缓存会很愉快地把用户 A 的医疗对话回给用户 B，只要他们问得相近 embedding 就对上。缓存 key 必须带 user \u002F tenant \u002F 鉴权上下文。\n- **错任务路由到错模型** — Tool 调用、JSON 模式结构化输出在很多便宜模型上会崩。上线前用真实生产流量分布跑路由分类器、不要用合成 benchmark。\n- **把「省 token」当「省钱」** — 大多数提供商输入 token 比输出 token 便宜 3–5 倍。压缩 8k 输入而模型输出本来就 200 token 的场景，省下的比表面比例小得多。盯美元、不要盯 token。\n- **把成本优化当一次性事** — 提供商价格按月变、便宜新模型按季度出。每季度重跑路由 benchmark，不然你会在 2026 年的硬件上付 2024 年的价。\n- **跳过 compression \u002F 路由的 eval 闸门** — 第三到第五层每次改动都必须挂在 eval 套件后面（golden set \u002F LLM-as-judge \u002F 生产影子流量）。「省 20% token、掉 4% 准确率」很少是你想要的交易。\n\n## 跟这些 pack 搭配\n\n这个 pack 是*成本*层。配 **Agent Observability + Tracing** 做调试侧 — 没法归因到 span 的花费、就没法优化。配 **LLM Eval & Guardrails** 让每次路由 \u002F 压缩改动都挂在质量门后面。配 **Vector DB + RAG** — 如果你的 prompt 是被检索上下文撑长的，最便宜的 token 就是不发的那条。",[109,112,115,118,121],{"q":110,"a":111},"这套实际能省多少？","10–50% 是诚实区间，重度依赖工作负载形态。下限（10–20%）是典型混合流量 API 只走第二到第三层能拿到的。上限（40–50%）只在两类特殊形态里出现：高缓存命中的聊天 \u002F FAQ \u002F 客服（Helicone 或 Cloudflare AI Gateway 抓到长尾重复）、大量窄任务（分类、抽取）用微调小模型替前沿。看见有人不说工作负载就报「70%+」的，不是在卖东西、就是有个特定场景（比如稳定机器人 95% 缓存命中）不能泛化。先测自己的基线。",{"q":113,"a":114},"语义缓存在私密 \u002F 多租户数据上安全吗？","只在你正确地划分 cache key 时才安全。默认 Helicone \u002F GPTCache \u002F Cloudflare 语义缓存按 prompt 内容做 key；如果用户 A 问「我的余额是多少」用户 B 问类似措辞，embedding 一对上就把 A 的答案给 B 了。一定要把 user_id \u002F tenant_id \u002F 鉴权上下文加进 cache key，绝对不要把响应里含 PII 的内容缓存。受监管行业（医疗 \u002F 金融）干脆把语义缓存从用户数据路径上撤掉，只缓存系统侧的东西（文档查询、tool 描述）。",{"q":116,"a":117},"微调什么时候才值得花工程时间？","三个条件同时成立才划算：（1）任务稳定 — 你不是还在每周改 prompt；（2）有 ≥1 万条标注样本、或者能从前沿模型 trace 里生成；（3）单这一个任务上前沿模型账单足够大、能 cover 1–2 个工程周和持续 eval 成本。经典赢家：PII 抽取、意图分类、半结构化文档抽取、领域内摘要。经典输家：「通用 agent 推理」、「创意写作」、任何 prompt \u002F 任务定义还在动的东西。Fireworks 无服务 LoRA on Llama 让权重可搬走 — 没有特殊理由就别选闭源微调。",{"q":119,"a":120},"OpenRouter 还是 LiteLLM，怎么挑？","OpenRouter 是托管答案：一个 API key、300+ 模型、自动 failover，给他们一点 markup、他们处理多提供商管道。LiteLLM 是自部署答案：你跑 proxy（或当 Python 库用）、自己带提供商 key、只付底层模型成本。要一张账单、快速上手、不想运维 proxy，选 OpenRouter。有直签提供商合同（规模上通常更便宜）、关心数据自主、要按项目成本看板、本来就在跑基建，选 LiteLLM。很多团队两个一起用：生产关键路径用 LiteLLM、原型和冷门模型走 OpenRouter。",{"q":122,"a":123},"最便宜的开始监控成本方式是什么？","从 TokenCost 开始 — 免费离线库、覆盖 400+ 模型、在任何脚本或 PR 里打印改动前后估算。生产侧最便宜的实时方案是 Cloudflare AI Gateway 免费档（缓存 + 分析 + 按模型分布、不装 SDK — 把 base_url 改到它就行），或者自部署开源版 Langfuse \u002F Helicone。如果已经在跑 LiteLLM Proxy，开内置的成本追踪就是阻力最小的路径 — 同一个 proxy、不加新服务。托管 Helicone \u002F Portkey \u002F Datadog LLM Observability 都不错，但通常要到每月 100 万调用以上才值得付费。",{"@context":125,"@type":126,"name":13,"description":127,"numberOfItems":128,"inLanguage":129,"itemListElement":130,"publisher":171},"https:\u002F\u002Fschema.org","ItemList","10 件资产、五层（观测 \u002F 缓存 \u002F 路由 \u002F 压缩 \u002F 微调），给 SaaS 和 agent 团队在不掉质量的前提下砍 10–50% LLM 花费。",10,"zh-CN",[131,136,140,144,148,152,156,160,164,168],{"@type":132,"position":133,"name":134,"url":135},"ListItem",1,"LiteLLM Proxy","https:\u002F\u002Ftokrepo.com\u002Fzh\u002Fworkflows\u002Flitellm-proxy-unified-gateway-for-100-llm-apis-0f113965",{"@type":132,"position":137,"name":138,"url":139},2,"OpenRouter 统一 API","https:\u002F\u002Ftokrepo.com\u002Fzh\u002Fworkflows\u002Fopenrouter-unified-api-for-300-llms-with-auto-failover-7bb772b3",{"@type":132,"position":141,"name":142,"url":143},3,"Manifest 智能 LLM 路由","https:\u002F\u002Ftokrepo.com\u002Fzh\u002Fworkflows\u002Fmanifest-smart-llm-router-cuts-costs-70-15266cba",{"@type":132,"position":145,"name":146,"url":147},4,"Portkey AI Gateway","https:\u002F\u002Ftokrepo.com\u002Fzh\u002Fworkflows\u002Fportkey-ai-gateway-route-250-llms-585d3a26",{"@type":132,"position":149,"name":150,"url":151},5,"Helicone 缓存","https:\u002F\u002Ftokrepo.com\u002Fzh\u002Fworkflows\u002Fhelicone-cache-cut-llm-spend-with-drop-in-response-caching-5d1acc2e",{"@type":132,"position":153,"name":154,"url":155},6,"Cloudflare AI Gateway","https:\u002F\u002Ftokrepo.com\u002Fzh\u002Fworkflows\u002Fcloudflare-ai-gateway-llm-proxy-cache-analytics-b1962c77",{"@type":132,"position":157,"name":158,"url":159},7,"LLMLingua","https:\u002F\u002Ftokrepo.com\u002Fzh\u002Fworkflows\u002Fllmlingua-compress-prompts-20x-minimal-loss-1510da0c",{"@type":132,"position":161,"name":162,"url":163},8,"LiteLLM 成本看板","https:\u002F\u002Ftokrepo.com\u002Fzh\u002Fworkflows\u002Flitellm-cost-tracking-per-project-llm-spend-dashboard-72b2e16c",{"@type":132,"position":165,"name":166,"url":167},9,"TokenCost 价格计算器","https:\u002F\u002Ftokrepo.com\u002Fzh\u002Fworkflows\u002Ftokencost-llm-price-calculator-400-models-43b26691",{"@type":132,"position":128,"name":169,"url":170},"Fireworks 无服务 LoRA 微调","https:\u002F\u002Ftokrepo.com\u002Fzh\u002Fworkflows\u002Ffireworks-fine-tuning-serverless-lora-on-llama-in-30-min-2f07f6a8",{"@type":172,"name":173,"url":174},"Organization","TokRepo","https:\u002F\u002Ftokrepo.com",[176,180,184],{"url":177,"anchor":178,"reason":179},"\u002Fzh\u002Fpacks\u002Fagent-observability-tracing","Agent 可观测 + 全链路追踪","调试层 — 归因不到 span 的花费、就没法优化",{"url":181,"anchor":182,"reason":183},"\u002Fzh\u002Fpacks\u002Fllm-observability","LLM 可观测性","运行时延迟 \u002F 成本 \u002F 版本趋势 dashboard，是这个 pack 优化工具的互补",{"url":185,"anchor":186,"reason":187},"\u002Fzh\u002Fpacks\u002Fml-engineer-rag-eval","ML 工程师 RAG + Eval 套件","把每次路由 \u002F 压缩改动挂在 eval 门后面，避免「便宜」悄悄变成「变差」",[189,193,197],{"claim":190,"source_name":191,"source_url":192},"LiteLLM proxy 把 100+ LLM 提供商映射到一个 OpenAI 兼容端点，自带成本追踪和 fallback","BerriAI\u002Flitellm","https:\u002F\u002Fgithub.com\u002FBerriAI\u002Flitellm",{"claim":194,"source_name":195,"source_url":196},"LLMLingua 可达 20× prompt 压缩比、性能损失最小","microsoft\u002FLLMLingua","https:\u002F\u002Fgithub.com\u002Fmicrosoft\u002FLLMLingua",{"claim":198,"source_name":199,"source_url":200},"Helicone 通过 proxy header 提供响应缓存和按调用成本分析","Helicone 文档","https:\u002F\u002Fdocs.helicone.ai\u002Ffeatures\u002Fadvanced-usage\u002Fcaching",905,"2026-05-23T12:00:00Z"]