QWED Protocol — Deterministic LLM Verification

Practical Notes

• Per README: pip install qwed installs core engines (Math, Code, Facts); qwed[full] adds SQL + logic/Z3 and more.
• Per README: qwed test runs 12 deterministic tests as a pre-production gate.
• Treat verification failures as product signals: log failures, route to fallback, and add tests for recurring mistakes.

Main

A safe integration pattern:

1. Wrap your model call with a verification step (or verify the final answer + key tool outputs).
2. Classify failures into: correctable (auto-fix), requires human review, and block.
3. Start with deterministic domains: math, SQL, code safety checks, logic constraints.
4. Add a CI gate: if your “golden tests” fail, don’t deploy the agent prompt/toolchain update.

Verification works best when it’s explicit: decide what must be true (constraints) instead of hoping the model “usually” gets it right.

FAQ

Q: Does verification reduce hallucinations? A: Per README: it focuses on making hallucinations irrelevant by catching/blocking unverified outputs before production.

Q: What’s a good first use case? A: Math and SQL checks are low-cost and high-impact. Add code-safety checks for tools that run commands.

Q: How strict should I be? A: Start strict on critical facts/tool actions, but allow fallbacks (re-ask model, different tool, or human review) to avoid dead ends.