Giskard Checks — Evals and Safety Tests for LLM Agents

Practical Notes

• Quant: the current README requires Python 3.12+ and splits the project into modular packages such as giskard-checks.
• Quant: built-in checks explicitly include Groundedness, Conformity, regex matching, semantic similarity, and LLM-as-judge patterns.

Rollout pattern

• Start with one regression scenario and one groundedness scenario around a user-facing workflow.
• Add pass/fail gates only after you understand variance across repeated runs and model versions.
• Keep old v2-only capabilities separate if you still rely on Scan or RAGET; the README is explicit that those are legacy paths.

Watchouts

Do not assume every historical Giskard feature still exists in the same package line; v3 is a rewrite and the README explicitly separates planned versus available modules.

FAQ

Q: Is this the old all-in-one Giskard package? A: No. The README frames v3 as a modular rewrite and points to v2 only for legacy Scan and RAGET use cases.

Q: Why is it useful for agents? A: It gives scenario-based checks for outputs that can vary while still needing quality gates.

Q: What should I test first? A: Groundedness and one regression path tied to a real business workflow, not synthetic toy prompts.