Bad Hit Rate: the metric every LLM cache needs

Direct answer

Why Safe Hit Rate and Bad Hit Rate matter more than raw cache hit rate when evaluating LLM response reuse. This guide is written for UK product and platform teams comparing model quality, spend, routing policy, and production rollout risk.

Why hit rate is misleading

A cache can look efficient on paper while still making the model look wrong. Bad Hit Rate captures the failures users actually notice: stale facts, broken formatting, wrong quantities, and semantic traps.

What to measure instead

Teams should measure Safe Hit Rate, Bad Hit Rate, Cost Saved / 1K Requests, and Semantic Trap Failure Rate before routing production traffic through a reuse layer.

• Safe Hit Rate measures invisible reuse.
• Bad Hit Rate measures the safety line.
• Semantic traps reveal whether similar prompts still need fresh answers.

How CacheSafety Bench fits

CacheSafety Bench is an open benchmark for measuring safe LLM response reuse locally first, with optional hosted evaluation on NextModel for larger replay jobs.