Bad Hit Rate: the metric every LLM cache needs

Why hit rate is misleading

A cache can look efficient on paper while still making the model look wrong. Bad Hit Rate captures the failures users actually notice: stale facts, broken formatting, wrong quantities, and semantic traps.

What to measure instead

Teams should measure Safe Hit Rate, Bad Hit Rate, Cost Saved / 1K Requests, and Semantic Trap Failure Rate before routing production traffic through a reuse layer.

• Safe Hit Rate measures invisible reuse.
• Bad Hit Rate measures the safety line.
• Semantic traps reveal whether similar prompts still need fresh answers.

How CacheSafety Bench fits

CacheSafety Bench is an open benchmark for measuring safe LLM response reuse locally first, with optional hosted evaluation on NextModel for larger replay jobs.