Five years of LLM caching and inference-system benchmarking. Led the design and maintenance of CacheSafety Bench's core metrics (SH/BH/TR) and ran multiple cross-model comparison rounds; previously built performance baselines on an inference-framework team. Advocates reproducible, auditable evaluation — every published dataset ships with full reproduction scripts. The benchmarking work also produced the real cost data behind NextModel's cost-estimation and model-selection writing.
专注大模型缓存与推理系统评测五年。主导设计并维护 CacheSafety Bench 的核心指标（SH/BH/TR），完成多轮跨模型横向对比实验。此前在推理框架团队负责性能基线建设，熟悉缓存命中率、语义相似度与端到端延迟的测量方法论。主张「可复现、可审计」的评测标准，所有公开数据均附带完整复现脚本。在评测工作中同步积累了大量真实成本数据与模型选型经验，因此兼写成本估算与 API 选型相关内容。
CacheSafety Bench · semantic caching · bad-hit-rate analysis · 成本估算 · 模型选型
Articles
- Semantic Caching for LLM APIs: A Practical Guide
- Bad Hit Rate: the metric every LLM cache needs
- Doubao Seed 2.0 Mini API guide: pricing, use cases, and OpenAI-compatible calls
- How to estimate AI API cost before you ship
- GPT-5 Mini cache safety review: 80% Safe Hit Rate on NextModel
- GLM-5 CN cache safety review: family data only, CN still unmeasured
- GPT-6 Astra pricing explained: the 272K cliff, cache, and worked examples
- 1M context in production: what GPT-6 Astra's retrieval numbers actually buy you
- GPT-6 Astra on a budget: cache, routing, and paying frontier rates only when it matters