Designs and operates NextModel's production LLM gateway: the OpenAI-compatible routing layer, dynamic model selection, and cost-aware load balancing. Years of production incidents around timeouts, retries, and billing consistency at concurrency. Previously contributed to open-source proxies and compatibility layers; optimizes for production-grade, not demo-grade. Everything written here is based on the configuration and failure experience of the gateway that is actually running.
负责 NextModel 生产环境 LLM 网关的设计与运维。主导实现 OpenAI-compatible 路由层、动态模型选择与成本感知负载均衡。长期处理高并发场景下的超时、重试与计费一致性问题。此前参与过多个开源代理与兼容层项目，注重「生产可用」而非 demo 级实现。所有网关相关文章均基于当前线上运行的真实配置与故障经验。
llm-gateway · openai-compatible · routing & billing
Articles
- LLM Gateway: Compare Options and Alternatives
- OpenRouter alternatives for developers who need AI API cost control
- GPT-5 is listed on NextModel: price, context, and the id you call
- DeepSeek V4 Flash CN on NextModel: 1 input, unpublished context
- Claude Sonnet 5 is listed on NextModel: 1M context, cache scores unpublished
- Gemini 3.5 Flash is listed on NextModel: 1,048,576 context, no cache row
- Kimi K3 is listed on NextModel: 1,048,576 context, cache scores unpublished
- GPT-5 Nano is listed on NextModel: 0.05 input, 400k context
- Claude Opus 5 is listed on NextModel: 1M context, cache scores unpublished
- Gemini 3 Flash Preview is listed on NextModel: 1,048,576 context
- Kimi K2.5 CN is listed on NextModel: 0.58 input, unpublished context
- GPT-5 Pro is listed on NextModel: 15 input, 400k context
- Claude Haiku 4.5 is listed on NextModel: 1 input, 200k context
- Gemini 3.1 Flash Lite is listed on NextModel: 0.25 input, 1,048,576 context
- DeepSeek V4 PRO CN is listed on NextModel: 1.65 input, unpublished context
- GPT-5.4 is listed on NextModel: 2.50 input, 1,050,000 context
- GPT-5.5 is listed on NextModel: 5 input, 1,050,000 context
- GLM 5.1 CN is listed on NextModel: 0.83 input, unpublished context
- GLM 5.2 CN is listed on NextModel: 1.18 input, unpublished context
- Claude Sonnet 4.6 is listed on NextModel: 1M context, cache scores unpublished
- Gemini 2.5 Flash is listed on NextModel: 0.30 input, 1,048,576 context
- Doubao Seed 2.0 Lite is listed on NextModel: 0.09 input, unpublished context
- DeepSeek V3.2 CN is listed on NextModel: 0.29 input, unpublished context
- Claude Sonnet 4.5 is listed on NextModel: 1M context, cache scores unpublished
- Gemini 3.5 Flash Lite is listed on NextModel: 0.30 input, 1,048,576 context
- GPT-4o-mini is listed on NextModel: 0.15 input, 128k context
- Doubao Seed 2.1 Turbo is listed on NextModel: 0.45 input, unpublished context
- GPT-4o is listed on NextModel: 2.50 input, 128k context
- GPT-4.1 is listed on NextModel: 2 input, 1,047,576 context
- GPT-5.1 is listed on NextModel: 1.25 input, 400k context
- Kimi K2.6 CN is listed on NextModel: 0.9 input, unpublished context
- GPT-6 Astra on NextModel: 1M context, computer use, one API
- GPT-6 Astra vs GPT-5.6 Sol: when the 2x upgrade pays for itself
- GPT-6 Astra vs Claude Fable 5.1: reading the launch-week debate
- Migrating to GPT-6 Astra: model ids, responses API, and a rollback plan