| 题号 | 主题 | 文章 |
|——|——|——|
| 236 | 设计一个支持10万QPS的大模型推理服务？ | 236-设计一个支持10万QPS的大模型推理服务.md |
| 237 | 推理服务的auto-scaling策略？HPA、VPA、K… | [237-推理服务的auto-scaling策略.md](/ai-infra-interview-305/18-%E6%8E%A8%E7%90%86%E5%B9%B3%E5%8F%B0%E8%AE%BE%E8%AE%A1/237-%E6%8E%A8%E7%90%86%E6%9C%8D%E5%8A%A1%E7%9A%84auto-scaling%E7%AD%96%E7%95%A5.html) |
| 238 | 多模型multi-tenancy的GPU共享？MPS vs M… | 238-多模型multi-tenancy的GPU共享.md |
| 239 | 推理请求的routing策略？least loaded、con… | [239-推理请求的routing策略.md](/ai-infra-interview-305/18-%E6%8E%A8%E7%90%86%E5%B9%B3%E5%8F%B0%E8%AE%BE%E8%AE%A1/239-%E6%8E%A8%E7%90%86%E8%AF%B7%E6%B1%82%E7%9A%84routing%E7%AD%96%E7%95%A5.html) |
| 240 | 边缘推理的model caching策略？CDN for mod… | [240-边缘推理的model-caching策略.md](/ai-infra-interview-305/18-%E6%8E%A8%E7%90%86%E5%B9%B3%E5%8F%B0%E8%AE%BE%E8%AE%A1/240-%E8%BE%B9%E7%BC%98%E6%8E%A8%E7%90%86%E7%9A%84model-caching%E7%AD%96%E7%95%A5.html) |
| 241 | 推理服务的A/B testing和canary deploymen… | 241-推理服务的A-B-testing和canary-deployment.md |
| 242 | 长文本生成的streaming实现？Server-Sent Eve… | [242-长文本生成的streaming实现.md](/ai-infra-interview-305/18-%E6%8E%A8%E7%90%86%E5%B9%B3%E5%8F%B0%E8%AE%BE%E8%AE%A1/242-%E9%95%BF%E6%96%87%E6%9C%AC%E7%94%9F%E6%88%90%E7%9A%84streaming%E5%AE%9E%E7%8E%B0.html) |
| 243 | 推理结果的caching策略？prompt级别的deduplic… | [243-推理结果的caching策略.md](/ai-infra-interview-305/18-%E6%8E%A8%E7%90%86%E5%B9%B3%E5%8F%B0%E8%AE%BE%E8%AE%A1/243-%E6%8E%A8%E7%90%86%E7%BB%93%E6%9E%9C%E7%9A%84caching%E7%AD%96%E7%95%A5.html) |
| 244 | 多模态推理的pipeline设计？图文、视频？ | [244-多模态推理的pipeline设计.md](/ai-infra-interview-305/18-%E6%8E%A8%E7%90%86%E5%B9%B3%E5%8F%B0%E8%AE%BE%E8%AE%A1/244-%E5%A4%9A%E6%A8%A1%E6%80%81%E6%8E%A8%E7%90%86%E7%9A%84pipeline%E8%AE%BE%E8%AE%A1.html) |
| 245 | 推理服务的safety guardrails？内容过滤、速率限制？ | [245-推理服务的safety-guardrails.md](/ai-infra-interview-305/18-%E6%8E%A8%E7%90%86%E5%B9%B3%E5%8F%B0%E8%AE%BE%E8%AE%A1/245-%E6%8E%A8%E7%90%86%E6%9C%8D%E5%8A%A1%E7%9A%84safety-guardrails.html) |
| 246 | 模型fine-tuning as a service的架构？ | [246-模型fine-tuning-as-a-service的架构.md](/ai-infra-interview-305/18-%E6%8E%A8%E7%90%86%E5%B9%B3%E5%8F%B0%E8%AE%BE%E8%AE%A1/246-%E6%A8%A1%E5%9E%8Bfine-tuning-as-a-service%E7%9A%84%E6%9E%B6%E6%9E%84.html) |
| 247 | 推理服务的cost optimization？spot insta… | 247-推理服务的cost-optimization.md |
| 248 | 模型on-device部署的infra支持？Core ML、T… | [248-模型on-device部署的infra支持.md](/ai-infra-interview-305/18-%E6%8E%A8%E7%90%86%E5%B9%B3%E5%8F%B0%E8%AE%BE%E8%AE%A1/248-%E6%A8%A1%E5%9E%8Bon-device%E9%83%A8%E7%BD%B2%E7%9A%84infra%E6%94%AF%E6%8C%81.html) |
| 249 | 推理服务的latency SLO保证？p99 < 100ms如何… | [249-推理服务的latency-SLO保证.md](/ai-infra-interview-305/18-%E6%8E%A8%E7%90%86%E5%B9%B3%E5%8F%B0%E8%AE%BE%E8%AE%A1/249-%E6%8E%A8%E7%90%86%E6%9C%8D%E5%8A%A1%E7%9A%84latency-SLO%E4%BF%9D%E8%AF%81.html) |
| 250 | 推理请求的admission control？防止过载？ | [250-推理请求的admission-control.md](/ai-infra-interview-305/18-%E6%8E%A8%E7%90%86%E5%B9%B3%E5%8F%B0%E8%AE%BE%E8%AE%A1/250-%E6%8E%A8%E7%90%86%E8%AF%B7%E6%B1%82%E7%9A%84admission-control.html) |
| 251 | 模型ensemble和cascade的架构设计？ | [251-模型ensemble和cascade的架构设计.md](/ai-infra-interview-305/18-%E6%8E%A8%E7%90%86%E5%B9%B3%E5%8F%B0%E8%AE%BE%E8%AE%A1/251-%E6%A8%A1%E5%9E%8Bensemble%E5%92%8Ccascade%E7%9A%84%E6%9E%B6%E6%9E%84%E8%AE%BE%E8%AE%A1.html) |
| 252 | 推理服务的debuggability？request tracin… | 252-推理服务的debuggability.md |
| 253 | 实时推理和离线推理的统一架构？ | 253-实时推理和离线推理的统一架构.md |
| 254 | 推理服务的security？模型加密、API认证？ | 254-推理服务的security.md |
| 255 | 推理平台的carbon footprint优化？绿色AI？ | 255-推理平台的carbon-footprint优化.md |