| 题号 | 主题 | 文章 |
|——|——|——|
| 196 | FlashAttention的IO-aware优化原理？til… | [196-FlashAttention的IO-aware优化原理.md](/ai-infra-interview-305/16-%E4%BC%98%E5%8C%96%E6%8A%80%E6%9C%AF/196-FlashAttention%E7%9A%84IO-aware%E4%BC%98%E5%8C%96%E5%8E%9F%E7%90%86.html) |
| 197 | FlashAttention-2和FlashAttention的… | [197-FlashAttention-2和FlashAttention的改进点.md](/ai-infra-interview-305/16-%E4%BC%98%E5%8C%96%E6%8A%80%E6%9C%AF/197-FlashAttention-2%E5%92%8CFlashAttention%E7%9A%84%E6%94%B9%E8%BF%9B%E7%82%B9.html) |
| 198 | xFormers的memory_efficient_attenti… | 198-xFormers的memory_efficient_attention使用.md |
| 199 | cuDNN的fused attention如何调用？ | 199-cuDNN的fused-attention如何调用.md |
| 200 | 算子融合的边界判断？融合后register pressure过高？ | 200-算子融合的边界判断.md |
| 201 | 内存带宽bound vs 计算bound的判断？arithmetic … | [201-内存带宽bound-vs-计算bound的判断.md](/ai-infra-interview-305/16-%E4%BC%98%E5%8C%96%E6%8A%80%E6%9C%AF/201-%E5%86%85%E5%AD%98%E5%B8%A6%E5%AE%BDbound-vs-%E8%AE%A1%E7%AE%97bound%E7%9A%84%E5%88%A4%E6%96%AD.html) |
| 202 | kernel fusion的手动实现 vs 编译器自动生成？ | [202-kernel-fusion的手动实现-vs-编译器自动生成.md](/ai-infra-interview-305/16-%E4%BC%98%E5%8C%96%E6%8A%80%E6%9C%AF/202-kernel-fusion%E7%9A%84%E6%89%8B%E5%8A%A8%E5%AE%9E%E7%8E%B0-vs-%E7%BC%96%E8%AF%91%E5%99%A8%E8%87%AA%E5%8A%A8%E7%94%9F%E6%88%90.html) |
| 203 | CUDA Graph的捕获和重放？torch.cuda.make_… | 203-CUDA-Graph的捕获和重放.md |
| 204 | 动态shape的优化困境？torch.compile的dynami… | [204-动态shape的优化困境.md](/ai-infra-interview-305/16-%E4%BC%98%E5%8C%96%E6%8A%80%E6%9C%AF/204-%E5%8A%A8%E6%80%81shape%E7%9A%84%E4%BC%98%E5%8C%96%E5%9B%B0%E5%A2%83.html) |
| 205 | torch.backends.cudnn.benchmark的作用和… | [205-torch.backends.cudnn.benchmark的作用和副作用.md](/ai-infra-interview-305/16-%E4%BC%98%E5%8C%96%E6%8A%80%E6%9C%AF/205-torch.backends.cudnn.benchmark%E7%9A%84%E4%BD%9C%E7%94%A8%E5%92%8C%E5%89%AF%E4%BD%9C%E7%94%A8.html) |
| 206 | CPU offload的pin_memory和non_blocki… | 206-CPU-offload的pin_memory和non_blocking.md |
| 207 | 数据预处理的multi-processing优化？num_work… | [207-数据预处理的multi-processing优化.md](/ai-infra-interview-305/16-%E4%BC%98%E5%8C%96%E6%8A%80%E6%9C%AF/207-%E6%95%B0%E6%8D%AE%E9%A2%84%E5%A4%84%E7%90%86%E7%9A%84multi-processing%E4%BC%98%E5%8C%96.html) |
| 208 | DALI的GPU decoding和augmentation… | [208-DALI的GPU-decoding和augmentation.md](/ai-infra-interview-305/16-%E4%BC%98%E5%8C%96%E6%8A%80%E6%9C%AF/208-DALI%E7%9A%84GPU-decoding%E5%92%8Caugmentation.html) |
| 209 | 混合精度训练的numerical stability问题？ | [209-混合精度训练的numerical-stability问题.md](/ai-infra-interview-305/16-%E4%BC%98%E5%8C%96%E6%8A%80%E6%9C%AF/209-%E6%B7%B7%E5%90%88%E7%B2%BE%E5%BA%A6%E8%AE%AD%E7%BB%83%E7%9A%84numerical-stability%E9%97%AE%E9%A2%98.html) |
| 210 | gradient accumulation的effective b… | 210-gradient-accumulation的effective-batch-.md |
| 211 | 大batch训练的learning rate scaling规则？ | 211-大batch训练的learning-rate-scaling规则.md |
| 212 | LARS、LAMB优化器在大batch场景的应用？ | 212-LARS、LAMB优化器在大batch场景的应用.md |
| 213 | 模型并行的communication hiding技术？ | 213-模型并行的communication-hiding技术.md |
| 214 | pipeline bubble的数学分析和优化？ | 214-pipeline-bubble的数学分析和优化.md |
| 215 | 稀疏attention的优化？Sparse Transformer、… | 215-稀疏attention的优化.md |