| 题号 | 主题 | 文章 |
|——|——|——|
| 146 | Megatron-LM的代码结构？megatron/core的新设计… | 146-Megatron-LM的代码结构.md |
| 147 | DeepSpeed的ZeRO-Infinity和ZeRO-Offl… | [147-DeepSpeed的ZeRO-Infinity和ZeRO-Offload的代.md](/ai-infra-interview-305/12-%E8%AE%AD%E7%BB%83%E6%A1%86%E6%9E%B6/147-DeepSpeed%E7%9A%84ZeRO-Infinity%E5%92%8CZeRO-Offload%E7%9A%84%E4%BB%A3.html) |
| 148 | Colossal-AI的Gemini和PatrickStar的异… | [148-Colossal-AI的Gemini和PatrickStar的异同.md](/ai-infra-interview-305/12-%E8%AE%AD%E7%BB%83%E6%A1%86%E6%9E%B6/148-Colossal-AI%E7%9A%84Gemini%E5%92%8CPatrickStar%E7%9A%84%E5%BC%82%E5%90%8C.html) |
| 149 | FairScale的FullyShardedDataParallel… | [149-FairScale的FullyShardedDataParallel（FSD.md](/ai-infra-interview-305/12-%E8%AE%AD%E7%BB%83%E6%A1%86%E6%9E%B6/149-FairScale%E7%9A%84FullyShardedDataParallel%EF%BC%88FSD.html) |
| 150 | Hugging Face的Accelerate如何简化分布式训练？ | [150-Hugging-Face的Accelerate如何简化分布式训练.md](/ai-infra-interview-305/12-%E8%AE%AD%E7%BB%83%E6%A1%86%E6%9E%B6/150-Hugging-Face%E7%9A%84Accelerate%E5%A6%82%E4%BD%95%E7%AE%80%E5%8C%96%E5%88%86%E5%B8%83%E5%BC%8F%E8%AE%AD%E7%BB%83.html) |
| 151 | torch.distributed.fsdp的limit_all_… | 151-torch.distributed.fsdp的limit_all_gathe.md |
| 152 | 大模型训练的checkpoint格式？sharded checkp… | [152-大模型训练的checkpoint格式.md](/ai-infra-interview-305/12-%E8%AE%AD%E7%BB%83%E6%A1%86%E6%9E%B6/152-%E5%A4%A7%E6%A8%A1%E5%9E%8B%E8%AE%AD%E7%BB%83%E7%9A%84checkpoint%E6%A0%BC%E5%BC%8F.html) |
| 153 | 训练恢复的consistency问题？如何确保resume后的los… | [153-训练恢复的consistency问题.md](/ai-infra-interview-305/12-%E8%AE%AD%E7%BB%83%E6%A1%86%E6%9E%B6/153-%E8%AE%AD%E7%BB%83%E6%81%A2%E5%A4%8D%E7%9A%84consistency%E9%97%AE%E9%A2%98.html) |
| 154 | 大模型训练的data pipeline优化？WebDataset… | [154-大模型训练的data-pipeline优化.md](/ai-infra-interview-305/12-%E8%AE%AD%E7%BB%83%E6%A1%86%E6%9E%B6/154-%E5%A4%A7%E6%A8%A1%E5%9E%8B%E8%AE%AD%E7%BB%83%E7%9A%84data-pipeline%E4%BC%98%E5%8C%96.html) |
| 155 | 多模态大模型的训练基础设施挑战？图文数据的加载？ | [155-多模态大模型的训练基础设施挑战.md](/ai-infra-interview-305/12-%E8%AE%AD%E7%BB%83%E6%A1%86%E6%9E%B6/155-%E5%A4%9A%E6%A8%A1%E6%80%81%E5%A4%A7%E6%A8%A1%E5%9E%8B%E7%9A%84%E8%AE%AD%E7%BB%83%E5%9F%BA%E7%A1%80%E8%AE%BE%E6%96%BD%E6%8C%91%E6%88%98.html) |
| 156 | RLHF训练的infra设计？PPO的分布式实现？ | [156-RLHF训练的infra设计.md](/ai-infra-interview-305/12-%E8%AE%AD%E7%BB%83%E6%A1%86%E6%9E%B6/156-RLHF%E8%AE%AD%E7%BB%83%E7%9A%84infra%E8%AE%BE%E8%AE%A1.html) |
| 157 | MoE模型的all-to-all通信优化？Tutel、Fast… | 157-MoE模型的all-to-all通信优化.md |
| 158 | 长上下文训练（100K+ tokens）的显存优化？Ring Atte… | [158-长上下文训练（100K+-tokens）的显存优化.md](/ai-infra-interview-305/12-%E8%AE%AD%E7%BB%83%E6%A1%86%E6%9E%B6/158-%E9%95%BF%E4%B8%8A%E4%B8%8B%E6%96%87%E8%AE%AD%E7%BB%83%EF%BC%88100K+-tokens%EF%BC%89%E7%9A%84%E6%98%BE%E5%AD%98%E4%BC%98%E5%8C%96.html) |
| 159 | 训练任务的fault tolerance？elastic trai… | 159-训练任务的fault-tolerance.md |
| 160 | 大模型训练的experiment tracking？`Weights… | 160-大模型训练的experiment-tracking.md |