| Bloom: A 176b-parameter open-access multilingual language model T Le Scao, A Fan, C Akiki, E Pavlick, S Iliĉ, D Hesslow, R Castagné, ... | 1350 | 2023 |
| {Zero-offload}: Democratizing {billion-scale} model training J Ren, S Rajbhandari, RY Aminabadi, O Ruwase, S Yang, M Zhang, D Li, ... 2021 USENIX Annual Technical Conference (USENIX ATC 21), 551-564, 2021 | 292 | 2021 |
| Memcached design on high performance RDMA capable interconnects J Jose, H Subramoni, M Luo, M Zhang, J Huang, M Wasi-ur-Rahman, ... 2011 International Conference on Parallel Processing, 743-752, 2011 | 263 | 2011 |
| Zeroquant: Efficient and affordable post-training quantization for large-scale transformers Z Yao, R Yazdani Aminabadi, M Zhang, X Wu, C Li, Y He Advances in Neural Information Processing Systems 35, 27168-27183, 2022 | 243 | 2022 |
| Deepspeed-inference: enabling efficient inference of transformer models at unprecedented scale RY Aminabadi, S Rajbhandari, AA Awan, C Li, D Li, E Zheng, O Ruwase, ... SC22: International Conference for High Performance Computing, Networking …, 2022 | 189 | 2022 |
| Deepspeed-moe: Advancing mixture-of-experts inference and training to power next-generation ai scale S Rajbhandari, C Li, Z Yao, M Zhang, RY Aminabadi, AA Awan, J Rasley, ... International conference on machine learning, 18332-18346, 2022 | 175 | 2022 |
| Learning intrinsic sparse structures within long short-term memory W Wen, Y He, S Rajbhandari, M Zhang, W Wang, F Liu, B Hu, Y Chen, ... arXiv preprint arXiv:1709.05027, 2017 | 151 | 2017 |
| OpenFold: Retraining AlphaFold2 yields new insights into its learning mechanisms and capacity for generalization G Ahdritz, N Bouatta, C Floristean, S Kadyan, Q Xia, W Gerecke, ... Nature Methods, 1-11, 2024 | 125 | 2024 |
| {DeepCPU}: Serving {RNN-based} Deep Learning Models 10x Faster M Zhang, S Rajbhandari, W Wang, Y He 2018 USENIX Annual Technical Conference (USENIX ATC 18), 951-965, 2018 | 116 | 2018 |
| Accelerating training of transformer-based language models with progressive layer dropping M Zhang, Y He Advances in neural information processing systems 33, 14011-14023, 2020 | 88 | 2020 |
| Valor: Efficient, software-only region conflict exceptions S Biswas, M Zhang, MD Bond, B Lucia ACM SIGPLAN Notices 50 (10), 241-259, 2015 | 72 | 2015 |
| Octet: Capturing and controlling cross-thread dependences efficiently MD Bond, M Kulkarni, M Cao, M Zhang, M Fathi Salmi, S Biswas, ... ACM SIGPLAN Notices 48 (10), 693-712, 2013 | 57 | 2013 |
| Hm-ann: Efficient billion-point nearest neighbor search on heterogeneous memory J Ren, M Zhang, D Li Advances in Neural Information Processing Systems 33, 10672-10684, 2020 | 56 | 2020 |
| Sentinel: Efficient tensor migration and allocation on heterogeneous memory systems for deep learning J Ren, J Luo, K Wu, M Zhang, H Jeon, D Li 2021 IEEE International Symposium on High-Performance Computer Architecture …, 2021 | 55 | 2021 |
| Improving approximate nearest neighbor search through learned adaptive early termination C Li, M Zhang, DG Andersen, Y He Proceedings of the 2020 ACM SIGMOD International Conference on Management of …, 2020 | 51 | 2020 |
| Navigating with graph representations for fast and scalable decoding of neural language models M Zhang, W Wang, X Liu, J Gao, Y He Advances in neural information processing systems 31, 2018 | 48 | 2018 |
| Model tells you what to discard: Adaptive kv cache compression for llms S Ge, Y Zhang, L Liu, M Zhang, J Han, J Gao arXiv preprint arXiv:2310.01801, 2023 | 45 | 2023 |
| Hybrid static–dynamic analysis for statically bounded region serializability A Sengupta, S Biswas, M Zhang, MD Bond, M Kulkarni ACM SIGPLAN Notices 50 (4), 561-575, 2015 | 45 | 2015 |
| Bamboo: Making preemptible instances resilient for affordable training of large {DNNs} J Thorpe, P Zhao, J Eyolfson, Y Qiao, Z Jia, M Zhang, R Netravali, GH Xu 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI …, 2023 | 40 | 2023 |
| Deepspeed-chat: Easy, fast and affordable rlhf training of chatgpt-like models at all scales Z Yao, RY Aminabadi, O Ruwase, S Rajbhandari, X Wu, AA Awan, ... arXiv preprint arXiv:2308.01320, 2023 | 39 | 2023 |
He YuxiongMicrosoft ResearchVerified email at microsoft.com
