ATLAS: Full-Stack Simulator for 3D-DRAM LLM Accelerators
Oct 13, 2026
This episode examines ATLAS, a simulator for 3D-DRAM-based LLM accelerators that the authors say was checked against real silicon. It explains why decoding is memory-bound on GPUs, and how hybrid bonding shrinks the data path from millimeters to micrometers. That cuts access energy by 77 to 83 percent compared with HBM. It covers how the design differs from processing-in-memory, and the trade-offs of small capacity and a heat limit of 85 degrees Celsius. The discussion shows why existing tools like Ramulator and earlier accelerator simulators fall short. It also goes through DRAM details such as logical rows, row-buffer locality, channel interleaving, and the tension block-paged KV cache adds. The episode ends on a skeptical question: how far conclusions can be trusted beyond the specific chip points that were validated.
Interactive Visualization: ATLAS: Full-Stack Simulator for 3D-DRAM LLM Accelerators
Sources:
1. A Full-Stack Performance Evaluation Infrastructure for 3D-DRAM-based LLM Accelerators — Cong Li, Chenhao Xue, Yi Ren, Xiping Dong, Yu Cheng, Yinbo Hu, Fujun Bai, Yixin Guo, Xiping Jiang, Qiang Wu, Zhi Yang, Zhe Cheng, Yuan Xie, Guangyu Sun, 2026
http://arxiv.org/abs/2604.08044v1
2. Tetris: Scalable and Efficient Neural Network Acceleration with 3D Memory — Mingyu Gao, Jing Pu, Xuan Yang, Mark Horowitz, Christos Kozyrakis, 2017
https://scholar.google.com/scholar?q=Tetris%3A+Scalable+and+Efficient+Neural+Network+Acceleration+with+3D+Memory
3. Neurocube: A Programmable Digital Neuromorphic Architecture with High-Density 3D Memory — Duckhwan Kim, Jaeha Kung, Sek Chai, Sudhakar Yalamanchili, Saibal Mukhopadhyay, 2016
https://scholar.google.com/scholar?q=Neurocube%3A+A+Programmable+Digital+Neuromorphic+Architecture+with+High-Density+3D+Memory
4. AttAcc! Unleashing the Power of PIM for Batched Transformer-based Generative Model Inference — Jaehyun Park, Jaewan Choi, Kwanhee Kyung, Michael Jaemin Kim, Yongsuk Kwon, Nam Sung Kim, Jung Ho Ahn, 2024
https://scholar.google.com/scholar?q=AttAcc%21+Unleashing+the+Power+of+PIM+for+Batched+Transformer-based+Generative+Model+Inference
5. A 1ynm 1.25V 8Gb, 16Gb/s/pin GDDR6-based Accelerator-in-Memory supporting 1TFLOPS MAC Operation and Various Activation Functions for Deep-Learning Applications — Seongju Lee, Sukhan Lee, et al. (SK hynix), 2022
https://scholar.google.com/scholar?q=A+1ynm+1.25V+8Gb%2C+16Gb%2Fs%2Fpin+GDDR6-based+Accelerator-in-Memory+supporting+1TFLOPS+MAC+Operation+and+Various+Activation+Functions+for+Deep-Learning+Applications
6. The gem5 Simulator — Nathan Binkert, Bradford Beckmann, Gabriel Black, Steven Reinhardt, et al., 2011
https://scholar.google.com/scholar?q=The+gem5+Simulator
7. Ramulator: A Fast and Extensible DRAM Simulator — Yoongu Kim, Weikun Yang, Onur Mutlu, 2016
https://scholar.google.com/scholar?q=Ramulator%3A+A+Fast+and+Extensible+DRAM+Simulator
8. Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling — Mahmoud Khairy, Zhesheng Shen, Tor M. Aamodt, Timothy G. Rogers, 2020
https://scholar.google.com/scholar?q=Accel-Sim%3A+An+Extensible+Simulation+Framework+for+Validated+GPU+Modeling
9. LLMCompass: Enabling Efficient Hardware Design for Large Language Model Inference — Hengrui Zhang, August Ning, Rohan Baskar Prabhakar, David Wentzlaff, 2024
https://scholar.google.com/scholar?q=LLMCompass%3A+Enabling+Efficient+Hardware+Design+for+Large+Language+Model+Inference
10. Halide: A Language and Compiler for Optimizing Parallelism, Locality, and Recomputation in Image Processing Pipelines — Jonathan Ragan-Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Frédo Durand, Saman Amarasinghe, 2013
https://scholar.google.com/scholar?q=Halide%3A+A+Language+and+Compiler+for+Optimizing+Parallelism%2C+Locality%2C+and+Recomputation+in+Image+Processing+Pipelines
11. TVM: An Automated End-to-End Optimizing Compiler for Deep Learning — Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Meghan Cowan, Haichen Shen, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, Arvind Krishnamurthy, 2018
https://scholar.google.com/scholar?q=TVM%3A+An+Automated+End-to-End+Optimizing+Compiler+for+Deep+Learning
12. Triton: An Intermediate Language and Compiler for Tiled Neural Network Computations — Philippe Tillet, H. T. Kung, David Cox, 2019
https://scholar.google.com/scholar?q=Triton%3A+An+Intermediate+Language+and+Compiler+for+Tiled+Neural+Network+Computations
13. Spatial: A Language and Compiler for Application Accelerators — David Koeplinger, Matthew Feldman, Raghu Prabhakar, Yaqi Zhang, Stefan Hadjis, Ruben Fiszel, Tian Zhao, Luigi Nardi, Ardavan Pedram, Christos Kozyrakis, Kunle Olukotun, 2018
https://scholar.google.com/scholar?q=Spatial%3A+A+Language+and+Compiler+for+Application+Accelerators
14. Stratum: System-Hardware Co-Design with Tiered Monolithic 3D-Stackable DRAM for Efficient MoE Serving — Yue Pan et al., 2025
https://scholar.google.com/scholar?q=Stratum%3A+System-Hardware+Co-Design+with+Tiered+Monolithic+3D-Stackable+DRAM+for+Efficient+MoE+Serving
15. H2-LLM: Hardware-Dataflow Co-Exploration for Heterogeneous Hybrid-Bonding-based Low-Batch LLM Inference — Cong Li, Yihan Yin, et al., 2025
https://scholar.google.com/scholar?q=H2-LLM%3A+Hardware-Dataflow+Co-Exploration+for+Heterogeneous+Hybrid-Bonding-based+Low-Batch+LLM+Inference
16. Ramulator 2.0: A Modern, Modular, and Extensible DRAM Simulator — Haocong Luo et al., 2023
https://scholar.google.com/scholar?q=Ramulator+2.0%3A+A+Modern%2C+Modular%2C+and+Extensible+DRAM+Simulator
17. Vidur: A Large-Scale Simulation Framework for LLM Inference — Amey Agrawal et al., 2024
https://scholar.google.com/scholar?q=Vidur%3A+A+Large-Scale+Simulation+Framework+for+LLM+Inference
18. Timeloop: A Systematic Approach to DNN Accelerator Evaluation — Angshuman Parashar et al., 2019
https://scholar.google.com/scholar?q=Timeloop%3A+A+Systematic+Approach+to+DNN+Accelerator+Evaluation
19. HotSpot 7.0 / Thermal Modeling for 3D ICs — Skadron et al.; Zhang et al., 2004-2020
https://scholar.google.com/scholar?q=HotSpot+7.0+%2F+Thermal+Modeling+for+3D+ICs
20. Efficient Memory Management for Large Language Model Serving with PagedAttention — Woosuk Kwon et al., 2023
https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention
21. Samsung HBM-PIM / SK hynix AiM / AttAcc / NeuPIMs — Kwon et al.; Park et al.; Heo et al., 2021-2024
https://scholar.google.com/scholar?q=Samsung+HBM-PIM+%2F+SK+hynix+AiM+%2F+AttAcc+%2F+NeuPIMs
22. Splitwise / DistServe / Mooncake (PD disaggregation) — Patel et al.; Zhong et al.; Qin et al., 2024
https://scholar.google.com/scholar?q=Splitwise+%2F+DistServe+%2F+Mooncake+%28PD+disaggregation%29
23. DeepSeek-V3 / MLA technical reports — DeepSeek-AI, 2024-2025
https://scholar.google.com/scholar?q=DeepSeek-V3+%2F+MLA+technical+reports
Interactive Visualization: ATLAS: Full-Stack Simulator for 3D-DRAM LLM Accelerators
1. A Full-Stack Performance Evaluation Infrastructure for 3D-DRAM-based LLM Accelerators — Cong Li, Chenhao Xue, Yi Ren, Xiping Dong, Yu Cheng, Yinbo Hu, Fujun Bai, Yixin Guo, Xiping Jiang, Qiang Wu, Zhi Yang, Zhe Cheng, Yuan Xie, Guangyu Sun, 2026
http://arxiv.org/abs/2604.08044v1
2. Tetris: Scalable and Efficient Neural Network Acceleration with 3D Memory — Mingyu Gao, Jing Pu, Xuan Yang, Mark Horowitz, Christos Kozyrakis, 2017
https://scholar.google.com/scholar?q=Tetris%3A+Scalable+and+Efficient+Neural+Network+Acceleration+with+3D+Memory
3. Neurocube: A Programmable Digital Neuromorphic Architecture with High-Density 3D Memory — Duckhwan Kim, Jaeha Kung, Sek Chai, Sudhakar Yalamanchili, Saibal Mukhopadhyay, 2016
https://scholar.google.com/scholar?q=Neurocube%3A+A+Programmable+Digital+Neuromorphic+Architecture+with+High-Density+3D+Memory
4. AttAcc! Unleashing the Power of PIM for Batched Transformer-based Generative Model Inference — Jaehyun Park, Jaewan Choi, Kwanhee Kyung, Michael Jaemin Kim, Yongsuk Kwon, Nam Sung Kim, Jung Ho Ahn, 2024
https://scholar.google.com/scholar?q=AttAcc%21+Unleashing+the+Power+of+PIM+for+Batched+Transformer-based+Generative+Model+Inference
5. A 1ynm 1.25V 8Gb, 16Gb/s/pin GDDR6-based Accelerator-in-Memory supporting 1TFLOPS MAC Operation and Various Activation Functions for Deep-Learning Applications — Seongju Lee, Sukhan Lee, et al. (SK hynix), 2022
https://scholar.google.com/scholar?q=A+1ynm+1.25V+8Gb%2C+16Gb%2Fs%2Fpin+GDDR6-based+Accelerator-in-Memory+supporting+1TFLOPS+MAC+Operation+and+Various+Activation+Functions+for+Deep-Learning+Applications
6. The gem5 Simulator — Nathan Binkert, Bradford Beckmann, Gabriel Black, Steven Reinhardt, et al., 2011
https://scholar.google.com/scholar?q=The+gem5+Simulator
7. Ramulator: A Fast and Extensible DRAM Simulator — Yoongu Kim, Weikun Yang, Onur Mutlu, 2016
https://scholar.google.com/scholar?q=Ramulator%3A+A+Fast+and+Extensible+DRAM+Simulator
8. Accel-Sim: An Extensible Simulation Framework for Validated GPU Modeling — Mahmoud Khairy, Zhesheng Shen, Tor M. Aamodt, Timothy G. Rogers, 2020
https://scholar.google.com/scholar?q=Accel-Sim%3A+An+Extensible+Simulation+Framework+for+Validated+GPU+Modeling
9. LLMCompass: Enabling Efficient Hardware Design for Large Language Model Inference — Hengrui Zhang, August Ning, Rohan Baskar Prabhakar, David Wentzlaff, 2024
https://scholar.google.com/scholar?q=LLMCompass%3A+Enabling+Efficient+Hardware+Design+for+Large+Language+Model+Inference
10. Halide: A Language and Compiler for Optimizing Parallelism, Locality, and Recomputation in Image Processing Pipelines — Jonathan Ragan-Kelley, Connelly Barnes, Andrew Adams, Sylvain Paris, Frédo Durand, Saman Amarasinghe, 2013
https://scholar.google.com/scholar?q=Halide%3A+A+Language+and+Compiler+for+Optimizing+Parallelism%2C+Locality%2C+and+Recomputation+in+Image+Processing+Pipelines
11. TVM: An Automated End-to-End Optimizing Compiler for Deep Learning — Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Meghan Cowan, Haichen Shen, Leyuan Wang, Yuwei Hu, Luis Ceze, Carlos Guestrin, Arvind Krishnamurthy, 2018
https://scholar.google.com/scholar?q=TVM%3A+An+Automated+End-to-End+Optimizing+Compiler+for+Deep+Learning
12. Triton: An Intermediate Language and Compiler for Tiled Neural Network Computations — Philippe Tillet, H. T. Kung, David Cox, 2019
https://scholar.google.com/scholar?q=Triton%3A+An+Intermediate+Language+and+Compiler+for+Tiled+Neural+Network+Computations
13. Spatial: A Language and Compiler for Application Accelerators — David Koeplinger, Matthew Feldman, Raghu Prabhakar, Yaqi Zhang, Stefan Hadjis, Ruben Fiszel, Tian Zhao, Luigi Nardi, Ardavan Pedram, Christos Kozyrakis, Kunle Olukotun, 2018
https://scholar.google.com/scholar?q=Spatial%3A+A+Language+and+Compiler+for+Application+Accelerators
14. Stratum: System-Hardware Co-Design with Tiered Monolithic 3D-Stackable DRAM for Efficient MoE Serving — Yue Pan et al., 2025
https://scholar.google.com/scholar?q=Stratum%3A+System-Hardware+Co-Design+with+Tiered+Monolithic+3D-Stackable+DRAM+for+Efficient+MoE+Serving
15. H2-LLM: Hardware-Dataflow Co-Exploration for Heterogeneous Hybrid-Bonding-based Low-Batch LLM Inference — Cong Li, Yihan Yin, et al., 2025
https://scholar.google.com/scholar?q=H2-LLM%3A+Hardware-Dataflow+Co-Exploration+for+Heterogeneous+Hybrid-Bonding-based+Low-Batch+LLM+Inference
16. Ramulator 2.0: A Modern, Modular, and Extensible DRAM Simulator — Haocong Luo et al., 2023
https://scholar.google.com/scholar?q=Ramulator+2.0%3A+A+Modern%2C+Modular%2C+and+Extensible+DRAM+Simulator
17. Vidur: A Large-Scale Simulation Framework for LLM Inference — Amey Agrawal et al., 2024
https://scholar.google.com/scholar?q=Vidur%3A+A+Large-Scale+Simulation+Framework+for+LLM+Inference
18. Timeloop: A Systematic Approach to DNN Accelerator Evaluation — Angshuman Parashar et al., 2019
https://scholar.google.com/scholar?q=Timeloop%3A+A+Systematic+Approach+to+DNN+Accelerator+Evaluation
19. HotSpot 7.0 / Thermal Modeling for 3D ICs — Skadron et al.; Zhang et al., 2004-2020
https://scholar.google.com/scholar?q=HotSpot+7.0+%2F+Thermal+Modeling+for+3D+ICs
20. Efficient Memory Management for Large Language Model Serving with PagedAttention — Woosuk Kwon et al., 2023
https://scholar.google.com/scholar?q=Efficient+Memory+Management+for+Large+Language+Model+Serving+with+PagedAttention
21. Samsung HBM-PIM / SK hynix AiM / AttAcc / NeuPIMs — Kwon et al.; Park et al.; Heo et al., 2021-2024
https://scholar.google.com/scholar?q=Samsung+HBM-PIM+%2F+SK+hynix+AiM+%2F+AttAcc+%2F+NeuPIMs
22. Splitwise / DistServe / Mooncake (PD disaggregation) — Patel et al.; Zhong et al.; Qin et al., 2024
https://scholar.google.com/scholar?q=Splitwise+%2F+DistServe+%2F+Mooncake+%28PD+disaggregation%29
23. DeepSeek-V3 / MLA technical reports — DeepSeek-AI, 2024-2025
https://scholar.google.com/scholar?q=DeepSeek-V3+%2F+MLA+technical+reports
Interactive Visualization: ATLAS: Full-Stack Simulator for 3D-DRAM LLM Accelerators