Doubling GPT-3 Inference Speed and Eliminating Silicon Furnaces: Vertical Memory Architectures V-Die and MOSAIC Revolutionize HBM Cooling and Bandwidth for AI Accelerators

Future AI accelerators may soon feature memory chips literally standing on their edges. Researchers from South Korea and Japan have proposed rotating DRAM dies vertically to boost both the speed and capacity of HBM without turning multilayer stacks into poorly cooled silicon furnaces.

Two new architectures, V-Die and MOSAIC, were presented in June 2026 at the IEEE Symposium on VLSI Circuits. Both teams abandoned the conventional approach of stacking dynamic memory dies horizontally on top of a base die and connecting them through TSV (through-silicon via) channels. Instead, the engineers assemble the package first and then rotate the entire structure so that individual dies stand vertically, functioning like the fins of a radiator.

Current HBM places multiple memory dies above a base die and links the layers with short, wide TSV buses that deliver several terabytes per second of bandwidth, making it the foundation of powerful AI accelerators. However, increasing the number of layers makes heat removal increasingly difficult. Lower dies become extremely hot, and heat must travel through silicon, solder, insulating materials, and additional package layers. The TSVs themselves also consume valuable die area that could otherwise be used for memory cells.

The Korean V-Die architecture removes TSVs from the memory dies altogether. Each vertical die receives its own input/output lines along the bottom edge and connects directly to the substrate. According to the researchers’ calculations, the design provides four times more connections than HBM4 and cuts data read time by 37 percent. Microfluidic channels running between the dies maintain temperatures around 45 °C, whereas dense conventional HBM packages can exceed 80 °C under heavy load.

Simulations of a 16-die system demonstrated significant performance gains. When processing GPT-3-scale workloads, the V-Die architecture achieved 540 tokens per second compared with 296 tokens per second for an equivalent-capacity HBM4 configuration. First-token latency dropped by 32 percent, or roughly 24 milliseconds. These results are currently simulation-based; the research team is now preparing physical prototypes to validate thermal and electrical characteristics.

The Japanese MOSAIC project addresses a different challenge of vertical assembly: even small thickness variations among dozens of dies can misalign contact pads. Engineers from the University of Tokyo, Tohoku University, and RIKEN proposed transmitting data without direct metal contact. Miniature coils on the dies and substrate exchange signals through electromagnetic induction, eliminating the need for perfect physical alignment during assembly.

The experimental MOSAIC interface reached speeds of up to 4 Gbit/s per channel. Researchers expect to double the memory volume of HBM4 by placing the vertical block directly above the GPU while increasing maximum temperature by only about one degree. One configuration accommodates 98 dies and 294 GB of memory; further thinning of the dies could theoretically push capacity to 882 GB.

Neither V-Die nor MOSAIC is yet ready to replace production HBM. The Korean architecture exists primarily as calculations, while the Japanese prototype must still demonstrate acceptable cost, reliability, and high manufacturing yield. Nevertheless, both developments outline a promising route around the thermal barrier that increasingly constrains memory scaling for AI accelerators. Instead of endlessly stacking dies higher, engineers are proposing to rotate the entire structure and turn the memory dies themselves into an integrated heat-dissipation system.