Kioxia's latest 5TB flash module boasts a speedy 64 GB/s, bringing NAND closer to the memory bus for AI-focused GPUs, as demonstrated in the HBF prototype, which retains the familiar SSD physical structure.

Kioxia's High-Bandwidth Flash (HBF) Module Revolutionises Data Centre Storage

Kioxia, a leading manufacturer of flash memory, has unveiled its latest innovation - the High-Bandwidth Flash (HBF) module. This new technology promises to revolutionise data centre storage, offering efficient access to large AI datasets while significantly reducing power consumption.

The HBF module adapts the concept of High Bandwidth Memory (HBM) to NAND flash, offering 8-16 times the capacity of DRAM-based HBM. By using daisy-chained controllers, the module allows for adding more modules without consuming additional bandwidth, and performance scales linearly with capacity. On a GB/s per Watt basis, the HBF module is significantly more efficient than traditional SSDs.

Data is passed along in series in the HBF system, with each link pushing 128 Gbps using PAM4 signaling. This signaling doubles the data rate per symbol compared to traditional NRZ signaling, but it's more sensitive to noise and bit errors. To maintain signal integrity, Kioxia relies on equalization, error correction, and stronger pre-emphasis.

Kioxia's vision for the HBF module suggests that flash is moving up the hierarchy, potentially competing for bandwidth bragging rights alongside GPUs in the next generation of data centres. The bandwidth of the HBF module is over 4 times faster than the fastest PCIe 5.0 drives shipping today.

However, latency is the main tradeoff with the HBF system. NAND flash still accesses data in tens of microseconds, which is slower than HBM memory. To mitigate this issue for sequential workloads, aggressive prefetching and controller-level caching are employed.

The company that developed the prototype of the 5TB high-speed flash memory module with a bandwidth of 64 GB/s, especially suitable for GPUs, is Kioxia. The prototype status of the HBF module leaves unanswered questions about how it handles mixed random workloads, how ECC scaling impacts latency, and what real-world throughput looks like under AI training conditions.

Kioxia has announced fab expansions in Japan, driven by an expectation that flash demand will nearly triple by 2028. The HBF module draws under 40W per module, compared to traditional Gen5 SSDs that can draw up to 15W for ~14 GB/s, making it dramatically more efficient.

In a hyperscale rack, a few hundred drives can easily consume multiple kilowatts. AI datacentres, already ballooning in power budgets, need every watt saved at the storage layer. The HBF module delivers 5 TB of capacity and 64 GB/s of sustained bandwidth over PCIe 6.0.

Kioxia has also been experimenting with long-reach PCIe SSDs and GPU peer-to-peer flash links, including research with Nvidia into XL-Flash drives. These advancements position Kioxia at the forefront of flash memory technology, poised to reshape the landscape of data centre storage efficiency.

Kioxia's latest 5TB flash module boasts a speedy 64 GB/s, bringing NAND closer to the memory bus for AI-focused GPUs, as demonstrated in the HBF prototype, which retains the familiar SSD physical structure.