Marvel develops customized HBM memory solution: interface reduction, compatible with XPU
Dec 13,2024
On Analyst Day 2024, Marvel announced the launch of a customized high bandwidth memory (CHBM) solution for custom XPUs aimed at AI applications. This solution is developed in collaboration with leading memory manufacturers, and CHBM promises to optimize performance, power consumption, memory capacity, chip size, and cost for specific XPU designs. CHBM will be compatible with Marvel's custom XPU and will not be part of the HBM standard defined by JEDEC at least initially.
Marvel's customized HBM solution allows for customized interfaces and stacks for specific applications, although the company has not disclosed any details yet. One of Marvel's goals is to reduce the space occupied by industry standard HBM interfaces within processors, thereby freeing up areas available for computation and functionality. The company claims that with its proprietary inter chip I/O, it can not only package 25% more logic in its customized XPU, but may also install 33% more CHBM memory packages adjacent to the computing Chiplet, thereby increasing the amount of DRAM available to the processor. In addition, the company expects a reduction of up to 70% in memory interface power consumption.
Due to Marvel's CHBM not relying on JEDEC specified standards, it will require a new controller and customizable physical interfaces, new inter chip interfaces, and a completely revamped HBM base chip in terms of hardware. The bandwidth of the HBM interface between the new Marvel chips is 20Tbps/mm (2.5TB/s per millimeter), which is significantly higher than the current 5Tbps/mm (625GB/s per millimeter) provided by HBM. Over time, Marvel envisions a bandwidth of 50Tbps/mm (6.25TB/s per millimeter) without buffer memory.
Marvel did not specify the width of its CHBM interface. Apart from stating that it enhances XPU by serializing and accelerating the I/O interface between its internal AI computing accelerator silicon chip and HBM base chip, Marvel did not disclose many details of its customized HBM solution, which to some extent implies a narrower interface width compared to industry standard HBM3E or HBM4 solutions. However, it appears that the CHBM solution will be customizable.
Customizing HBM for specific performance, power consumption, and total cost of ownership to enhance XPU is the latest step in a new paradigm of AI accelerator design and delivery, "said Will Chu, Senior Vice President and General Manager of the Customization, Computing, and Storage Group at Marvel. We are very grateful for collaborating with leading memory designers to accelerate this revolution and help cloud data center operators continue to expand their XPUs and infrastructure to embrace the AI era
The collaboration with Micron, Samsung, and SK Hynix is crucial for the successful implementation of the Marvel CHBM, as it lays the foundation for the relatively widespread use of customized HBM.
Marvel's customized HBM solution allows for customized interfaces and stacks for specific applications, although the company has not disclosed any details yet. One of Marvel's goals is to reduce the space occupied by industry standard HBM interfaces within processors, thereby freeing up areas available for computation and functionality. The company claims that with its proprietary inter chip I/O, it can not only package 25% more logic in its customized XPU, but may also install 33% more CHBM memory packages adjacent to the computing Chiplet, thereby increasing the amount of DRAM available to the processor. In addition, the company expects a reduction of up to 70% in memory interface power consumption.
Due to Marvel's CHBM not relying on JEDEC specified standards, it will require a new controller and customizable physical interfaces, new inter chip interfaces, and a completely revamped HBM base chip in terms of hardware. The bandwidth of the HBM interface between the new Marvel chips is 20Tbps/mm (2.5TB/s per millimeter), which is significantly higher than the current 5Tbps/mm (625GB/s per millimeter) provided by HBM. Over time, Marvel envisions a bandwidth of 50Tbps/mm (6.25TB/s per millimeter) without buffer memory.
Marvel did not specify the width of its CHBM interface. Apart from stating that it enhances XPU by serializing and accelerating the I/O interface between its internal AI computing accelerator silicon chip and HBM base chip, Marvel did not disclose many details of its customized HBM solution, which to some extent implies a narrower interface width compared to industry standard HBM3E or HBM4 solutions. However, it appears that the CHBM solution will be customizable.
Customizing HBM for specific performance, power consumption, and total cost of ownership to enhance XPU is the latest step in a new paradigm of AI accelerator design and delivery, "said Will Chu, Senior Vice President and General Manager of the Customization, Computing, and Storage Group at Marvel. We are very grateful for collaborating with leading memory designers to accelerate this revolution and help cloud data center operators continue to expand their XPUs and infrastructure to embrace the AI era
The collaboration with Micron, Samsung, and SK Hynix is crucial for the successful implementation of the Marvel CHBM, as it lays the foundation for the relatively widespread use of customized HBM.