We need a physical architectural change if we really want to improve this. You can only physically wire so much memory next to a chip in a 2 dimensional (flat)

We need a physical architectural change if we really want to improve this. You can only physically wire so much memory next to a chip in a 2 dimensional (flat) design - limited by the edge lengths of the chip (CPU or GPU) Basically what we need is a chip that also has pins or some type of attachement system on the top (physically) or maybe below where the chip itself connects to the motherboard. Imagine a CPU you can just plug in a block of HBM memory on top of (or on "bottom" of). This would allow a much larger physical surface area for putting ram cache near the compute cores itself because you would not be limited to edge lengths. Cooling the whole thing would be a methodology change (might need liquid coolers that sandwhich in between the ram cache and compute and cool both)