Completely eliminating the separation between RAM and compute is how FPGAs are so fast, they do most of the computing as a series of Look Up Tables (LUTs), and

Completely eliminating the separation between RAM and compute is how FPGAs are so fast, they do most of the computing as a series of Look Up Tables (LUTs), and optimize for latency and utilization with fancy switching fabrics. The downside of the switching fabrics is that optimizing a design to fit an FPGA can sometimes take days.