idk, they do give the same results, but given the memory bottleneck it feels like we are at a point when architecture innovations matter again, for example chec

idk, they do give the same results, but given the memory bottleneck it feels like we are at a point when architecture innovations matter again, for example check out DeepSeek V2 tech report, they modded model arch specifically for lower cost inference (by making k/v cache smaller)