The whole cost of performing a given LLM computation has to be at least an order of magnitude less than the state of the art if you want to make a dent in the m

The whole cost of performing a given LLM computation has to be at least an order of magnitude less than the state of the art if you want to make a dent in the market. A large part of that cost is transferring weights and values to the compute fabric. I've long thought that compute in memory could solve this, but the die areas go through the roof for the same amount of compute, even though the power is far less. I think smaller process nodes are the only way forward.