> Current models take a nuclear reactors worth of power to run back prop on top of a small countries GDP worth of hardware. Nice one. Perhaps we are to conclud

> Current models take a nuclear reactors worth of power to run back prop on top of a small countries GDP worth of hardware. Nice one. Perhaps we are to conclude the whole transformer architecture is amazingly overblown in storage/computation costs. AGI or not, we need better approach to what transformers are doing.