2 ms·
Inference compute costs and training compute costs aren’t the same. Training costs are an order of magnitude higher.
by initplus 3y ago
Inference compute costs and training compute costs aren’t the same. Training costs are an order of magnitude higher.
- ActorNightly 3y agoTraining will be significantly cheaper and take less time once we have the correct software for the 4090s Right now, the idea is that every time you build an LLM, you start the training from scratch, because thats all we know how to do. Human like AI will most definitely not be trained like that. Humans can look at a piece of information once or twice and remember it. Just like the attention paper, at some point someone will publish a paper that describes a methodology of feeding one piece of data back through the network only a few times to fully train it.