3 ms·
LLMs output a probability distribution for the next word. Searching the space of what the best next word to use is takes more time than just picking a good one
by charcircuit 3y ago
LLMs output a probability distribution for the next word. Searching the space of what the best next word to use is takes more time than just picking a good one and assuming that it was a good enough choice.