3 ms·
Pretty sure this wouldn't produce anything useful. Pretty sure this would generate incoherent gibberish that looks and sounds like English but makes no sense. T
by WheatMillington 2y ago
Pretty sure this wouldn't produce anything useful. Pretty sure this would generate incoherent gibberish that looks and sounds like English but makes no sense. This ignores perhaps the most important element of LLM's, the attention mechanism.
- bob1029 2y agoAnd, the attention mechanism scales quadratically with context length. This is where all of the insane memory bandwidth requirements come from.