4 ms·
There quite literally are internals that are called attention heads in modern LLMs. Saying 'there are no attention heads' is objectively false, which definition
by Catloafdev 2mo ago
There quite literally are internals that are called attention heads in modern LLMs. Saying 'there are no attention heads' is objectively false, which definition you follow is irrelevant.
- thaumasiotes 2mo agoImagine the following conversation: How was your trip to Los Angeles? There were no movie stars! Is the second person's statement "objectively false"? Or is it more likely that you, Catloafdev, enjoy intentionally misinterpreting simple statements? You might notice that in the remainder of the sentence you quoted, hungryhobbit takes the trouble to define the attention heads that don't exist.
- Catloafdev 2mo ago[dead]