minimaltom

6 ms·

1. 91.▲
   by minimaltom 9mo ago

   Speaking generally: It makes sense for a acquisition price to be at a premium to valuation, between the dynamics where you have to convince leadership its better to be bought than to keep growing, and the expected risk posed by them as comp
2. 92.▲
   by minimaltom 9mo ago

   Analogies are all messy here, but I would compare the values of the residual stream to what you are describing as thought. We force this residual stream to project to the logprobs of all tokens, just as a human in the act of speaking a sent
3. 93.▲
   by minimaltom 9mo ago

   "becomes" as in the locus of focus as my career progresses. Vibes-wise, i think we agree / are saying the same thing.
4. 94.▲
   by minimaltom 9mo ago

   I like OP's representation, but I feel like a lot of people arent saying 'LLMs are the bomb dot com _right now_' (though some are), but rather the trend is evident: these things will keep getting better, and the writing is on
5. 95.▲
   by minimaltom 9mo ago

   > In contrast, human thinking doesn’t involve picking a word at a time based on the words that came before Do we have science that demonstrates humans don't autoregressively emit words? (Genuinely curious / uninformed). From th
6. 96.▲
   by minimaltom 9mo ago

   A few things I think are useful to emphasize on the training side, beyond what the article says: 1. Pre-training nowadays do not just use the 'next word/token' as a training signal, but also the next N words, because that app