6 ms·

Of course I do, who doesn't? Can you pick 1 thing I need to focus on most so I can get some value out of this comment? :D

by _akhe 3y ago

nmstoker 3y ago
I would suggest first removing the reference to LLM in the description: "build fast LLMs from scratch!"
- _akhe 3y ago
  Why? If I build a chat bot from it would it seem more like an LLM to you?
  - mirekrusin 3y ago
    You're cargo culting "LLM".
    - _akhe 3y ago
      You don't know what an LLM is - there's a difference.
hnfong 3y ago
You can't "Demystify LLMs for people" if you don't know how it works. And it definitely isn't a Marcov model (https://en.wikipedia.org/wiki/Hidden_Markov_model https://en.wikipedia.org/wiki/Hidden_Markov_model) like you implied it is. As far as the "emergent" apparent intelligence of SoTA LLM goes, it's not just counting the number of words that appear after a phrase. I don't think anyone has a good explanation of how it works so far, only that empirically a bunch of techniques combined together seems to do the trick and LLMs do seem to acquire intelligence far beyond what a Marcov model could acquire after the training process. Therefore, see eg. https://www.reddit.com/r/LocalLLaMA/comments/1bgh9h4/the_truth_about_llms/ https://www.reddit.com/r/LocalLLaMA/comments/1bgh9h4/the_tru... A joke, but it's the truth as we know it. > Of course I do, who doesn't? Implying you (who apparently just watched a single video) are as ignorant as people who has read extensively on the subject (not me, but presumably many others here) isn't helpful. I mean, there's nothing stopping you from believing you know enough to explain what LLMs are to other people, but the GP is honestly telling you what you need to hear IMHO.
- _akhe 3y ago
  I never said or implied it's a Markov model, someone else did in this thread, and someone parroted them. A Markov chain is a much looser concept than next token prediction, and would apply to any finite state machine where you can predict the next state. The React state management library XState is a Markov chain. Has nothing to do with language at all. I don't appreciate you calling me ignorant, but I think the source of your projection is this phrase: "I don't think anyone has a good explanation of how it works so far" which translates to you don't know the basics. I get it, makes you mad for some reason lol But maybe it's you that needs to read up on what a language model actually is - what next token prediction means, and how it applies to LLMs you know and love. It really is not rocket science :) Thanks for your comment!
  - hnfong 3y ago
    Love doesn't always come in the form of flattery or praise, and if you took that as an assault or even an insult (which TBH it wasn't) instead of pointing out the limits of your knowledge, then that's a shame. I mean, it's pointless to compare the size of our intellectual capabilities on the internet since we don't know each other, but perhaps I was wrong for a second that you might have actually wanted to "get some value" of the comment of "author has a lot to learn".
    - _akhe 3y ago
      What is love? Baby, don't hurt me Don't hurt me no more xD Typically falsely accusing then calling a person ignorant is somewhat insulting! But now that you said it was in the name of love I'm tearing up (in a good way). And you're right - what was I thinking? I'm in the presence of a literal genius here. THANK YOU for pointing out the "limits of my knowledge", sometimes it takes a really smart person like you to point it out, even though you got the wrong guy (Markov) and have yet to make a single point about LLMs or next-token prediction, beyond: "Nobody knows how LLMs work" and a link to a Reddit meme. I can't be too far behind your advanced state of intellectual genius with zingers like these. Finally, it may not seem like it, but I have derived value from your comments, just not any realizations you'd likely care to hear :)
mirekrusin 3y ago
You should start from chapter 1.
- _akhe 3y ago
  You've been coming back to this post for 3 days to leave more negative comments, how sad.

Of course I do, who doesn't? Can you pick 1 thing I need to focus on most so I can get some value out of this comment? :D

by _akhe 3y ago

Of course I do, who doesn't? Can you pick 1 thing I need to focus on most so I can get some value out of this comment? :D

nmstoker 3y ago

I would suggest first removing the reference to LLM in the description: "build fast LLMs from scratch!"

_akhe 3y ago
Why? If I build a chat bot from it would it seem more like an LLM to you?
- mirekrusin 3y ago
  You're cargo culting "LLM".
  - _akhe 3y ago
    You don't know what an LLM is - there's a difference.

hnfong 3y ago

You can't "Demystify LLMs for people" if you don't know how it works. And it definitely isn't a Marcov model (https://en.wikipedia.org/wiki/Hidden_Markov_model https://en.wikipedia.org/wiki/Hidden_Markov_model) like you implied it is. As far as the "emergent" apparent intelligence of SoTA LLM goes, it's not just counting the number of words that appear after a phrase. I don't think anyone has a good explanation of how it works so far, only that empirically a bunch of techniques combined together seems to do the trick and LLMs do seem to acquire intelligence far beyond what a Marcov model could acquire after the training process. Therefore, see eg. https://www.reddit.com/r/LocalLLaMA/comments/1bgh9h4/the_truth_about_llms/ https://www.reddit.com/r/LocalLLaMA/comments/1bgh9h4/the_tru... A joke, but it's the truth as we know it. > Of course I do, who doesn't? Implying you (who apparently just watched a single video) are as ignorant as people who has read extensively on the subject (not me, but presumably many others here) isn't helpful. I mean, there's nothing stopping you from believing you know enough to explain what LLMs are to other people, but the GP is honestly telling you what you need to hear IMHO.

_akhe 3y ago
I never said or implied it's a Markov model, someone else did in this thread, and someone parroted them. A Markov chain is a much looser concept than next token prediction, and would apply to any finite state machine where you can predict the next state. The React state management library XState is a Markov chain. Has nothing to do with language at all. I don't appreciate you calling me ignorant, but I think the source of your projection is this phrase: "I don't think anyone has a good explanation of how it works so far" which translates to you don't know the basics. I get it, makes you mad for some reason lol But maybe it's you that needs to read up on what a language model actually is - what next token prediction means, and how it applies to LLMs you know and love. It really is not rocket science :) Thanks for your comment!
- hnfong 3y ago
  Love doesn't always come in the form of flattery or praise, and if you took that as an assault or even an insult (which TBH it wasn't) instead of pointing out the limits of your knowledge, then that's a shame. I mean, it's pointless to compare the size of our intellectual capabilities on the internet since we don't know each other, but perhaps I was wrong for a second that you might have actually wanted to "get some value" of the comment of "author has a lot to learn".
  - _akhe 3y ago
    What is love? Baby, don't hurt me Don't hurt me no more xD Typically falsely accusing then calling a person ignorant is somewhat insulting! But now that you said it was in the name of love I'm tearing up (in a good way). And you're right - what was I thinking? I'm in the presence of a literal genius here. THANK YOU for pointing out the "limits of my knowledge", sometimes it takes a really smart person like you to point it out, even though you got the wrong guy (Markov) and have yet to make a single point about LLMs or next-token prediction, beyond: "Nobody knows how LLMs work" and a link to a Reddit meme. I can't be too far behind your advanced state of intellectual genius with zingers like these. Finally, it may not seem like it, but I have derived value from your comments, just not any realizations you'd likely care to hear :)

mirekrusin 3y ago

You should start from chapter 1.

_akhe 3y ago
You've been coming back to this post for 3 days to leave more negative comments, how sad.