_akhe

31 ms·

391.
by _akhe 3y ago
I really appreciate this, thanks! Sea of haters on HN lol some are just confused or perpetually angry. I took some of them too seriously, shouldn't have even replied, especially those mentioning "Markov chains" as if that nar
392.
by _akhe 3y ago
What is love? Baby, don't hurt me Don't hurt me no more xD Typically falsely accusing then calling a person ignorant is somewhat insulting! But now that you said it was in the name of love I'm tearing up (in a good way).
393.
by _akhe 3y ago
Prepare to be underwhelmed - no need for a paper for this one: The only emergent ability of an LLM is that the model is more robust when there are more samples. When the number of samples is in the trillions instead of thousands, there are
394.
by _akhe 3y ago
Not joking, not trolling, thought it was widely known that JavaScript is generally significantly faster than Python. Haven't had this debate in 15 years, but let me explain: > What data structures? Everything in JavaScript is an obj
395.
by _akhe 3y ago
Not joking. You seem to be confused in thinking LLMs are built with other LLMs, but that's not the case. Otherwise why would you say "it says 'build fast LLMs from scratch' but the code is not an LLM" ?? Why would t
396.
by _akhe 3y ago
And LICENSE has no file type!
397.
by _akhe 3y ago
> inb4 "just a Markov Model" A Markov chain is a much looser concept than next token prediction, and would apply to any finite state machine where you can predict the next state. The React state management library XState is a M
398.
by _akhe 3y ago
I never said or implied it's a Markov model, someone else did in this thread, and someone parroted them. A Markov chain is a much looser concept than next token prediction, and would apply to any finite state machine where you can pred
399.
by _akhe 3y ago
> LLMs work on a completely different principle News to me - can you tell me what principle you think an LLM works on?
400.
by _akhe 3y ago
Instead of using someone else's language library pre-trained on a bunch of text, you can create your own language library trained on your own text. Use cases include: Auto-completion, auto-correct, spell checking, search/lookup, c
401.
by _akhe 3y ago
This looks amazing, like a modern take on Dreamweaver or something. The low code IDE may soon return! The ability to switch between no-code/code is great for teaching people to code. Dreamweaver (and MySpace profiles) basically taught
402.
by _akhe 3y ago
Two things: 1) This library will continue predicting words too for every word that exists in the model, it just makes less sense as you keep going. That's also true of any LLM. 2) An LLM is not synonymous with "chat bot".
403.
by _akhe 3y ago
Exactly right. The others here are confusing LLM with "chat bot", and also they seem to be confusing token prediction with LLM. I have a feeling the mainstream won't really get it until a ChatGPT clone is online and ready to
404.
by _akhe 3y ago
Yes the time complexity is how it can all be held in memory without crashing (because it isn't really all held in memory, like it would be with text) but that's only part of it. You could not choose "any programming language&
405.
by _akhe 3y ago
The library never claims to be an LLM - it's a next token prediction library that you can use to create LLMs. Python - one nice thing is all the "keys" retain their primitive type, where as in JS they all turn into strings. I
406.
by _akhe 3y ago
It's a next token prediction library. "Markov chain" basically means finite state machine and is a looser concept. If you want to call the token prediction methodology Markovian you can - sounds cool! The implementation anoth
407.
by _akhe 3y ago
Why? If I build a chat bot from it would it seem more like an LLM to you?
408.
by _akhe 3y ago
For chat, there's basically a missing NLP layer - what pop culture calls "multi-headed attention blocks", but for next token prediction it's pretty good. Maybe the philosophical concept of a Markov chain abstractly descr
409.
by _akhe 3y ago
Yes adding a human interface layer (like ChatGPT) will make it more capable, but keep in mind LLM does not mean "chat bot".
410.
by _akhe 3y ago
It's a next token prediction library. "Markov chain" basically means finite state machine and is a looser concept. If you want to call the token prediction methodology Markovian you can - sounds cool! The implementation anoth
411.
by _akhe 3y ago
It's a next token prediction library, not an LLM. The title says "Build LLMs from scratch" and that's exactly what you can do with it :) Keep in mind an LLM is not "ChatGPT" though ChatGPT uses an LLM.
412.
by _akhe 3y ago
Of course I do, who doesn't? Can you pick 1 thing I need to focus on most so I can get some value out of this comment? :D
413.
by _akhe 3y ago
Yep it can work for "next-pixel prediction" too where the token is a pixel. If I have enough samples of arrangements of pixels, it's not fundamentally different than samples of arrangements of words. I'll check out that
414.
by _akhe 3y ago
> enough training data > good chat interface I'm going to try to do that here: https://github.com/bennyschmidt/llimo I totally understand your skepticism, and am also surprised it works so well as a sentence
415.
by _akhe 3y ago
Hell yeah :) At first I thought a simple Show HN titled "Next token prediction in 100 lines of JavaScript" would be cool, but the project grew and now it's a few different modules and things going on. A couple people have bas
416.
by _akhe 3y ago
This is awesome, thanks. I've been messing with wink's NLP library ( https://winkjs.org/wink-nlp/ ) to transform user queries and format responses so I can make a proper chat bot - will see what I can learn fro
417.
by _akhe 3y ago
Amazing resource thank you! Agree regarding videos for learning technical concepts - strangely when things are over-simplified they can become harder to understand on a technical level, I guess just being so abstracted. Example: The embeddi
418.
by _akhe 3y ago
Awesome video, and really cool. I loved the ATTCares convo haha. I first heard about Markov chains from a philosopher named Donald Hoffman, but this demo makes it more clear! Thanks for sharing
419.
by _akhe 3y ago
I love where you're going with this, followed on github :) Looking forward to seeing progress here. Edit: Oh it's 5 years old, welp
420.
by _akhe 3y ago
I would use it for game dev. You know the radio in GTA - something like that. Especially if I can have some control over how the song is made.

More ›