3 ms·

> Sure, artificial neural network is a great analogy. But that leaves out what it literally is It's not an analogy - an artificial neural network is literally

> Sure, artificial neural network is a great analogy. But that leaves out what it literally is It's not an analogy - an artificial neural network is literally what an LLM is. A trie is not an LLM, because it's not a neural network. If someone asked how many parameters a trie loaded with many sentences has, that question wouldn't even make sense, because it doesn't model weights between neurons, unlike an LLM. I think perhaps you're getting confused between the class of 'next token predicting algorithms' (which is a class with lots of different algorithms in it) and 'LLMs'. LLMs are a specific kind of next token predicting algorithm that work in a particular way. You can make a next token predictor out of a trie (as you have), but it is not an LLM.

_akhe 3y ago
Again, the library does not claim to "be an LLM" - it's not that at all, it's actually a next-token prediction library that has methods like `getTokenPrediction` etc. Though you are set up to build an LLM with this if you train enough data :) The library is not going to provide a `chat` function like GPT, not going to provide an img2img API like Stable Diffusion, that would be whatever chat-based or image-based LLM is being made with this library. (reply was not visible before, so I appended as an Edit)
- kybernetikos 3y ago
  > Though you are set up to build an LLM with this if you train enough data But you aren't. No amount of data loaded into a trie will turn the trie into a neural net. > The library is not going to provide a `chat` function like GPT, Providing a 'chat' function is nothing to do with what a LLM is. Here's the first sentence from the second paragraph of wikipedia on Large Language Models: "LLMs are artificial neural networks." When someone points out that you wouldn't use this kind of library to create an LLM, you generally say something about how you think they're complaining that it doesn't chat with you. It's not about chat, it's about the fact that the functionality you've written about has nothing to do with the way that LLMs work. LLMs are artificial neural networks. What you've written is a trie, not an artificial neural network. Artificial neural networks need fast matrix multiplication, not tries.
  - _akhe 3y ago
    The entire library is not just a data structure, it just uses it to model language. You still haven't npm installed have you? If you spent 5 minutes with it, it would answer your questions. You use it like this: `agent.getTokenPrediction('when')` It's not "bro it's just a trie" like you keep saying no matter what I tell you or try to show you. "A trie" is not a JavaScript library you can install with methods like "getTokenPrediction" or "complete" that help you auto-complete sequences. And since you love the neuron model so much, know that you can train on data that begets the same node/weight based "learning" that an ANN does, I just haven't gone all-in on the "neuron" analogy. You know - other people here want me to call it a Markov chain, not a trie at all, and don't want me to call the semantic score "embeddings" because they aren't stored as numbers, so one thing I've learned is this phrasing ("branding") matters more than functionality. Your last paragraph - yeah we agree, and I've been trying to tell a lot of other people that "LLM !== chat bot". Providing a `chat` method doesn't make it equivalent to an LLM so I didn't provide anything related to chat. I have no idea what kind of AI model you might build - could be next-pixel prediction and you get into Stable Diffusion-like functionality, not chat at all. "What you've written is a trie" I didn't write a trie, they're automatically generated during training. The library has nothing to do with that, but you keep saying it does. What I did write is a next-token prediction library for giving you the next likeliest letter, word, pixel, etc. Hope this makes sense!

_akhe 3y ago

Again, the library does not claim to "be an LLM" - it's not that at all, it's actually a next-token prediction library that has methods like `getTokenPrediction` etc. Though you are set up to build an LLM with this if you train enough data :) The library is not going to provide a `chat` function like GPT, not going to provide an img2img API like Stable Diffusion, that would be whatever chat-based or image-based LLM is being made with this library. (reply was not visible before, so I appended as an Edit)

kybernetikos 3y ago
> Though you are set up to build an LLM with this if you train enough data But you aren't. No amount of data loaded into a trie will turn the trie into a neural net. > The library is not going to provide a `chat` function like GPT, Providing a 'chat' function is nothing to do with what a LLM is. Here's the first sentence from the second paragraph of wikipedia on Large Language Models: "LLMs are artificial neural networks." When someone points out that you wouldn't use this kind of library to create an LLM, you generally say something about how you think they're complaining that it doesn't chat with you. It's not about chat, it's about the fact that the functionality you've written about has nothing to do with the way that LLMs work. LLMs are artificial neural networks. What you've written is a trie, not an artificial neural network. Artificial neural networks need fast matrix multiplication, not tries.
- _akhe 3y ago
  The entire library is not just a data structure, it just uses it to model language. You still haven't npm installed have you? If you spent 5 minutes with it, it would answer your questions. You use it like this: `agent.getTokenPrediction('when')` It's not "bro it's just a trie" like you keep saying no matter what I tell you or try to show you. "A trie" is not a JavaScript library you can install with methods like "getTokenPrediction" or "complete" that help you auto-complete sequences. And since you love the neuron model so much, know that you can train on data that begets the same node/weight based "learning" that an ANN does, I just haven't gone all-in on the "neuron" analogy. You know - other people here want me to call it a Markov chain, not a trie at all, and don't want me to call the semantic score "embeddings" because they aren't stored as numbers, so one thing I've learned is this phrasing ("branding") matters more than functionality. Your last paragraph - yeah we agree, and I've been trying to tell a lot of other people that "LLM !== chat bot". Providing a `chat` method doesn't make it equivalent to an LLM so I didn't provide anything related to chat. I have no idea what kind of AI model you might build - could be next-pixel prediction and you get into Stable Diffusion-like functionality, not chat at all. "What you've written is a trie" I didn't write a trie, they're automatically generated during training. The library has nothing to do with that, but you keep saying it does. What I did write is a next-token prediction library for giving you the next likeliest letter, word, pixel, etc. Hope this makes sense!