9 ms·

Trained on the same number of tokens as the 14b, which was only a little more than the 7b. Seems like they just ran out of good tokens?

MuffinFlavored 3y ago
One thing I struggle to understand is, are LLMs just a large way of spitting out which token it thinks is best based on scoring? aka it's guessing at best? How will they ever grow to the point where they are nothing more than just really good at guessing what to regurgitate based on what it's already seen? Like when it spits out what would have been a StackOverflow answer (where the community votes if it is right/wrong and it's a living thing with comments to keep it up to date as it goes stale) but mixes up a few of the tokens making the whole thing mildly broken... are these systems just going to get really good? will they ever be trustable/perfect?
- lukev 3y ago
  Not without extra machinery around the model itself. Token generation is always probabilistic to some degree.
- mrb 3y ago
  "what to regurgitate based on what it's already seen" ...which is no different than what humans do :-) Our neurons fire in a way only because they fire in patterns they have already seen before.
- porphyra 3y ago
  One piece of evidence that LLMs form some sort of internal reasoning model is the stochastic parrot chess engine: https://parrotchess.com/ https://parrotchess.com/ Despite it being statistically improbable for it to have seen every chess position before, it nonetheless plays very strongly.
  - MuffinFlavored 3y ago
    > One piece of evidence that LLMs form some sort of internal reasoning mode Isn't this a fancy way of saying "there's a scoring mechanism" like when you brute-force train a model, and certain chains/patterns of tokens score higher than others? Like in my mind, it can learn English by you showing it a bunch of known-good sentences, telling it they score highly, and bad sentences score poorly. Noun verb structure and all that. Am I greatly oversimplifying it? I feel like I am.
    - I_Am_Nous 3y ago
      I believe you would be correct to call it a scoring mechanism, but just because it can score a sentence that doesn't mean it understands what the sentence actually said. So it isn't "learning English" as much as it is knowing which tokens would be the least wrong. "Happy" could be followed with "Birthday", "Anniversary", or "New Year" but is far less likely to be followed by "Funeral" or "Wake". Except it doesn't understand that the token 66 is the word "Happy", just that it is usually followed by tokens 789, 264, or 699 (contrived example, not real token IDs). Humans can then interpret what that actually means in English and deem it correct or incorrect and adjust the weights if it's too incorrect. But at no point does the LLM learn any language other than token math.
      - MuffinFlavored 3y ago
        > But at no point does the LLM learn any language other than token math. Then can we make the argument that in the next 3 years, we aren't really due for any major AI breakthroughs and the current LLMs are about as good as they're going to get / they are kind of already at the limits of what they can do? What would more GPU power, more time to training models, more tokens, more resources for high context size do if at the end of the day it's a random-nonsense generator with a really good scoring mechanism? How much more robust/advanced can you make it? Take the context of replacing people with AI, if you can't trust the AI to be reliable + find a way to make it as accurate as humans, is there really an upcoming "AI revolution" to come?
        I_Am_Nous 3y ago
- emodendroket 3y ago
  In the end it seems like a more sophisticated version of a Markov chain.
  - acchow 3y ago
    You might think that, but there is a great deal of compression and it seems that compression is understanding, giving rise to emergent properties https://www.youtube.com/watch?v=qbIk7-JPB2c https://www.youtube.com/watch?v=qbIk7-JPB2c
- staunton 3y ago
  > will they ever be trustable/perfect? Humans aren't trustable or perfect either. Also, keep in mind that people strongly disagree on what "perfect" means...
  - MuffinFlavored 3y ago
    Good point. How far off would you say LLMs are today compared to "human levels of trust-worthiness / perfection / output"?
    - staunton 3y ago
      I think it depends very much on the domain. LLMs are good at certain things and bad at others and these strengths and weaknesses are different from humans. The trick is to figure out how good they are at certain tasks and where it's a benefit to use them. For example, in many situations it's much harder to have a good idea/solution than to check any single solution and see if it really works. A lot of the time you don't necessarily need trust-worthiness. On the other had, of course LLMs will improve and get more trustworthy (and we will learn what to expect from them, which will make them seem even more trustworthy). I don't pretend to know how fast or big improvements will be...
- filterfiber 3y ago
  > are LLMs just a large way of spitting out which token it thinks is best based on scoring? aka it's guessing at best? They are 100% exactly this. More specifically they spit out a list of "logits" aka the probability of every token (llama has a vocab size of 32000, so you'll get 32000 probabilities). Taking the highest probability is called "greedy sampling". Often you actually want to sample from either the top 5 (top k sampling), or the ones over say 90% (top p). If you're doing things like programming or doing multiple choice questions, you can choose to only sample from those logits - for example, if the only outputs should be "true" or "false", ignore any token that isn't "t", "tr", "true", "f", "fa", "false", etc. This makes it adhere to a schema. > How will they ever grow to the point where they are nothing more than just really good at guessing what to regurgitate based on what it's already seen? This is what humans do the big difference is our "weights" aren't frozen and we update them in realtime based on real world feedback. Once you loop in the real world feedback you get much better results, for example if a llm recommends a command with a syntax error, if you give it back the error message it can correct it. This is why training on synthetic data is possible. > will they ever be trustable/perfect? This is why I hate the word "hallucinate", it's not a hallucination, it's a "miss-prediction". Humans do the exact same thing all the time - miss remembering, misspeaking, doing fast math, etc. In many ways LLMs are already more "trustable" than inexperienced humans. The main thing you need is some mechanism of double-checking the output, the same as we do with humans. We can "trust" them once their probability of acceptable answers is high enough.
  - MuffinFlavored 3y ago
    > More specifically they spit out a list of "logits" aka the probability of every token (llama has a vocab size of 32000, so you'll get 32000 probabilities). I would have thought you have 7B probabilities (7b possible tokens) and 32k is just the context. So for every token (up to 32k, because that's when it runs out of context/size), you have a 7b probability. I feel like you mixed up context size with # of parameters?
    - filterfiber 3y ago
      > I feel like you mixed up context size with # of parameters? Respectfully I do not think I did :) The 7B is the parameter count. 32k is the vocab size, or list of tokens. I don't have access to the llama repo at the moment so I'll use this one - https://huggingface.co/NousResearch/Llama-2-7b-chat-hf/blob/main/config.json#L24 https://huggingface.co/NousResearch/Llama-2-7b-chat-hf/blob/... The vocab list is here https://huggingface.co/NousResearch/Llama-2-7b-chat-hf/raw/main/tokenizer.json https://huggingface.co/NousResearch/Llama-2-7b-chat-hf/raw/m... vocab_size is 32k. max_position_embeddings is 4096 for the context. Note that you can use a longer or shorter context length, but unless you use some tricks, longer context will result in rapidly decreasing performance. You will get 32k logits as the end result.
- deleted 3y ago
  [deleted]
- I_Am_Nous 3y ago
  >guessing at best Yes, but at the same time no. It's guessing based on math, so it is guessing the answer out of a filtered set of "likely correct" answers. The reason it's novel is because it can go so fast now that it seems conversational, but as nothing is free the trade off is to get better math, better probabilities, and better guesses, you have to keep expanding the size of the model which costs immense amounts of electricity for diminishing returns. There is also a limit to the amount of times you can "just make it bigger" before it actually performs more poorly, as I understand it. LLMs feel like a shortcut because it is a shortcut compared to developing something that actually understands human language. We just train them on data so it knows what words usually comes next in a series. So if the training data is good, the output is more likely to be good. But it doesn't understand what the "correct" answer is and is entirely unaware of what the content it was trained on actually means. So they will likely never be perfect because they were designed to be "good enough". I can foresee them being a part of a greater AI in the future though.
  - MuffinFlavored 3y ago
    > you have to keep expanding the size of the model Can you be more specific here? Expand the size in possible parameters? Expand the size in context? I get what you are saying in theory: what if every GPT-4 instance had 100TB of RAM it could use, what could it achieve then, but I can't think of how/what would fill that RAM that we are limited on today? I thought the more parameters you introduce, the more risk you have for hallucination because it's harder to pick which token should be next given the "more possible outcomes" total.
    - I_Am_Nous 3y ago
      I believe you are correct, I mentioned "There is also a limit to the amount of times you can "just make it bigger" before it actually performs more poorly, as I understand it." But there may also be a certain sweet spot we can slowly increase as we do more research. We lose accuracy when we quantize from a larger model to a smaller one (such as 16b quantized to 4b) and it's usually done for speed, but a larger model quantized down tends to be more accurate than a model trained to be that particular size. So it would be interesting to see how it would work if researchers build up to much larger parameter sizes, but quantized them down so they were easier to run but still kept some of the benefits of the increased parameter count. I also imagine parameter count is currently more limited by how much we can process at once, so we could be looking at the NES and trying to imagine PS5 games based on what we know about the NES :)
- nkingsy 3y ago
  From my interactions, models seem to be able to perform general purpose thinking. As in one, focused thought. I have yet to encounter a situation where a capable llm hallucinates due to faulty logic, but that is from a perspective that the llms “knowledge” is a funny side effect from the process of making it able to think, and never to be trusted. The chatty knowledge side effect is somehow obscuring the fact that we have a general purpose logic processor on our hands. Building up reliable tasks on top of this is going to play out similarly to the computing revolution, but accelerated by huge global interest. Comparing this to the internet boom is mistaken. The internet was the missing piece of the computing puzzle for achieving mass consumer appeal. We haven’t hit that yet. We’re still thinking about these things as calculators and speed running mainframe deployments.

MuffinFlavored 3y ago

One thing I struggle to understand is, are LLMs just a large way of spitting out which token it thinks is best based on scoring? aka it's guessing at best? How will they ever grow to the point where they are nothing more than just really good at guessing what to regurgitate based on what it's already seen? Like when it spits out what would have been a StackOverflow answer (where the community votes if it is right/wrong and it's a living thing with comments to keep it up to date as it goes stale) but mixes up a few of the tokens making the whole thing mildly broken... are these systems just going to get really good? will they ever be trustable/perfect?

lukev 3y ago
Not without extra machinery around the model itself. Token generation is always probabilistic to some degree.
mrb 3y ago
"what to regurgitate based on what it's already seen" ...which is no different than what humans do :-) Our neurons fire in a way only because they fire in patterns they have already seen before.
porphyra 3y ago
One piece of evidence that LLMs form some sort of internal reasoning model is the stochastic parrot chess engine: https://parrotchess.com/ https://parrotchess.com/ Despite it being statistically improbable for it to have seen every chess position before, it nonetheless plays very strongly.
- MuffinFlavored 3y ago
  > One piece of evidence that LLMs form some sort of internal reasoning mode Isn't this a fancy way of saying "there's a scoring mechanism" like when you brute-force train a model, and certain chains/patterns of tokens score higher than others? Like in my mind, it can learn English by you showing it a bunch of known-good sentences, telling it they score highly, and bad sentences score poorly. Noun verb structure and all that. Am I greatly oversimplifying it? I feel like I am.
  - I_Am_Nous 3y ago
    I believe you would be correct to call it a scoring mechanism, but just because it can score a sentence that doesn't mean it understands what the sentence actually said. So it isn't "learning English" as much as it is knowing which tokens would be the least wrong. "Happy" could be followed with "Birthday", "Anniversary", or "New Year" but is far less likely to be followed by "Funeral" or "Wake". Except it doesn't understand that the token 66 is the word "Happy", just that it is usually followed by tokens 789, 264, or 699 (contrived example, not real token IDs). Humans can then interpret what that actually means in English and deem it correct or incorrect and adjust the weights if it's too incorrect. But at no point does the LLM learn any language other than token math.
    - MuffinFlavored 3y ago
      > But at no point does the LLM learn any language other than token math. Then can we make the argument that in the next 3 years, we aren't really due for any major AI breakthroughs and the current LLMs are about as good as they're going to get / they are kind of already at the limits of what they can do? What would more GPU power, more time to training models, more tokens, more resources for high context size do if at the end of the day it's a random-nonsense generator with a really good scoring mechanism? How much more robust/advanced can you make it? Take the context of replacing people with AI, if you can't trust the AI to be reliable + find a way to make it as accurate as humans, is there really an upcoming "AI revolution" to come?
      - I_Am_Nous 3y ago
emodendroket 3y ago
In the end it seems like a more sophisticated version of a Markov chain.
- acchow 3y ago
  You might think that, but there is a great deal of compression and it seems that compression is understanding, giving rise to emergent properties https://www.youtube.com/watch?v=qbIk7-JPB2c https://www.youtube.com/watch?v=qbIk7-JPB2c
staunton 3y ago
> will they ever be trustable/perfect? Humans aren't trustable or perfect either. Also, keep in mind that people strongly disagree on what "perfect" means...
- MuffinFlavored 3y ago
  Good point. How far off would you say LLMs are today compared to "human levels of trust-worthiness / perfection / output"?
  - staunton 3y ago
    I think it depends very much on the domain. LLMs are good at certain things and bad at others and these strengths and weaknesses are different from humans. The trick is to figure out how good they are at certain tasks and where it's a benefit to use them. For example, in many situations it's much harder to have a good idea/solution than to check any single solution and see if it really works. A lot of the time you don't necessarily need trust-worthiness. On the other had, of course LLMs will improve and get more trustworthy (and we will learn what to expect from them, which will make them seem even more trustworthy). I don't pretend to know how fast or big improvements will be...
filterfiber 3y ago
> are LLMs just a large way of spitting out which token it thinks is best based on scoring? aka it's guessing at best? They are 100% exactly this. More specifically they spit out a list of "logits" aka the probability of every token (llama has a vocab size of 32000, so you'll get 32000 probabilities). Taking the highest probability is called "greedy sampling". Often you actually want to sample from either the top 5 (top k sampling), or the ones over say 90% (top p). If you're doing things like programming or doing multiple choice questions, you can choose to only sample from those logits - for example, if the only outputs should be "true" or "false", ignore any token that isn't "t", "tr", "true", "f", "fa", "false", etc. This makes it adhere to a schema. > How will they ever grow to the point where they are nothing more than just really good at guessing what to regurgitate based on what it's already seen? This is what humans do the big difference is our "weights" aren't frozen and we update them in realtime based on real world feedback. Once you loop in the real world feedback you get much better results, for example if a llm recommends a command with a syntax error, if you give it back the error message it can correct it. This is why training on synthetic data is possible. > will they ever be trustable/perfect? This is why I hate the word "hallucinate", it's not a hallucination, it's a "miss-prediction". Humans do the exact same thing all the time - miss remembering, misspeaking, doing fast math, etc. In many ways LLMs are already more "trustable" than inexperienced humans. The main thing you need is some mechanism of double-checking the output, the same as we do with humans. We can "trust" them once their probability of acceptable answers is high enough.
- MuffinFlavored 3y ago
  > More specifically they spit out a list of "logits" aka the probability of every token (llama has a vocab size of 32000, so you'll get 32000 probabilities). I would have thought you have 7B probabilities (7b possible tokens) and 32k is just the context. So for every token (up to 32k, because that's when it runs out of context/size), you have a 7b probability. I feel like you mixed up context size with # of parameters?
  - filterfiber 3y ago
    > I feel like you mixed up context size with # of parameters? Respectfully I do not think I did :) The 7B is the parameter count. 32k is the vocab size, or list of tokens. I don't have access to the llama repo at the moment so I'll use this one - https://huggingface.co/NousResearch/Llama-2-7b-chat-hf/blob/main/config.json#L24 https://huggingface.co/NousResearch/Llama-2-7b-chat-hf/blob/... The vocab list is here https://huggingface.co/NousResearch/Llama-2-7b-chat-hf/raw/main/tokenizer.json https://huggingface.co/NousResearch/Llama-2-7b-chat-hf/raw/m... vocab_size is 32k. max_position_embeddings is 4096 for the context. Note that you can use a longer or shorter context length, but unless you use some tricks, longer context will result in rapidly decreasing performance. You will get 32k logits as the end result.
deleted 3y ago
[deleted]
I_Am_Nous 3y ago
>guessing at best Yes, but at the same time no. It's guessing based on math, so it is guessing the answer out of a filtered set of "likely correct" answers. The reason it's novel is because it can go so fast now that it seems conversational, but as nothing is free the trade off is to get better math, better probabilities, and better guesses, you have to keep expanding the size of the model which costs immense amounts of electricity for diminishing returns. There is also a limit to the amount of times you can "just make it bigger" before it actually performs more poorly, as I understand it. LLMs feel like a shortcut because it is a shortcut compared to developing something that actually understands human language. We just train them on data so it knows what words usually comes next in a series. So if the training data is good, the output is more likely to be good. But it doesn't understand what the "correct" answer is and is entirely unaware of what the content it was trained on actually means. So they will likely never be perfect because they were designed to be "good enough". I can foresee them being a part of a greater AI in the future though.
- MuffinFlavored 3y ago
  > you have to keep expanding the size of the model Can you be more specific here? Expand the size in possible parameters? Expand the size in context? I get what you are saying in theory: what if every GPT-4 instance had 100TB of RAM it could use, what could it achieve then, but I can't think of how/what would fill that RAM that we are limited on today? I thought the more parameters you introduce, the more risk you have for hallucination because it's harder to pick which token should be next given the "more possible outcomes" total.
  - I_Am_Nous 3y ago
    I believe you are correct, I mentioned "There is also a limit to the amount of times you can "just make it bigger" before it actually performs more poorly, as I understand it." But there may also be a certain sweet spot we can slowly increase as we do more research. We lose accuracy when we quantize from a larger model to a smaller one (such as 16b quantized to 4b) and it's usually done for speed, but a larger model quantized down tends to be more accurate than a model trained to be that particular size. So it would be interesting to see how it would work if researchers build up to much larger parameter sizes, but quantized them down so they were easier to run but still kept some of the benefits of the increased parameter count. I also imagine parameter count is currently more limited by how much we can process at once, so we could be looking at the NES and trying to imagine PS5 games based on what we know about the NES :)
nkingsy 3y ago
From my interactions, models seem to be able to perform general purpose thinking. As in one, focused thought. I have yet to encounter a situation where a capable llm hallucinates due to faulty logic, but that is from a perspective that the llms “knowledge” is a funny side effect from the process of making it able to think, and never to be trusted. The chatty knowledge side effect is somehow obscuring the fact that we have a general purpose logic processor on our hands. Building up reliable tasks on top of this is going to play out similarly to the computing revolution, but accelerated by huge global interest. Comparing this to the internet boom is mistaken. The internet was the missing piece of the computing puzzle for achieving mass consumer appeal. We haven’t hit that yet. We’re still thinking about these things as calculators and speed running mainframe deployments.