5 ms·

They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercom

They cannot say "I dont know" because they dont actually know anything. The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match. There is no thinking brain with a conceptual understanding of its own limitations. Getting an "i dont know" from current AI is like asking navigation software how far it is to the Simpsons house in Springfield: the machine spits out answers but cannot fathom the cultural reference that makes the answer impossible. Instead, it finds someone named simpson in the nearest realworld Springfield.

williamcotton 3y ago
What if you worked on the problem and tried to come up with some kind of solution?
- sandworm101 3y ago
  The solution is older non-AI tech. Google search can say "no good results found" because it returns actual data rather than creating anything new. If you want a hard answer about the presence or absence of something, AI isnt the correct tool.
  - williamcotton 3y ago
    So there are no other possibilities for us other than using a system that can be gamed for substandard results? Are we sure about this?
  - tempest_ 3y ago
    Can, but doesn't. I can't remember the last time google actually returned no results.
    - tsimionescu 3y ago
      It does reply with no results, but only for very long queries. E.g. If you search for two concatenated GUIDs, you can easily see a no results page.
    - johnny22 3y ago
      ah, i get no results pages often when i search for quoted error strings from many different sources. Thing is, I have a hard time believing that no one has actually talked about at least some of those errors :(
paulnpace 3y ago
My observation is that comments similar to GP come from the constant anthropomorphizing of things by marketers and without realizing that this subtle influence on language can alter one's view on what "I" means. The first time I really noticed this was when someone using Siri produced a response from Siri that included "I". Ever since I am acutely aware of this every time I hear or read it.
caditinpiscinam 3y ago
In real world conversations, people are constantly saying "I don't know"; but that doesn't really happen online. If you're on reddit or stack overflow or hacker news and you see a question you don't know the answer to, you normally just don't say anything. If LLMs are being trained on conversations pulled from the internet then they're missing out on a ton of uncertain responses. Maybe LLMs don't truly "understand" questions, but they're good at looking like they understand questions. If they were trained with more uncertain content, perhaps they'd be better at expressing uncertainty as well.
- username332211 3y ago
  If they were trained on more uncertain content, what happens if the most probable answer to a question is "I don't know", even though an answer exists in it's training set? Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? When the LLM answers "I don't know.", this could be a hallucination just as easily as anything else.
  - deleted 3y ago
    [deleted]
  - caditinpiscinam 3y ago
    > Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? I don't know :) Actually though, I think the best response would be to say that the answer to the question isn't clear, but that 11 m/s is sometimes given as an estimate. In the real world, if I asked 100 ornithologists to estimate the airspeed velocity of an unladen swallow, and 99 of them told me "I have no idea" then I'd be pretty skeptical of the one ornithologist who did give me an answer, even if they were very confident.
    - ta8645 3y ago
      I think the best response is to steal the joke and repeat it without comment. "Eleven meters per second." Full stop. It's humorous, and any reasonable interlocutor understands not to take it seriously. Of course, there are more serious questions that demand more serious answers. LLMs will eventually need to be able to understand the current context and assess the appropriate level of confidence required in any answer.
DiogenesKynikos 3y ago
> The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. Are you sure you're not also describing the human brain? At some point, after we have sufficiently demystified the workings of the human brain, it will probably also sound something like, "Well, the brain is just a large machine that does X, Y and Z [insert banal-sounding technical jargon from the future] - it doesn't really understand anything." My point here is that understanding ultimately comes down to having an effective internal model of the world, which is capable of taking novel inputs and generating reasonable descriptions of them or reactions to them. It turns out that LLMs are one way of achieving that. They don't function exactly like human brains, but they certainly do exhibit intelligence and understanding. I can ask an LLM a question that it has never seen before, and it will give me a reasonable answer that synthesizes and builds on various facts that it knows. Often the answer is more intelligent than what one would get from most humans. That's understanding.
- beardedwizard 3y ago
  Human brains form new connections dynamically. Llms are trained on connections human brains have already made. They never make new connections that aren't in training data. Nothing was synthesized, all the data was seen before and related to each other by vector similarity. It can just parrot the collective understanding humans already have and teach it.
  - DiogenesKynikos 3y ago
    > It can just parrot the collective understanding humans already have and teach it. The problem with calling an LLM a parrot is that anyone who has actually interacted with an LLM knows that it produces completely novel responses to questions it has never seen before. These answers are usually logical and reasonable, based on both the information you gave the LLM and its previous knowledge of the world. Doing that requires understanding. > They never make new connections that aren't in training data. This is just categorically untrue. They make all sorts of logical connections that are not explicitly contained in the training data. Making logical inferences about subjects one has never heard about - based on the things one does know - is an expression of understanding. LLMs do that.
fl7305 3y ago
> a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns That was perhaps true of earlier and smaller LLMs, like GPT-1 and GPT-2. But as they grew larger and were trained with more and more data, they changed from pure pattern matching to implementing algorithms to compress more information into their structure than pure pattern matching can achieve. These algorithms are incomplete and buggy, but they are nonetheless executing algorithms, and not just pattern matching. This phenomenom can be seen in toy-sized neural networks. For instance, addition of two input values modulo a constant. As a small network is trained, at some point the internal structure can change from pattern matching to implementing addition using Fourier transforms. This is clearly visible in its structure. The network now performs the task perfectly for all inputs, regardless of having seen them in training. You can ask ChatGPT 4 to execute an algorithm for you. I just tried this one: I would like to play a game, where you are the host. We start off with a score that is 1234143143. At the start of each turn, you tell me the current score and ask me if I want to play a or b. If I choose a, the score is halved, and 30 is added. If I choose b, the score is doubled, and 40 is subtracted. Only use integers and round down. It will happily execute this algorithm. For large numbers, it is slightly off on the arithmetic. When I asked it to double check, it did so using Python code. After that, it kept using Python code to perform the math. It was also able to reason intelligently about different outcomes if always picking a (or b) given different starting points. Now, if you have enough memory and training data, of course you can build a gigantic lookup table that has this exact text sequence in it to replicate "executing the algorithm" I described. Is that your claim? How much memory are we talking about? My feeling is that it'd be far more than the number of atoms in the universe. PS Me: How far it is to the Simpsons house in Springfield? ChatGPT: The Simpsons' house in Springfield is a fictional location from the animated TV series "The Simpsons." Since Springfield is a fictional town and its location is not consistently specified in the series, it's not possible to determine a real-world distance to the Simpsons' house. Me: Do I have a brother? Please answer with either of: a) The answer b) There is an answer, but I do not know c) There is no answer ChatGPT: b) There is an answer, but I do not know
- andsoitis 3y ago
  > It will happily execute this algorithm. For large numbers, it is slightly off on the arithmetic. When I asked it to double check, it did so using Python code. After that, it kept using Python code to perform the math. It was also able to reason intelligently about different outcomes if always picking a (or b) given different starting points. Notice that you had to notice the error and had to prompt it to double check. Lots of complicated things going on here. Many (most?) humans will fail somewhere along this trajectory. Did it double check the Python code to make sure it is correct (not just in the sense that it is valid, executable code, but that it is the correct check in the first place)? Or did you double check that its modified algorithm is correct? Fool me once and all that… Upon reflection it appears as if you have a heuristic (algorithm? that leverages logic, awareness, critical thinking, experience, a goal in mind, intuition, etc. to push towards better results. “It was able to reason intelligently” imbues qualities that I am skeptical is reasonable to attribute to this very narrow domain - what’s an example where it showed intelligent reasoning capabilities?
furyofantares 3y ago
But the can say "I don't know." They can be trained to do so ("as of my knowledge cutoff in September 2020 I don't know who Bob Whatgenflabl is") and they can be given context that makes it more likely they do so (I've had good success with this for RAG applications, and extremely little, but not zero, for general prompts.)
cultureswitch 3y ago
> The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match Can you tell that's not how you yourself function?
jncfhnb 3y ago
> They cannot say "I dont know" because they dont actually know anything. print(“I don’t know”) You don’t need proper cognition to identify that the answer is not stored in source data. Your conception of the model is incomplete as is easily demonstrable by testing such cases now. Chat gpt does just fine on your simpsons test. You, however, have made up an answer of how something works that you don’t actually know despite your cognition
- jijijijij 3y ago
  > to identify that the answer is not stored in source data How would an LLM do that?
  - jncfhnb 3y ago
    They do this already all the time. Probably the majority of the time. The problem is that a minority of the time is still very problematic. How do they do this? The same as they do now. The most likely token is that the bot doesn’t know the answer. Which is a behavior emergent from its tuning. I don’t get how people believe it can parse complex questions to produce novel ideas but can’t defer to saying “idk” when the answer isn’t known.
    - jijijijij 3y ago
      So, you are basing your assessment on your gut feel and personal impression with ChatGPT? Maybe you should tone down the spice a bit, then. Unless you can explain how an actual understanding emerges within an LLM, you can't explain how it would answer the question definitely - it doesn't know, if it does, or does not know something. Generally speaking.
      - jncfhnb 3y ago
        I’m basing it on my being a data scientist who does this. > Unless you can explain how an actual understanding emerges within an LLM Tuning creates the contextual framework on which test is mapped to a latent space that encodes the meaning and most likely next sequences of text rather than just raw most likely sequence of text as seen in training data. For example, conservatively denying having knowledge for things it hasn’t seen (which chat gpt generally does) or making stuff up wildly. > you can't explain how it would answer the question definitely Of course not. It’s a random behavior. It has plenty of flaws.
naasking 3y ago
> The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. Sorry, how do you know that "thinking minds" are not also just "complex pattern-fitting supercomputers hovering over a massive table of precomputed patterns"?

williamcotton 3y ago

What if you worked on the problem and tried to come up with some kind of solution?

sandworm101 3y ago
The solution is older non-AI tech. Google search can say "no good results found" because it returns actual data rather than creating anything new. If you want a hard answer about the presence or absence of something, AI isnt the correct tool.
- williamcotton 3y ago
  So there are no other possibilities for us other than using a system that can be gamed for substandard results? Are we sure about this?
- tempest_ 3y ago
  Can, but doesn't. I can't remember the last time google actually returned no results.
  - tsimionescu 3y ago
    It does reply with no results, but only for very long queries. E.g. If you search for two concatenated GUIDs, you can easily see a no results page.
  - johnny22 3y ago
    ah, i get no results pages often when i search for quoted error strings from many different sources. Thing is, I have a hard time believing that no one has actually talked about at least some of those errors :(

paulnpace 3y ago

My observation is that comments similar to GP come from the constant anthropomorphizing of things by marketers and without realizing that this subtle influence on language can alter one's view on what "I" means. The first time I really noticed this was when someone using Siri produced a response from Siri that included "I". Ever since I am acutely aware of this every time I hear or read it.

caditinpiscinam 3y ago

In real world conversations, people are constantly saying "I don't know"; but that doesn't really happen online. If you're on reddit or stack overflow or hacker news and you see a question you don't know the answer to, you normally just don't say anything. If LLMs are being trained on conversations pulled from the internet then they're missing out on a ton of uncertain responses. Maybe LLMs don't truly "understand" questions, but they're good at looking like they understand questions. If they were trained with more uncertain content, perhaps they'd be better at expressing uncertainty as well.

username332211 3y ago
If they were trained on more uncertain content, what happens if the most probable answer to a question is "I don't know", even though an answer exists in it's training set? Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? When the LLM answers "I don't know.", this could be a hallucination just as easily as anything else.
- deleted 3y ago
  [deleted]
- caditinpiscinam 3y ago
  > Suppose 99.3% of answers to 'What is the airspeed velocity of an unladen swallow?" are "I don't know that." and the remainder are "11 m/s". What would the model answer? I don't know :) Actually though, I think the best response would be to say that the answer to the question isn't clear, but that 11 m/s is sometimes given as an estimate. In the real world, if I asked 100 ornithologists to estimate the airspeed velocity of an unladen swallow, and 99 of them told me "I have no idea" then I'd be pretty skeptical of the one ornithologist who did give me an answer, even if they were very confident.
  - ta8645 3y ago
    I think the best response is to steal the joke and repeat it without comment. "Eleven meters per second." Full stop. It's humorous, and any reasonable interlocutor understands not to take it seriously. Of course, there are more serious questions that demand more serious answers. LLMs will eventually need to be able to understand the current context and assess the appropriate level of confidence required in any answer.

DiogenesKynikos 3y ago

> The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. Are you sure you're not also describing the human brain? At some point, after we have sufficiently demystified the workings of the human brain, it will probably also sound something like, "Well, the brain is just a large machine that does X, Y and Z [insert banal-sounding technical jargon from the future] - it doesn't really understand anything." My point here is that understanding ultimately comes down to having an effective internal model of the world, which is capable of taking novel inputs and generating reasonable descriptions of them or reactions to them. It turns out that LLMs are one way of achieving that. They don't function exactly like human brains, but they certainly do exhibit intelligence and understanding. I can ask an LLM a question that it has never seen before, and it will give me a reasonable answer that synthesizes and builds on various facts that it knows. Often the answer is more intelligent than what one would get from most humans. That's understanding.

beardedwizard 3y ago
Human brains form new connections dynamically. Llms are trained on connections human brains have already made. They never make new connections that aren't in training data. Nothing was synthesized, all the data was seen before and related to each other by vector similarity. It can just parrot the collective understanding humans already have and teach it.
- DiogenesKynikos 3y ago
  > It can just parrot the collective understanding humans already have and teach it. The problem with calling an LLM a parrot is that anyone who has actually interacted with an LLM knows that it produces completely novel responses to questions it has never seen before. These answers are usually logical and reasonable, based on both the information you gave the LLM and its previous knowledge of the world. Doing that requires understanding. > They never make new connections that aren't in training data. This is just categorically untrue. They make all sorts of logical connections that are not explicitly contained in the training data. Making logical inferences about subjects one has never heard about - based on the things one does know - is an expression of understanding. LLMs do that.

fl7305 3y ago

> a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns That was perhaps true of earlier and smaller LLMs, like GPT-1 and GPT-2. But as they grew larger and were trained with more and more data, they changed from pure pattern matching to implementing algorithms to compress more information into their structure than pure pattern matching can achieve. These algorithms are incomplete and buggy, but they are nonetheless executing algorithms, and not just pattern matching. This phenomenom can be seen in toy-sized neural networks. For instance, addition of two input values modulo a constant. As a small network is trained, at some point the internal structure can change from pattern matching to implementing addition using Fourier transforms. This is clearly visible in its structure. The network now performs the task perfectly for all inputs, regardless of having seen them in training. You can ask ChatGPT 4 to execute an algorithm for you. I just tried this one: I would like to play a game, where you are the host. We start off with a score that is 1234143143. At the start of each turn, you tell me the current score and ask me if I want to play a or b. If I choose a, the score is halved, and 30 is added. If I choose b, the score is doubled, and 40 is subtracted. Only use integers and round down. It will happily execute this algorithm. For large numbers, it is slightly off on the arithmetic. When I asked it to double check, it did so using Python code. After that, it kept using Python code to perform the math. It was also able to reason intelligently about different outcomes if always picking a (or b) given different starting points. Now, if you have enough memory and training data, of course you can build a gigantic lookup table that has this exact text sequence in it to replicate "executing the algorithm" I described. Is that your claim? How much memory are we talking about? My feeling is that it'd be far more than the number of atoms in the universe. PS Me: How far it is to the Simpsons house in Springfield? ChatGPT: The Simpsons' house in Springfield is a fictional location from the animated TV series "The Simpsons." Since Springfield is a fictional town and its location is not consistently specified in the series, it's not possible to determine a real-world distance to the Simpsons' house. Me: Do I have a brother? Please answer with either of: a) The answer b) There is an answer, but I do not know c) There is no answer ChatGPT: b) There is an answer, but I do not know

andsoitis 3y ago
> It will happily execute this algorithm. For large numbers, it is slightly off on the arithmetic. When I asked it to double check, it did so using Python code. After that, it kept using Python code to perform the math. It was also able to reason intelligently about different outcomes if always picking a (or b) given different starting points. Notice that you had to notice the error and had to prompt it to double check. Lots of complicated things going on here. Many (most?) humans will fail somewhere along this trajectory. Did it double check the Python code to make sure it is correct (not just in the sense that it is valid, executable code, but that it is the correct check in the first place)? Or did you double check that its modified algorithm is correct? Fool me once and all that… Upon reflection it appears as if you have a heuristic (algorithm? that leverages logic, awareness, critical thinking, experience, a goal in mind, intuition, etc. to push towards better results. “It was able to reason intelligently” imbues qualities that I am skeptical is reasonable to attribute to this very narrow domain - what’s an example where it showed intelligent reasoning capabilities?

furyofantares 3y ago

But the can say "I don't know." They can be trained to do so ("as of my knowledge cutoff in September 2020 I don't know who Bob Whatgenflabl is") and they can be given context that makes it more likely they do so (I've had good success with this for RAG applications, and extremely little, but not zero, for general prompts.)

cultureswitch 3y ago

> The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. It computes your input then looks to those patterns and spits out the best match Can you tell that's not how you yourself function?

jncfhnb 3y ago

> They cannot say "I dont know" because they dont actually know anything. print(“I don’t know”) You don’t need proper cognition to identify that the answer is not stored in source data. Your conception of the model is incomplete as is easily demonstrable by testing such cases now. Chat gpt does just fine on your simpsons test. You, however, have made up an answer of how something works that you don’t actually know despite your cognition

jijijijij 3y ago
> to identify that the answer is not stored in source data How would an LLM do that?
- jncfhnb 3y ago
  They do this already all the time. Probably the majority of the time. The problem is that a minority of the time is still very problematic. How do they do this? The same as they do now. The most likely token is that the bot doesn’t know the answer. Which is a behavior emergent from its tuning. I don’t get how people believe it can parse complex questions to produce novel ideas but can’t defer to saying “idk” when the answer isn’t known.
  - jijijijij 3y ago
    So, you are basing your assessment on your gut feel and personal impression with ChatGPT? Maybe you should tone down the spice a bit, then. Unless you can explain how an actual understanding emerges within an LLM, you can't explain how it would answer the question definitely - it doesn't know, if it does, or does not know something. Generally speaking.
    - jncfhnb 3y ago
      I’m basing it on my being a data scientist who does this. > Unless you can explain how an actual understanding emerges within an LLM Tuning creates the contextual framework on which test is mapped to a latent space that encodes the meaning and most likely next sequences of text rather than just raw most likely sequence of text as seen in training data. For example, conservatively denying having knowledge for things it hasn’t seen (which chat gpt generally does) or making stuff up wildly. > you can't explain how it would answer the question definitely Of course not. It’s a random behavior. It has plenty of flaws.

naasking 3y ago

> The answers are not comming from a thinking mind but a complex pattern-fitting supercomputer hovering over a massive table of precomputed patterns. Sorry, how do you know that "thinking minds" are not also just "complex pattern-fitting supercomputers hovering over a massive table of precomputed patterns"?