5 ms·

> So, ChatGPT seems to understand how filesystems work, how files are stored and can be retrieved later. It understands that linux machines are stateful, and co

by RC_ITR 4y ago

> So, ChatGPT seems to understand how filesystems work, how files are stored and can be retrieved later. It understands that linux machines are stateful, and correctly retrieves this information and displays it. No. ChatGPT has seen millions of file systems and can replicate them. We want to believe it “understands” because we are so used to writing being a representation of understanding (since that’s how human intelligence works). If you asked it an easy to answer question (based on the rules of file systems) that isn’t part of its training dataset, it will fail miserably.

kilgnad 4y ago
Did you follow that article to the end? It goes way beyond just filesystems. Filesystems are just some minor blip in that full story. Go read it. chatGPT is not trained on what a filesystem is. It's inferred what a file system is and how it should act based off of scraped text from the internet. But again this is a minor point. Finish the article. Trust me.
- deleted 4y ago
  [deleted]
- _dain_ 4y ago
  I read the article to the end and I don't see why the ending part is any more impressive than the beginning. We already know it can talk about itself, and write fiction featuring itself as a participant. Doing it in a faked linux terminal is just window dressing.
  - Sai_ 4y ago
    I agree with you that the blog’s outcomes seem quite mundane once you realise that chatGPT is simply parroting what it has seen as the output of a terminal. Not sure why kilgnad is presenting this post as if they’ve seen The Truth. At the same time, isn’t learning also remembered knowledge remixed to respond to a situation. E.g., if I were asked to write a haiku, I’d draw on my memory of haiku structure and haiku samples to come up with an approximation of a haiku. Isn’t that what chatGPT is also doing except with ping, lynx, and ls outputs?
    - RC_ITR 4y ago
      There’s various kinds of learning (and to be clear transformer-based LLMs do sort of “learn”). ChatGPT is a really finely tuned probability model (the tuning came during the training and that’s what I would equate to learning), but it lacks understanding. The example that’s most glaring is asking it simple math (or in Dall-E’s case, asking it to generate images of text). You’ll get outputs that are definitely in the correct form, but you won’t get the right answer, because the model links words that it’s seen in a particular before, it is ignorant to the rules of math. The funny caveat is that equations like 2+2 are so common that it will get that right, but two random four digit numbers (rarer in text) will almost always be wrong. That’s why ultimately, AI is an outcome and we will need many different tools working in concert to achieve it (to be fair, the same is true of your brain, there are plenty of uniquely architectured parts of the human brain).
      - kilgnad 4y ago
        Well the AI has to infer what math is with many examples. You give it enough examples it will be able to calculate equations it's never seen before. But when you don't give it enough examples it does a best guess. This is what you would also do as a human if you were trying to infer the concept of Arabic numerals and math from a series of random examples. I would say based on what you have hear dall-e2 has a misunderstanding of math. Which would be isomorphic to our misunderstanding of math if we were to be put in the exact same situation.
        RC_ITR 4y ago
        >Well the AI has to infer what math is with many examples. You give it enough examples it will be able to calculate equations it's never seen before. This is categorically not true and an important distinction to understand. All OpenAI products are based on GPT, which is a language model, not a general learning model. You could feed it every math textbook in existence and the model will learn absolutely no math, other than how to repeat math that others have done for it (and the point of multiplying 4 digit numbers is that it's nearly impossible to brute force that). This is an extremely important distinction - we assume being able to describe math means that something understands math (because that is how humans work), but math is rules-based and GPT is stochastic, so it can sound like it understands math, but it does not. Now, it would be trivial to put a filter on that recognizes equations and returns an output based on regular rules-based algorithms (Google search's LLMs do this for example), but this again points out that AI is an outcome and learning isn't the solution to the problem, so much as synthesis is (again, the brain metaphor is that no matter how well trained your pre-frontal cortex is, it will be bad at making your heart beat all the time).
  - kilgnad 4y ago
    If we already know this then we already know it's self aware. No training data exists for faking itself in a Linux terminal. The concept of understanding must exist for chatGPT to create this concept out of thin air.
- RC_ITR 4y ago
  You’re not supposed to ask people if they read the article on here, because usually the answer is yes (and you should expect it to be). I did read it. The point of neural nets is that they are really good at probabilisticly linking tokens (in this case words with some embedded positional encoding). That is not the same as understanding file systems. If you look at any architectural overview of GPT-3, you’ll see it’s just really good at linking words together in order. Even OpenAI will admit it doesn’t know anything outside its dataset (try asking it weather questions).
  - kilgnad 4y ago
    [dead]

by RC_ITR 4y ago

kilgnad 4y ago

Did you follow that article to the end? It goes way beyond just filesystems. Filesystems are just some minor blip in that full story. Go read it. chatGPT is not trained on what a filesystem is. It's inferred what a file system is and how it should act based off of scraped text from the internet. But again this is a minor point. Finish the article. Trust me.

deleted 4y ago
[deleted]
_dain_ 4y ago
I read the article to the end and I don't see why the ending part is any more impressive than the beginning. We already know it can talk about itself, and write fiction featuring itself as a participant. Doing it in a faked linux terminal is just window dressing.
- Sai_ 4y ago
  I agree with you that the blog’s outcomes seem quite mundane once you realise that chatGPT is simply parroting what it has seen as the output of a terminal. Not sure why kilgnad is presenting this post as if they’ve seen The Truth. At the same time, isn’t learning also remembered knowledge remixed to respond to a situation. E.g., if I were asked to write a haiku, I’d draw on my memory of haiku structure and haiku samples to come up with an approximation of a haiku. Isn’t that what chatGPT is also doing except with ping, lynx, and ls outputs?
  - RC_ITR 4y ago
    There’s various kinds of learning (and to be clear transformer-based LLMs do sort of “learn”). ChatGPT is a really finely tuned probability model (the tuning came during the training and that’s what I would equate to learning), but it lacks understanding. The example that’s most glaring is asking it simple math (or in Dall-E’s case, asking it to generate images of text). You’ll get outputs that are definitely in the correct form, but you won’t get the right answer, because the model links words that it’s seen in a particular before, it is ignorant to the rules of math. The funny caveat is that equations like 2+2 are so common that it will get that right, but two random four digit numbers (rarer in text) will almost always be wrong. That’s why ultimately, AI is an outcome and we will need many different tools working in concert to achieve it (to be fair, the same is true of your brain, there are plenty of uniquely architectured parts of the human brain).
    - kilgnad 4y ago
      Well the AI has to infer what math is with many examples. You give it enough examples it will be able to calculate equations it's never seen before. But when you don't give it enough examples it does a best guess. This is what you would also do as a human if you were trying to infer the concept of Arabic numerals and math from a series of random examples. I would say based on what you have hear dall-e2 has a misunderstanding of math. Which would be isomorphic to our misunderstanding of math if we were to be put in the exact same situation.
      - RC_ITR 4y ago
        >Well the AI has to infer what math is with many examples. You give it enough examples it will be able to calculate equations it's never seen before. This is categorically not true and an important distinction to understand. All OpenAI products are based on GPT, which is a language model, not a general learning model. You could feed it every math textbook in existence and the model will learn absolutely no math, other than how to repeat math that others have done for it (and the point of multiplying 4 digit numbers is that it's nearly impossible to brute force that). This is an extremely important distinction - we assume being able to describe math means that something understands math (because that is how humans work), but math is rules-based and GPT is stochastic, so it can sound like it understands math, but it does not. Now, it would be trivial to put a filter on that recognizes equations and returns an output based on regular rules-based algorithms (Google search's LLMs do this for example), but this again points out that AI is an outcome and learning isn't the solution to the problem, so much as synthesis is (again, the brain metaphor is that no matter how well trained your pre-frontal cortex is, it will be bad at making your heart beat all the time).
- kilgnad 4y ago
  If we already know this then we already know it's self aware. No training data exists for faking itself in a Linux terminal. The concept of understanding must exist for chatGPT to create this concept out of thin air.
RC_ITR 4y ago
You’re not supposed to ask people if they read the article on here, because usually the answer is yes (and you should expect it to be). I did read it. The point of neural nets is that they are really good at probabilisticly linking tokens (in this case words with some embedded positional encoding). That is not the same as understanding file systems. If you look at any architectural overview of GPT-3, you’ll see it’s just really good at linking words together in order. Even OpenAI will admit it doesn’t know anything outside its dataset (try asking it weather questions).
- kilgnad 4y ago
  [dead]