3 ms·

> As I see it, there is clearly no way to advance ChatGPT into anything more than it is today. Impressive as it is, the curtain is wide open for all to see, and

> As I see it, there is clearly no way to advance ChatGPT into anything more than it is today. Impressive as it is, the curtain is wide open for all to see, and the art can be viewed plainly as what it truly is: magic, and nothing more. (I only RTFA after writing this comment, and I now see that the below is what they're doing) I'm an outsider to this field. My unexpert thought was that perhaps this model could be used to identify the maths components of that question and then we (programmatically) feed that into a different system that gives the answer. That answer then could be provided as context when asking the real question, so the model has access to the mathematical facts. E.g., I just put this question into the playground: "Bob was asked to add 234241.24211 and 58352342.52544, and he wanted to know the result. What is the result?" Identify the mathematical question in the preceding question and write it as an equation. The answer I got back was: 234241.24211 + 58352342.52544 = ? That could then be fed into a different system that is designed to do mathematical calculations (or logic). In short, not by doing more of what has been done so far, but instead combining these models with different systems. We take the result of that sum from a separate system, and redo the same question, providing the mathematical details as context. E.g., now asking: 234241.24211 + 58352342.52544 = 58586583.76755 Bob was asked to add 234241.24211 and 58352342.52544, and he wanted to know the result. What is the result? With the response: The result is 58586583.76755. Thus getting mathematically accurate answers. Note that if I don't include this context with the calculation completed, I get back the answer: 6,179,563.76754

amelius 4y ago
You are asking ChatGPT to interface with a calculator. This is sort of what Wolfram proposes too: interface ChatGPT with a math language. It could work for simple problems. But extracting a mathematical description from a piece of text can require logic too, I suppose, which is where the difficulties start.
thomastjeffery 4y ago
You would be trading the problem for another instance of the same problem. When you ask ChatGPT to construct a mathematical question, it will do so the same way it does everything else: by semantic popularity. And that is the problem we are trying to avoid. The semantically popular guess might be logically sound, but it might not. It's a gamble no matter when or where it is done. -- All it takes is what I call a "semantic off-by-one error". That might look like our first problem: > Bob was asked to add 234241.24211 and 58352342.52544, and he wanted to know the result. What is the result? The problem is that a semantically close response is nothing like "234241.24211 + 58352342.52544". It's just going to be whatever arbitrary text that already exists in the training dataset is closest to the semantic phrasing of the question. That might be the correct number, is more likely to be an incorrect number, and is even likely to be a wordy response. -- So if we follow your thought process to interrupt the guesswork, it would involve restructuring the prompt. > Please restructure the following prompt into a mathematical statement: "Bob was asked to add 234241.24211 and 58352342.52544, and he wanted to know the result. What is the result?" The output you are hoping for >> 234241.24211 + 58352342.52544 Another completely valid and possible output: >> 234241.24211 / 58352342.52544 Another: >> 2424332.34434 + 535858932.5358 -- There is no way to guarantee the reformulated question is logically equivalent to its original. That's the problem, and the problem cannot be moved. With every step, ChatGPT is guessing. ChatGPT cannot do anything at all without making a guess, because "guess" is everything that ChatGPT is. The only place you can pause an interaction with ChatGPT to do some logic is instead.
- Winsaucerer 4y ago
  > It's just going to be whatever arbitrary text that already exists in the training dataset is closest to the semantic phrasing of the question. My understanding of machine learning in general is that this is not how it works, and rather it uses a neural network for a lot of what it does (which isn't merely picking the closest arbitrary text in its training set), though I don't know details about NLP specifically. I am aware that some activities involve ranking the similarity of particular texts, and picking the closest matches (that's how embeddings get used for providing context to prompts), but I didn't think that's how the underlying models operate when, say, asked the question I asked it. Regarding the rest of your remarks, the aim isn't necessarily perfection. You listed some other possible outputs, but the fact is that it didn't offer these options, and instead it offered the correct one, and it seems that a system that uses ChatGPT + something else can provide an improvement over ChatGPT on its own (with the downside being that it requires more resources to provide those answers). Is the other "completely valid and possible output" you provided actually likely? Even a human could possibly make the mistakes you list, but possible doesn't mean probable, and being possible doesn't take away from the results of such a hybrid system being an improvement over ChatGPT on its own. I didn't think guarantees are the game here. Achieving something closer to what humans are (and even some humans struggle with logic and mathematics), and even exceeding it in some places, would be of immense value. > ChatGPT cannot do anything at all without making a guess, because "guess" is everything that ChatGPT is Is this different to what humans do? There are accounts of the brain where it takes sensory input, updates an internal model, and makes predictions (guesses) about what inputs it expects to receive. Getting better at guessing might be the right game to play. i.e., it's not a criticism to say it's guessing, because that's what these models are trying to do. If it can guess reasonably well when a question is mathematical or logical, and can guess reasonably well about how to structure a mathematical or logical equation or statement based on that text, then that could be fed into another system to produce a more accurate answer.
  - thomastjeffery 4y ago
    > but the fact is that it didn't offer these options, and instead it offered the correct one ...because you gave it a "correct" prompt; or in other words, you gave it a prompt that ChatGPT can respond "successfully" to. But if you keep trying, you will surely find yourself with "failed" responses. And ChatGPT has no way of knowing the difference. That's my point. > Is the other "completely valid and possible output" you provided actually likely? We're splitting hairs here. I didn't go and test a bunch of prompts to find an example failure. You are welcome to do that if you want. It's not very hard, especially when you start adding variables. This is well-known behavior that ChatGPT itself warns its users about. > > ChatGPT cannot do anything at all without making a guess, because "guess" is everything that ChatGPT is > Is this different to what humans do? Yes! We don't just spit out the first thing that comes to mind, unless that's the task we are intending to do - and anyone who has played music can tell you just how difficult that task is. Most of the time, we conceptualize. We comprehend the meaning of language - sure, it takes some guesswork along the way, but that's not the main event - then we construct a new idea, then we express it. We don't just want accurate answers. We want conceptualization. That's the feature we are all dreaming about with Natural Language Processors. If we can teach a computer to conceptualize language, then we can get it to interface with any arbitrary human knowledge! But ChatGPT doesn't interface. It parrots; and that's an insult to the birds themselves - unlike ChatGPT, they have an intelligence of their own. The only way ChatGPT could give you a valid response about any arbitrary concept is to write every thing from every angle about every concept ever, and feed it all to ChatGPT as training data. That's exactly the work we want to avoid!

amelius 4y ago

You are asking ChatGPT to interface with a calculator. This is sort of what Wolfram proposes too: interface ChatGPT with a math language. It could work for simple problems. But extracting a mathematical description from a piece of text can require logic too, I suppose, which is where the difficulties start.

thomastjeffery 4y ago

You would be trading the problem for another instance of the same problem. When you ask ChatGPT to construct a mathematical question, it will do so the same way it does everything else: by semantic popularity. And that is the problem we are trying to avoid. The semantically popular guess might be logically sound, but it might not. It's a gamble no matter when or where it is done. -- All it takes is what I call a "semantic off-by-one error". That might look like our first problem: > Bob was asked to add 234241.24211 and 58352342.52544, and he wanted to know the result. What is the result? The problem is that a semantically close response is nothing like "234241.24211 + 58352342.52544". It's just going to be whatever arbitrary text that already exists in the training dataset is closest to the semantic phrasing of the question. That might be the correct number, is more likely to be an incorrect number, and is even likely to be a wordy response. -- So if we follow your thought process to interrupt the guesswork, it would involve restructuring the prompt. > Please restructure the following prompt into a mathematical statement: "Bob was asked to add 234241.24211 and 58352342.52544, and he wanted to know the result. What is the result?" The output you are hoping for >> 234241.24211 + 58352342.52544 Another completely valid and possible output: >> 234241.24211 / 58352342.52544 Another: >> 2424332.34434 + 535858932.5358 -- There is no way to guarantee the reformulated question is logically equivalent to its original. That's the problem, and the problem cannot be moved. With every step, ChatGPT is guessing. ChatGPT cannot do anything at all without making a guess, because "guess" is everything that ChatGPT is. The only place you can pause an interaction with ChatGPT to do some logic is instead.

Winsaucerer 4y ago
> It's just going to be whatever arbitrary text that already exists in the training dataset is closest to the semantic phrasing of the question. My understanding of machine learning in general is that this is not how it works, and rather it uses a neural network for a lot of what it does (which isn't merely picking the closest arbitrary text in its training set), though I don't know details about NLP specifically. I am aware that some activities involve ranking the similarity of particular texts, and picking the closest matches (that's how embeddings get used for providing context to prompts), but I didn't think that's how the underlying models operate when, say, asked the question I asked it. Regarding the rest of your remarks, the aim isn't necessarily perfection. You listed some other possible outputs, but the fact is that it didn't offer these options, and instead it offered the correct one, and it seems that a system that uses ChatGPT + something else can provide an improvement over ChatGPT on its own (with the downside being that it requires more resources to provide those answers). Is the other "completely valid and possible output" you provided actually likely? Even a human could possibly make the mistakes you list, but possible doesn't mean probable, and being possible doesn't take away from the results of such a hybrid system being an improvement over ChatGPT on its own. I didn't think guarantees are the game here. Achieving something closer to what humans are (and even some humans struggle with logic and mathematics), and even exceeding it in some places, would be of immense value. > ChatGPT cannot do anything at all without making a guess, because "guess" is everything that ChatGPT is Is this different to what humans do? There are accounts of the brain where it takes sensory input, updates an internal model, and makes predictions (guesses) about what inputs it expects to receive. Getting better at guessing might be the right game to play. i.e., it's not a criticism to say it's guessing, because that's what these models are trying to do. If it can guess reasonably well when a question is mathematical or logical, and can guess reasonably well about how to structure a mathematical or logical equation or statement based on that text, then that could be fed into another system to produce a more accurate answer.
- thomastjeffery 4y ago
  > but the fact is that it didn't offer these options, and instead it offered the correct one ...because you gave it a "correct" prompt; or in other words, you gave it a prompt that ChatGPT can respond "successfully" to. But if you keep trying, you will surely find yourself with "failed" responses. And ChatGPT has no way of knowing the difference. That's my point. > Is the other "completely valid and possible output" you provided actually likely? We're splitting hairs here. I didn't go and test a bunch of prompts to find an example failure. You are welcome to do that if you want. It's not very hard, especially when you start adding variables. This is well-known behavior that ChatGPT itself warns its users about. > > ChatGPT cannot do anything at all without making a guess, because "guess" is everything that ChatGPT is > Is this different to what humans do? Yes! We don't just spit out the first thing that comes to mind, unless that's the task we are intending to do - and anyone who has played music can tell you just how difficult that task is. Most of the time, we conceptualize. We comprehend the meaning of language - sure, it takes some guesswork along the way, but that's not the main event - then we construct a new idea, then we express it. We don't just want accurate answers. We want conceptualization. That's the feature we are all dreaming about with Natural Language Processors. If we can teach a computer to conceptualize language, then we can get it to interface with any arbitrary human knowledge! But ChatGPT doesn't interface. It parrots; and that's an insult to the birds themselves - unlike ChatGPT, they have an intelligence of their own. The only way ChatGPT could give you a valid response about any arbitrary concept is to write every thing from every angle about every concept ever, and feed it all to ChatGPT as training data. That's exactly the work we want to avoid!