5 ms·

I did a set up where one of the tools I gave the LLM agent was a "human expert" (which was just a wrapper around Python's input() method). And lo and behold, th

I did a set up where one of the tools I gave the LLM agent was a "human expert" (which was just a wrapper around Python's input() method). And lo and behold, the LLM would use the "human" tool when it was stumped. So yes, science has definitely gone too far.

swatcoder 4y ago
By “when it was stumped” I assume you mean “when the text completion ranked invoking a human expert as a more likely next phrase than an immediate answer” It doesn’t invoke your human expert when it’s “stumped”. It invokes it when the question associates with training data where invoking an expert is more likely (i.e complicated question, esoteric questions, etc) — it has no knowledge and no insight into your question or its answer; it just can imitate that certain looking questions are followed by expert referrals. We really need to stop using the wrong language for these things.
- flangola7 4y ago
  We need to realize that the sum may be greater than its parts. I'm not aware of any concrete evidence that says text token prediction can't be just a good of a path to intelligence as whatever "code" our wetware is running. For something that has "no knowledge and no insight" it sure is damn good at taking unique social, technical, and other problems I have and offering an array of solutions, with pros/cons and probable outcomes of each.
  - swatcoder 4y ago
    > For something that has "no knowledge and no insight" it sure is damn good at taking unique social, technical, and other problems I have and offering an array of solutions, with pros/cons and probable outcomes of each. What that might be telling you is that your problems aren’t so unique after all. Getting a sound answer from ChatGPT is very literally like looking up some weird concern that never occurred to you before in a search engine and finding a hundred upvotes and a three page long discussion from 2012. Those upvotes and comments that happen to come up in Google are only a tiny share of the online and offline encounters with that same concern, and tell you that your completely novel-to-you problem is actually not so novel after all. For that class of problem, ChatGPT can usefully synthesize content that can almost always look right and often be accurate. Just like Google, it has a huge database of source material and just happens to synthesize an original text similar to many samples in it rather than pointing out one specific sample. But as you dig into more specificity, idiosyncrasy, and esoteric “problems”, the reference material gets too thin and it starts to generate noise because it lacks alternative ways of reasoning or knowing. There might be another leap forward in the next few years, but this generation of tools only do what they do.
    - chc 4y ago
      I tried asking ChatGPT easy but completely off-the-wall questions and it consistently got the answers surprisingly right. For example, I asked it "Which is more powerful, a DEC VAX or a GTX 1070?" and it both noted that this comparison was questionable, as the two are not used in the same context, and correctly answered that the GeForce has much more processing power than a VAX. There are no Google results that directly answer that question, and yet ChatGPT knew that was what it should say. So while it is obviously just guessing what to say, the idea that it is just regurgitating existing material seems wrong to me. It does some pretty impressive synthesis to come up with its answers.
      - jcranmer 4y ago
        I wonder how much of that is because the way such questions are asked is so formulaic that it's possible to "guess" the response from that information. If you were to ask a truly nonsense question, like "Which is more powerful, a GTX 1070 or an F-16?", what would the response be?
        azeirah 4y ago
        "It's difficult to compare a graphics card like the GTX 1070 and an aircraft like the F-16 as they are designed for very different purposes and operate on completely different scales. The GTX 1070 is a powerful graphics card that is designed to render high-quality graphics for gaming, video editing, and other visual computing applications. It has impressive processing power and can handle complex graphics and visual effects with ease. On the other hand, the F-16 is a highly advanced fighter aircraft designed for air-to-air and air-to-ground combat missions. It is a complex machine that requires a highly skilled pilot to operate and is equipped with state-of-the-art avionics, weapons systems, and engines. In terms of raw power, the F-16 is undoubtedly more powerful than the GTX 1070, with its advanced capabilities in speed, agility, and firepower. However, it's important to note that these are two very different types of machines, and a direct comparison of their power is not really possible."
        4y ago
  - freejazz 4y ago
    Damn, my magic 8 ball can't give me probable outcomes! How good are chatGPT's by the way? what are the odds I fart in the office tomorrow?
    - Karellen 4y ago
      > what are the odds I fart in the office tomorrow? Outlook not so good.
      - freejazz 4y ago
        So far so good, but it's early!
  - geraneum 4y ago
    "what can be asserted without evidence can also be dismissed without evidence." https://en.m.wikipedia.org/wiki/Hitchens's_razor https://en.m.wikipedia.org/wiki/Hitchens's_razor

swatcoder 4y ago

By “when it was stumped” I assume you mean “when the text completion ranked invoking a human expert as a more likely next phrase than an immediate answer” It doesn’t invoke your human expert when it’s “stumped”. It invokes it when the question associates with training data where invoking an expert is more likely (i.e complicated question, esoteric questions, etc) — it has no knowledge and no insight into your question or its answer; it just can imitate that certain looking questions are followed by expert referrals. We really need to stop using the wrong language for these things.

flangola7 4y ago
We need to realize that the sum may be greater than its parts. I'm not aware of any concrete evidence that says text token prediction can't be just a good of a path to intelligence as whatever "code" our wetware is running. For something that has "no knowledge and no insight" it sure is damn good at taking unique social, technical, and other problems I have and offering an array of solutions, with pros/cons and probable outcomes of each.
- swatcoder 4y ago
  > For something that has "no knowledge and no insight" it sure is damn good at taking unique social, technical, and other problems I have and offering an array of solutions, with pros/cons and probable outcomes of each. What that might be telling you is that your problems aren’t so unique after all. Getting a sound answer from ChatGPT is very literally like looking up some weird concern that never occurred to you before in a search engine and finding a hundred upvotes and a three page long discussion from 2012. Those upvotes and comments that happen to come up in Google are only a tiny share of the online and offline encounters with that same concern, and tell you that your completely novel-to-you problem is actually not so novel after all. For that class of problem, ChatGPT can usefully synthesize content that can almost always look right and often be accurate. Just like Google, it has a huge database of source material and just happens to synthesize an original text similar to many samples in it rather than pointing out one specific sample. But as you dig into more specificity, idiosyncrasy, and esoteric “problems”, the reference material gets too thin and it starts to generate noise because it lacks alternative ways of reasoning or knowing. There might be another leap forward in the next few years, but this generation of tools only do what they do.
  - chc 4y ago
    I tried asking ChatGPT easy but completely off-the-wall questions and it consistently got the answers surprisingly right. For example, I asked it "Which is more powerful, a DEC VAX or a GTX 1070?" and it both noted that this comparison was questionable, as the two are not used in the same context, and correctly answered that the GeForce has much more processing power than a VAX. There are no Google results that directly answer that question, and yet ChatGPT knew that was what it should say. So while it is obviously just guessing what to say, the idea that it is just regurgitating existing material seems wrong to me. It does some pretty impressive synthesis to come up with its answers.
    - jcranmer 4y ago
      I wonder how much of that is because the way such questions are asked is so formulaic that it's possible to "guess" the response from that information. If you were to ask a truly nonsense question, like "Which is more powerful, a GTX 1070 or an F-16?", what would the response be?
      - azeirah 4y ago
        "It's difficult to compare a graphics card like the GTX 1070 and an aircraft like the F-16 as they are designed for very different purposes and operate on completely different scales. The GTX 1070 is a powerful graphics card that is designed to render high-quality graphics for gaming, video editing, and other visual computing applications. It has impressive processing power and can handle complex graphics and visual effects with ease. On the other hand, the F-16 is a highly advanced fighter aircraft designed for air-to-air and air-to-ground combat missions. It is a complex machine that requires a highly skilled pilot to operate and is equipped with state-of-the-art avionics, weapons systems, and engines. In terms of raw power, the F-16 is undoubtedly more powerful than the GTX 1070, with its advanced capabilities in speed, agility, and firepower. However, it's important to note that these are two very different types of machines, and a direct comparison of their power is not really possible."
        4y ago
- freejazz 4y ago
  Damn, my magic 8 ball can't give me probable outcomes! How good are chatGPT's by the way? what are the odds I fart in the office tomorrow?
  - Karellen 4y ago
    > what are the odds I fart in the office tomorrow? Outlook not so good.
    - freejazz 4y ago
      So far so good, but it's early!
- geraneum 4y ago
  "what can be asserted without evidence can also be dismissed without evidence." https://en.m.wikipedia.org/wiki/Hitchens's_razor https://en.m.wikipedia.org/wiki/Hitchens's_razor