5 ms·

How many of these “machine does something humans do” articles are we going to get? It says a lot about how we’ve discovered core concepts and streamlined learni

How many of these “machine does something humans do” articles are we going to get? It says a lot about how we’ve discovered core concepts and streamlined learning that a machine can pass a test. Text is a wonderful medium for communication, and computers are finally somewhat up to it and it’s marvelous, but this is like producing solved sudokus — it doesn’t really create anything. ChatGPT isn’t an MBA any more than someone equipped with the answer key to the test is an MBA.

humanistbot 4y ago
Absolutely. You can Google the MBA questions and get directed to tons of pages that contain passing answers.
FormerBandmate 4y ago
IIRC it got a 1300 on the SAT
- csande17 4y ago
  That seems like the same situation -- it got 1300 on a practice test where a bunch of Internet sources discuss the exact questions and answers.
  - deleted 4y ago
    [deleted]
  - graeme 4y ago
    I tried feeding it an LSAT logic game which has multiple explanations on the internet. It got 2/5 correct, and for one answer it said “E, none of the above” E does not say none of the above on this logic game. Though many other different standardized tests do use E as none of the above. So it was pattern matching to that and not the text in front of it. And it wasn’t using explanations
zone411 4y ago
Oh how quickly the goalposts shift... If the test was done correctly and there was no data leakage, this absolutely does not compare to someone equipped with the answer key.
- salamo 4y ago
  There's even a specific term for shifting AI goalposts: https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect
  - chamwislothe2nd 4y ago
    https://en.m.wikipedia.org/wiki/Moving_the_goalposts https://en.m.wikipedia.org/wiki/Moving_the_goalposts reviewing the 'see also' section of the goalposts wikipage is like looking at a syllabus for PR. People are terrible.
  - bamboozled 4y ago
    I also think there is a novel new effect happening which is that if one question anything about AI. For example: I tired to use Dall-E for something useful and it sucked. You get labelled...Anti-AI. Being critical about certain things isn't allowed at the moment. Kind of like the 2023 replacement for Anti-vaxxer if you don't want to take medicine every 3 months. My honest opinion, is that while a lot of this technology is cool and novel, it's not perfect, it has uses and then it doesn't. What it does have is a lot of good marketing and PR behind it and so you're up against a lot of that too. One example I like is the Dall-e homepage shoes the curated "best" images generated, which might be showing only 1 in 1000 or more of the best images. I was excited when I first tried it based on the front page. But now it's drawing people with feet for hands for me and it's super super fun and entertaining to play with, I don't feel it works quite as advertised most of the time. Disclaimer: Not anti-ai and I took my Covid19 vaccines (in the beginning, no boosters)
- ipaddr 4y ago
  It compares to someone with a textbook and/or access to the internet.
janalsncm 4y ago
Probably a lot more? Part of it is we don’t have a definition of intelligence, so the best we can do is measure machines against our tests since passing tests is a thing intelligent people can do. This kind of performance (even if it’s less glamorous upon closer inspection) puts symbolic AI folks from the 80s to shame. That’s the whole point of Attention is All You Need, the paper that opened the door to web-scale language models. No, you don’t need a brittle and cumbersome knowledge base, at least not for this. MBA professors are perfectly happy accepting an autocomplete on steroids.
jxramos 4y ago
* ChatGPT wins the Nobel Peace Prize * ChatGPT wins the Nobel Prize for Literature * ChatGPT wins a Ford Foundation grant for the arts
- umeshunni 4y ago
  To be fair, the peace price has a pretty low bar.
- belter 4y ago
  * ChatGPT creates 50 new Startups who fill all seats for the new YC 2023 batch...
- mc32 4y ago
  Marshall ChatGPT leads Allied Armies to victory! [all ranks between Army General and Captain, non inclusive, have been declared redundant and eliminated with ChatGPT absorbing those ranks]

humanistbot 4y ago

Absolutely. You can Google the MBA questions and get directed to tons of pages that contain passing answers.

FormerBandmate 4y ago

IIRC it got a 1300 on the SAT

csande17 4y ago
That seems like the same situation -- it got 1300 on a practice test where a bunch of Internet sources discuss the exact questions and answers.
- deleted 4y ago
  [deleted]
- graeme 4y ago
  I tried feeding it an LSAT logic game which has multiple explanations on the internet. It got 2/5 correct, and for one answer it said “E, none of the above” E does not say none of the above on this logic game. Though many other different standardized tests do use E as none of the above. So it was pattern matching to that and not the text in front of it. And it wasn’t using explanations

zone411 4y ago

Oh how quickly the goalposts shift... If the test was done correctly and there was no data leakage, this absolutely does not compare to someone equipped with the answer key.

salamo 4y ago
There's even a specific term for shifting AI goalposts: https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect
- chamwislothe2nd 4y ago
  https://en.m.wikipedia.org/wiki/Moving_the_goalposts https://en.m.wikipedia.org/wiki/Moving_the_goalposts reviewing the 'see also' section of the goalposts wikipage is like looking at a syllabus for PR. People are terrible.
- bamboozled 4y ago
  I also think there is a novel new effect happening which is that if one question anything about AI. For example: I tired to use Dall-E for something useful and it sucked. You get labelled...Anti-AI. Being critical about certain things isn't allowed at the moment. Kind of like the 2023 replacement for Anti-vaxxer if you don't want to take medicine every 3 months. My honest opinion, is that while a lot of this technology is cool and novel, it's not perfect, it has uses and then it doesn't. What it does have is a lot of good marketing and PR behind it and so you're up against a lot of that too. One example I like is the Dall-e homepage shoes the curated "best" images generated, which might be showing only 1 in 1000 or more of the best images. I was excited when I first tried it based on the front page. But now it's drawing people with feet for hands for me and it's super super fun and entertaining to play with, I don't feel it works quite as advertised most of the time. Disclaimer: Not anti-ai and I took my Covid19 vaccines (in the beginning, no boosters)
ipaddr 4y ago
It compares to someone with a textbook and/or access to the internet.

janalsncm 4y ago

Probably a lot more? Part of it is we don’t have a definition of intelligence, so the best we can do is measure machines against our tests since passing tests is a thing intelligent people can do. This kind of performance (even if it’s less glamorous upon closer inspection) puts symbolic AI folks from the 80s to shame. That’s the whole point of Attention is All You Need, the paper that opened the door to web-scale language models. No, you don’t need a brittle and cumbersome knowledge base, at least not for this. MBA professors are perfectly happy accepting an autocomplete on steroids.

jxramos 4y ago

* ChatGPT wins the Nobel Peace Prize * ChatGPT wins the Nobel Prize for Literature * ChatGPT wins a Ford Foundation grant for the arts

umeshunni 4y ago
To be fair, the peace price has a pretty low bar.
belter 4y ago
* ChatGPT creates 50 new Startups who fill all seats for the new YC 2023 batch...
mc32 4y ago
Marshall ChatGPT leads Allied Armies to victory! [all ranks between Army General and Captain, non inclusive, have been declared redundant and eliminated with ChatGPT absorbing those ranks]