5 ms·
Honest question, not meaning to disagree with you: how do you view previous predictions of the success of AI, such as Marvin Minsky predicting in the 60s that A
by cmdli 3y ago
Honest question, not meaning to disagree with you: how do you view previous predictions of the success of AI, such as Marvin Minsky predicting in the 60s that AI “would substantially be solved in a generation”? What reasons would there be that the experts are correct this time?
- famouswaffles 3y agoCurrent Capability would be the biggest one. We're at the point where any testable definitions of GI that the sota LLM fails (GPT-4) is also failed by a good chunk of humans. You couldn't say that a few years ago nevermind 60. What we have now (so no hypotheticals) coupled with the fact that scaling hasn't yet shown any performance walls makes a pretty good shout that things will probably be different this time.
- skepticATX 3y agoThat's not really true though. LLMs are abysmal at planning, for example. Something that comes quite naturally to humans.
- runsWphotons 3y agoThey are probably better than 10% of people
- danielmarkbruce 3y agoYou meant 70%, right?
- famouswaffles 3y agoThey're really only abysmal if you attempt to one-shot it and probe with tasks that would require a human a scratchpad to accomplish. Humans can't one-shot non trivial planning tasks either. It's the one problem i have with all the papers that try to evaluate planning for LLMs. Step away from that approach and they're ok. https://innermonologue.github.io/ https://innermonologue.github.io/ https://tidybot.cs.princeton.edu/ https://tidybot.cs.princeton.edu/
- pulvinar 3y agoI'm curious as to your source, or particular examples, since they (or at least GPT-4) seem to me to be rather decent at planning. E.g., for writing code.
- climatologist 3y agoAsk it to write a backtracking sudoku solver with coroutines and/or fibers and let me know how it performs in your language of choice. We are nowhere near generally intelligent software systems.
- jtmoulia 3y agoI was curious where GPT-4 would come up short on the problem and I was surprised -- it seemed to solve it pretty well whether or not using coroutines. (I dropped both solns into a python interpreter and both appeared to solve the problem.) There could def be bugs I missed tho. https://chat.openai.com/share/ef77507e-cb75-4112-97f1-a16cfc03cd98 https://chat.openai.com/share/ef77507e-cb75-4112-97f1-a16cfc...
- climatologist 3y agoThat's a good attempt but the coroutine solution is incorrect. See if you can figure out why and how to improve it. You can also ask it to propagate constraints and see what happens.
- el_nahual 3y agoThis isn't the dunk you think it is since GP is a human (I presume), and he thought the solution worked.
- jtmoulia 3y agoSigh classic LLM -- without you, the expert, I can't quickly tell from the code / output how the answer the LLM produced is wrong. I also asked it to solve sudoku by "propagating constraints" and the answer seemed to work for me :/ Again, I'd guess the soln produced is wrong because I trust you more than the LLM but I don't have the mental horsepower to figure it out without resorting to tests & debugging.