8 ms·

2022 Letter

143 points by oli5679 4y ago

mdorazio 4y ago
Do any mainstream AGI researchers believe that GPT-style ML methods will plausibly get us to AGI? I have only a very shallow understanding of the state of the art, but from the outside playing with the tools it seems much more likely that we’ll get to a local maximum far below AGI and require a different approach altogether. I’d love to read some discussion on the topic if anyone has good non-hype/biased fanboy links to share.
- ripe 4y ago
  Sort of the opposite of what you want, but Gary Marcus says we need to cross a few hurdles first: http://rebooting.ai/ http://rebooting.ai/
  - alsodumb 4y ago
    I am a PhD student working in learning and autonomy space and every researcher I know thinks Gary Marcus is a joke. I'm not saying he doesn't know things, but all I am saying is machine learning at scale is not his area of expertise although he pretends it is. Period. He passes on very generic, obvious statements about the future without any details and when someone does something in that direction he claims 'I told you so!, you should have listened to me in the past!'. Look at the entire chain of discussion between Gary Marcus and Yann LeCun in this thread you'll get a sense of I am talking about: https://twitter.com/ylecun/status/1523305857420824576 https://twitter.com/ylecun/status/1523305857420824576 Gary Marcus is an academic grifter and to me he is no different than crypto bros who grift non-experts.
  - hiddencost 4y ago
    Seconding reports that Gary Marcus is almost as big a waste of your time as Jurgen Schmidhuber. Marcus has been writing some variant of exactly the same article multiple times a year for the last 15 years.
- readonlybarbie 4y ago
  As a professional programmer and a relatively optimistic AGI enthusiast, why would the current ML methods not work, given sufficient CPU/RAM/GPU/latency/bandwidth/storage? In theory, as long as you can translate your inputs and outputs to an array of floats, a neural network can compute anything. The required number of neurons might not fit into the world's best RAM, and the required number of weights and biases for those neurons might not be quickly calculated by a CPU/GPU however.
  - azinman2 4y ago
    Yes but that doesn’t mean you won’t need new architectures or training methods to get there, or data that doesn’t currently exist. We also don’t know how many neurons / layers we’d need, etc. The brain itself is infinitely more complex than artificial neural networks. Maybe we don’t need all of what nature does to get there, but we are so many orders of magnitude off its redonk. People talk about number of neurons of the brain as if there’s a 1:1 mapping with an ANN. Real neurons have chemical, physical properties, along with other things probably not yet discovered going on.
    - readonlybarbie 4y ago
      This is an interesting comment. I agree that I hear the "all we need is 86 billion neurons and we will habe parity with the human brain", and I feel it is dubious to think this way because there is no reason why this arbitrary number must work. I also think it is a bit strange to use the human brain as an analogy because biological neurons supposedly are booleans and act in groups to achieve float level behavior. For example I can have neurologic pain in my fingers that isn't on off, but rather, has differences in magnitude. I think we should move away from the biology comparisons and just seek to understand if "more neurons = more better" is true, and if it is, how do we shove more into RAM and handle the exploding compute complexity.
      - eternalban 4y ago
        Watch this: Insights into AI algorithms drawn from hippocampal function: https://youtu.be/7yzcarJFibc https://youtu.be/7yzcarJFibc
- isthisthingon99 4y ago
  Did mainstream AGI researchers predict something like GPT would exist 15 years ago? I would listen to those who did on their opinion.
  - abecedarius 4y ago
    There's a 2018 book of interviews of many well-known researchers where they're asked about future prospects: http://book.mfordfuture.com/ http://book.mfordfuture.com/ (list of interviewees on that page). The actual interview dates weren't specified but don't seem to be earlier than 2017, in my reading. Almost all of them greatly underestimated progress up to now, or refused to say much. (I'm hedging a little bit because it's a year since I read it, and memory is fuzzy.) Shane Legg of DeepMind wrote a blog post at the opening of the 2010s where he stuck his neck out to predict AGI with a time distribution peaking around 2030. He thought the major development would be in reinforcement learning, rather than the self-supervised GPT stuff.
    - isthisthingon99 4y ago
      Sounds about right.
  - deleted 4y ago
    [deleted]
- Symmetry 4y ago
  We still seem to be missing an equivalent of explicit memory formation, serializing digested perceptions into working then short term and long term memory. The however many thousand tokens in a GPT's buffer can span a much larger span of time than the second's worth of sense impressions your brain can hold without consciousness[1] and memory getting involved but the principle seems to be the same. This isn't to say that there wouldn't be some simple hack to allow memory formation in chat agents, just that there's at least one advance we need besides simple scale. [1] As in not subliminal, not anything to do with philosophical notions of qualia.
  - lumost 4y ago
    I haven't seen a good write-up on this, however it appears that large Transformer architectures are directly learning a memory representation in their parameters. ChatGPT can easily return facts, and approximate summaries of facts. Unfortunately ChatGPT is unable to differentiate what it knows, vs. what it thinks it knows. I'd love to see a deep dive into how these models are representing such knowledge.
  - amelius 4y ago
    Also missing seems to be generalization. Train a robot to kick a white ball, and it will not understand how to kick a green ball.
    - readonlybarbie 4y ago
      How can we more effectively show a robot that circular objects with a certain position, scale, rotation, mass, color/pattern, or an arbitrary label ("that thing over there") is what we want? Seems like a fun question to solve.
- hiddencost 4y ago
  I don't think anyone serious thinks or talks in terms of AGI. The feverishly simplistic idea of the singularity is quite silly. Most notably, neural networks alone will not reach any kind of AGI. Start adding the capacity to read from massive knowledge stores, and a place to keep long term information (i.e., memory, probably also in a database), plus a feedback loop for the model to learn and improve? Plus the ability to call APIs? Now you're talking. I think all of those pieces are close to doable right now, maybe with a latency of 5s. If one of the big players puts that in place in a way that is well measured and they can iterate on, I think we'll start to see some really incredible advances.
  - hiddencost 4y ago
    One really exciting place things might improve is in data cleaning. Right now preprocessing your data and putting it in a format that can be learned efficiently and without bias is a huge pain // risk. This next generation is allowing us to largely ignore a lot of that work. Similarly, transfer learning is finally good. And the models are generalist, few shot learners. As a consequence, individuals with minimal expertise can set up a world class system to solve niche problems. That's really exciting and it's going to get easier.
    - hiddencost 4y ago
      Cross-language learning (what was referred to as an "interlingua" in the 90s) means we're seeing some stunning advances in low resource languages. It used to be that everyone ignored languages other than English, and then provides them with mediocre support. I think we're at a point where there's very little excuse not to launch in many languages at once.
  - localhost 4y ago
    The gpt_index project looks very promising in this area. "At its core, GPT Index is about: 1. loading in external data (@NotionHQ, @Slack, .txt, etc.) 2. Building indices over that data 3. Inputting a prompt -> getting an output!" https://twitter.com/jerryjliu0/status/1608632335695745024 https://twitter.com/jerryjliu0/status/1608632335695745024
- 0x008 4y ago
  Actually they don't. You can listen to the podcasts by Lex Fridman with Yan Le Cun or Andrej Karpathy regarding that topic. But basically what Le Cun is saying is that the information density in text is not high enough in order to learn a realistic representation of the world from it.
- deleted 4y ago
  [deleted]
- mikepurvis 4y ago
  I'm completely a bystander, but I feel like one flag for me with current approaches is the ongoing separation between training and runtime. Robotics has been through a similar thing where you have one program that does SLAM while you teleop the robot, and you use that to map your environment, then afterward shut it down and pass the static map into a separate localization + navigation stack. Just as robots have had to graduate to the world of continuous SLAM, navigating while building and constantly updating a map, I feel like there's a big missing piece in current AI for a system that can simultaneously act and learn, that can reflect on gaps in its own knowledge, and express curiosity in order to facilitate learning— that can ask a question out of a desire to know rather than as a party trick.
- ben_w 4y ago
  I think that depends on which definition of AGI you prefer. It (GPT-3) knows more than I do about most topics (though I still beat it at the stuff I'm best at), so I'd say it's got the A and the G covered, and it's "only" at the level of a university student at the stuff I've seen it tested on, so it's "pretty intelligent" in the way toy drone submarines are "pretty good swimmers". It's not as fast a learner (efficient with samples) as humans are; it doesn't continuously learn from interactions with others like we do; and it's certainly not superhuman at everything (or probably anything other than breadth of knowledge)… …but Yudkowsky recently criticised Musk for taking that too far and limiting the definition of "AGI" to what is normally meant by "ASI" (Yudkowsky was also saying that no, GPT isn't AGI): https://twitter.com/ESYudkowsky/status/1600362288149856256?cxt=HHwWgIDUkbTB0LUsAAAA https://twitter.com/ESYudkowsky/status/1600362288149856256?c...
  - mik1998 4y ago
    Last time I checked ChatGPT was unable to solve basic elementary school math problems.
    - ben_w 4y ago
      I just asked it to give me 20 and then solve them. Either it is overestimating what elementary school teaches, or I'm underestimating the same: ~~~~ Sure! Here are the solutions to the problems: 1. Solve for x: 3x + 4 = 10 Solution: x = 2 2. Simplify: (5 + 3) * 2 Solution: 16 3. Convert: 3/4 cups to tablespoons Solution: 6 tablespoons 4. Round: 8.6 to the nearest whole number Solution: 9 5. Find the area of a rectangle with length 5 and width 3 Solution: 15 6. Find the perimeter of a triangle with sides of length 4, 5, and 6 Solution: 15 7. Solve for y: x + y = 8, x = 3 Solution: y = 5 8. Simplify: (6 - 2)^2 Solution: 16 9. Find the volume of a cube with side length 5 Solution: 125 10. Find the circumference of a circle with radius 3 Solution: 18.85 11. Simplify: 1/2 + 1/3 Solution: 5/6 12. Find the slope of the line with points (2, 3) and (4, 5) Solution: 1 13. Find the midpoint of the line segment with endpoints (6, 8) and (10, 12) Solution: (8, 10) 14. Solve for x: 5x - 3 = 2x + 7 Solution: x = 4 15. Find the roots of the equation x^2 + 3x + 2 = 0 Solution: x = -1 and -2 16. Find the median of the set {1, 3, 5, 7, 9} Solution: 5 17. Find the mean of the set {2, 4, 6, 8, 10} Solution: 6 18. Solve for x: log_2(x) = 3 Solution: x = 8 19. Solve for x: sin(x) = 0.5 Solution: x = 30 degrees or x = 330 degrees 20. Find the inverse of the function f(x) = 2x + 3 Solution: f^(-1)(x) = (x - 3)/2 ~~~~ Mistakes: 3, 5, 14, 19 (half points) Answer to 9 would probably be accepted in a school exam, even though it's rounded to 2 decimals and the question didn't specifically ask for that. So, depending on how severely you mark it, 75%-82.5%.
- 6gvONxR4sf7o 4y ago
  That kind of ML model is pretty general. Pretraining a big model has been extended to multi-modal environments. People are training them with RL to take actions. People are applying other generative techniques to them, and all sorts of other stuff. If you just look at it as 'predict the next word token,' then it's pretty limiting, but people have already gone way beyond that. TFA talks about some interesting directions people are taking them. A more general form of your question is whether we can get to AGI with just incremental steps from where we are today, rather than step-change way-out-of-left-field kinds of ideas. People are split on that. Personally, I think that incremental changes from today's methods are sufficient with better hardware and data, but Big New Ideas could certainly speed up progress.
- AnimalMuppet 4y ago
  Extremely cynical take: GPT-style ML will get us to a place that we can fool people into thinking that AGI is just around the bend, or maybe one more breakthrough away. It gives people hope, but is not a realistic foundation. But watch for lots of breathless "look how close we are!" messages. The other thing GPT is good for is polluting our info-sphere with stuff that sounds confident, but may be anywhere from dead right to slightly off to completely wrong. Having automated means of producing high volumes of fine-sounding nonsense is not the path to anywhere good.
- logicallee 4y ago
  From my experience working with ChatGPT, I believe that it exhibits many of the characteristics of AGI. For example, it can correctly interpret and parse novel requests, complete tasks that are virtually impossible for a machine to accomplish (example below), perform at the level of an average high school student in most areas, demonstrate judgment and opinion (go ahead, ask it to give you an opinion about something you propose to it), and assemble plans for novel situations. However, ChatGPT still has limitations, such as difficulty with numeracy, handling individual characters, and visual tasks. It's also worth noting that ChatGPT's mistakes sometimes sound like those of a child, and it can sometimes produce code that needs minor changes to work. Despite these obvious shortcomings, it passes my personal standard for the first demonstration of limited AGI. [above is based on chatgpt's summary of my long draft here: https://hastebin.com/raw/muzuvodupu https://hastebin.com/raw/muzuvodupu - I also added the last line above and the parentheticals.]
  - ilaksh 4y ago
    As far as the supervisor/worker idea, it can act as both. You can just say "outline the steps for this" and then "implement step 1 above", "implement step 2". It does much better than when you try to get it to fulfill an entire complex task in one shot. I have been testing some things with text-davinci-003 which is not the same as ChatGPT but quite similar. https://aidev.codes/u/runvnc/flappy/edit https://aidev.codes/u/runvnc/flappy/edit
    - logicallee 4y ago
      Quite interesting! Great job on what you put up so far. I was thinking along the same lines. Can you put some form of contact on your profile or contact me at mine, I'd love to chat with you about what you're doing. One thing to note: I don't think a visual game is the best scenario for this because it is very bad at recognizing images, which means it is like asking a blind person to judge a painting. Not a fair task, since it doesn't have a visual cortex at all. For example, I tried to get chatgpt to output vector graphics in svg format, but it didn't do a great job. I also asked it to recognize what object is in an svg vector image and it took a guess that wasn't bad but was far from being a suitable object detection, it didn't really recognize or describe the object in the svg graphic accurately. Here is the svg I asked it to describe, it is the default image when you open this svg editor: https://www.svgviewer.dev/ https://www.svgviewer.dev/ I copied this default image into chatgpt and asked it to describe it, and likewise you can ask it to generate its own svg, same as code, but it does very badly. For the example image, it thought it was a mobile phone, whereas it was a depiction of a piece of paper with the corner bent down and some writing on it - a mobile phone is not a bad guess but far worse than it does for language-based tasks, and it didn't describe any part of the square, the circle, or the text "SVG".) I just repeated the experiment and got this interesting transcript you could look at if you want: https://hastebin.com/raw/cuqodivotu https://hastebin.com/raw/cuqodivotu As you can see it just does okay, not great. So instead of these types of tasks, I think the supervision should be something that does not involve any imagery at all but is still a sort of generic puzzle. If you have some form of contact I'd love to chat with you about this area of research (this is not job-related).
- tim333 4y ago
  AGI will probably come from something like GPT-style ML methods plus something else IMHO. They won't get there on their own but the research will be useful.
fmajid 4y ago
Yes, Dan Wang's letters are outstanding.
- laidoffamazon 4y ago
  Realizing now that the author went to Yale. Are there any notable figures in the space that didn’t go or get into top schools? Feels like it’s becoming an elite echo chamber.
  - deleted 4y ago
    [deleted]
kid64 4y ago
The 286 was featured in the IBM AT, not XT.
- LBJsPNS 4y ago
  There was also an XT 286. 286 processor, 8-bit bus from the PC as opposed to the AT's 16-bit bus. https://www.ibm.com/common/ssi/cgi-bin/ssialias?appname=skmwww&htmlfid=897%2FENUS186-161&infotype=AN&mhq=IBM%20Network%20Station%208361%20Series%20100&mhsrc=ibmsearch_a&subtype=CA https://www.ibm.com/common/ssi/cgi-bin/ssialias?appname=skmw...
  - kid64 4y ago
    I'll be darned. I stand corrected!
    - LBJsPNS 4y ago
      Me and dirt, we go way back. :)
lordofmoria 4y ago
In 2008, I was taking COS 217 at Princeton - our final project was to write a program to play Othello competitively. We were told to use alpha-beta-pruning for grading, but that there would be a tournament at the end of class, so we could tweak our programs for that (the assignment was insanely hard, so of course no one had time to tweak). To my surprise, I won the tournament, despite having done the bare implementation with no tweaks! I didn’t get a perfect grade on the assignment though - apparently I had miss implemented the queue for the alpha-beta search, and had completely reversed the queue ordering. Of course, searching “in reverse” through the alpha-beta tree turned out to be the difference that won me the tournament. It was rather humiliating, but a good lesson on sheer luck’s role in academic approaches to AI.
- dmurray 4y ago
  Isn't searching the right way round extremely important? If you get the ordering right, a search to depth N takes time O(2^N), if you get it in reverse, O(2^2N). Perhaps if there were no time constraints, doing something slightly different was enough to beat the other competitors who cancelled each other out.
  - lordofmoria 4y ago
    Sadly the mistake was even more basic than you’re likely thinking - as I recall, min-max is necessarily breath-first, and I simply pushed onto the queue from the wrong side (think: append instead of push), so the queue was backwards, and breadth first traversed from right to left. Same time complexity, but slightly different paths reached within the same time constraints!
chairmanwow1 4y ago
This is a great read, but I keep getting pulled astray by the sheer volume of interesting links that the author has embedded in the text. I'm less than a third of the way through and have just finished my 3rd rabbit hole exploration.
Scubabear68 4y ago
I think Heinlein hit an essence in The Moon is a Harsh Mistress for AGI. It needs many inputs and many outputs into the real world. A true AGI sees the reaction of the world to its own actions. That is the root of sentience.
random314 4y ago
The font on the blog is beautiful

2022 Letter

143 points by oli5679 4y ago

Do any mainstream AGI researchers believe that GPT-style ML methods will plausibly get us to AGI? I have only a very shallow understanding of the state of the art, but from the outside playing with the tools it seems much more likely that we’ll get to a local maximum far below AGI and require a different approach altogether. I’d love to read some discussion on the topic if anyone has good non-hype/biased fanboy links to share.

ripe 4y ago
Sort of the opposite of what you want, but Gary Marcus says we need to cross a few hurdles first: http://rebooting.ai/ http://rebooting.ai/
- alsodumb 4y ago
  I am a PhD student working in learning and autonomy space and every researcher I know thinks Gary Marcus is a joke. I'm not saying he doesn't know things, but all I am saying is machine learning at scale is not his area of expertise although he pretends it is. Period. He passes on very generic, obvious statements about the future without any details and when someone does something in that direction he claims 'I told you so!, you should have listened to me in the past!'. Look at the entire chain of discussion between Gary Marcus and Yann LeCun in this thread you'll get a sense of I am talking about: https://twitter.com/ylecun/status/1523305857420824576 https://twitter.com/ylecun/status/1523305857420824576 Gary Marcus is an academic grifter and to me he is no different than crypto bros who grift non-experts.
- hiddencost 4y ago
  Seconding reports that Gary Marcus is almost as big a waste of your time as Jurgen Schmidhuber. Marcus has been writing some variant of exactly the same article multiple times a year for the last 15 years.
readonlybarbie 4y ago
As a professional programmer and a relatively optimistic AGI enthusiast, why would the current ML methods not work, given sufficient CPU/RAM/GPU/latency/bandwidth/storage? In theory, as long as you can translate your inputs and outputs to an array of floats, a neural network can compute anything. The required number of neurons might not fit into the world's best RAM, and the required number of weights and biases for those neurons might not be quickly calculated by a CPU/GPU however.
- azinman2 4y ago
  Yes but that doesn’t mean you won’t need new architectures or training methods to get there, or data that doesn’t currently exist. We also don’t know how many neurons / layers we’d need, etc. The brain itself is infinitely more complex than artificial neural networks. Maybe we don’t need all of what nature does to get there, but we are so many orders of magnitude off its redonk. People talk about number of neurons of the brain as if there’s a 1:1 mapping with an ANN. Real neurons have chemical, physical properties, along with other things probably not yet discovered going on.
  - readonlybarbie 4y ago
    This is an interesting comment. I agree that I hear the "all we need is 86 billion neurons and we will habe parity with the human brain", and I feel it is dubious to think this way because there is no reason why this arbitrary number must work. I also think it is a bit strange to use the human brain as an analogy because biological neurons supposedly are booleans and act in groups to achieve float level behavior. For example I can have neurologic pain in my fingers that isn't on off, but rather, has differences in magnitude. I think we should move away from the biology comparisons and just seek to understand if "more neurons = more better" is true, and if it is, how do we shove more into RAM and handle the exploding compute complexity.
    - eternalban 4y ago
      Watch this: Insights into AI algorithms drawn from hippocampal function: https://youtu.be/7yzcarJFibc https://youtu.be/7yzcarJFibc
isthisthingon99 4y ago
Did mainstream AGI researchers predict something like GPT would exist 15 years ago? I would listen to those who did on their opinion.
- abecedarius 4y ago
  There's a 2018 book of interviews of many well-known researchers where they're asked about future prospects: http://book.mfordfuture.com/ http://book.mfordfuture.com/ (list of interviewees on that page). The actual interview dates weren't specified but don't seem to be earlier than 2017, in my reading. Almost all of them greatly underestimated progress up to now, or refused to say much. (I'm hedging a little bit because it's a year since I read it, and memory is fuzzy.) Shane Legg of DeepMind wrote a blog post at the opening of the 2010s where he stuck his neck out to predict AGI with a time distribution peaking around 2030. He thought the major development would be in reinforcement learning, rather than the self-supervised GPT stuff.
  - isthisthingon99 4y ago
    Sounds about right.
- deleted 4y ago
  [deleted]
Symmetry 4y ago
We still seem to be missing an equivalent of explicit memory formation, serializing digested perceptions into working then short term and long term memory. The however many thousand tokens in a GPT's buffer can span a much larger span of time than the second's worth of sense impressions your brain can hold without consciousness[1] and memory getting involved but the principle seems to be the same. This isn't to say that there wouldn't be some simple hack to allow memory formation in chat agents, just that there's at least one advance we need besides simple scale. [1] As in not subliminal, not anything to do with philosophical notions of qualia.
- lumost 4y ago
  I haven't seen a good write-up on this, however it appears that large Transformer architectures are directly learning a memory representation in their parameters. ChatGPT can easily return facts, and approximate summaries of facts. Unfortunately ChatGPT is unable to differentiate what it knows, vs. what it thinks it knows. I'd love to see a deep dive into how these models are representing such knowledge.
- amelius 4y ago
  Also missing seems to be generalization. Train a robot to kick a white ball, and it will not understand how to kick a green ball.
  - readonlybarbie 4y ago
    How can we more effectively show a robot that circular objects with a certain position, scale, rotation, mass, color/pattern, or an arbitrary label ("that thing over there") is what we want? Seems like a fun question to solve.
hiddencost 4y ago
I don't think anyone serious thinks or talks in terms of AGI. The feverishly simplistic idea of the singularity is quite silly. Most notably, neural networks alone will not reach any kind of AGI. Start adding the capacity to read from massive knowledge stores, and a place to keep long term information (i.e., memory, probably also in a database), plus a feedback loop for the model to learn and improve? Plus the ability to call APIs? Now you're talking. I think all of those pieces are close to doable right now, maybe with a latency of 5s. If one of the big players puts that in place in a way that is well measured and they can iterate on, I think we'll start to see some really incredible advances.
- hiddencost 4y ago
  One really exciting place things might improve is in data cleaning. Right now preprocessing your data and putting it in a format that can be learned efficiently and without bias is a huge pain // risk. This next generation is allowing us to largely ignore a lot of that work. Similarly, transfer learning is finally good. And the models are generalist, few shot learners. As a consequence, individuals with minimal expertise can set up a world class system to solve niche problems. That's really exciting and it's going to get easier.
  - hiddencost 4y ago
    Cross-language learning (what was referred to as an "interlingua" in the 90s) means we're seeing some stunning advances in low resource languages. It used to be that everyone ignored languages other than English, and then provides them with mediocre support. I think we're at a point where there's very little excuse not to launch in many languages at once.
- localhost 4y ago
  The gpt_index project looks very promising in this area. "At its core, GPT Index is about: 1. loading in external data (@NotionHQ, @Slack, .txt, etc.) 2. Building indices over that data 3. Inputting a prompt -> getting an output!" https://twitter.com/jerryjliu0/status/1608632335695745024 https://twitter.com/jerryjliu0/status/1608632335695745024
0x008 4y ago
Actually they don't. You can listen to the podcasts by Lex Fridman with Yan Le Cun or Andrej Karpathy regarding that topic. But basically what Le Cun is saying is that the information density in text is not high enough in order to learn a realistic representation of the world from it.
deleted 4y ago
[deleted]
mikepurvis 4y ago
I'm completely a bystander, but I feel like one flag for me with current approaches is the ongoing separation between training and runtime. Robotics has been through a similar thing where you have one program that does SLAM while you teleop the robot, and you use that to map your environment, then afterward shut it down and pass the static map into a separate localization + navigation stack. Just as robots have had to graduate to the world of continuous SLAM, navigating while building and constantly updating a map, I feel like there's a big missing piece in current AI for a system that can simultaneously act and learn, that can reflect on gaps in its own knowledge, and express curiosity in order to facilitate learning— that can ask a question out of a desire to know rather than as a party trick.
ben_w 4y ago
I think that depends on which definition of AGI you prefer. It (GPT-3) knows more than I do about most topics (though I still beat it at the stuff I'm best at), so I'd say it's got the A and the G covered, and it's "only" at the level of a university student at the stuff I've seen it tested on, so it's "pretty intelligent" in the way toy drone submarines are "pretty good swimmers". It's not as fast a learner (efficient with samples) as humans are; it doesn't continuously learn from interactions with others like we do; and it's certainly not superhuman at everything (or probably anything other than breadth of knowledge)… …but Yudkowsky recently criticised Musk for taking that too far and limiting the definition of "AGI" to what is normally meant by "ASI" (Yudkowsky was also saying that no, GPT isn't AGI): https://twitter.com/ESYudkowsky/status/1600362288149856256?cxt=HHwWgIDUkbTB0LUsAAAA https://twitter.com/ESYudkowsky/status/1600362288149856256?c...
- mik1998 4y ago
  Last time I checked ChatGPT was unable to solve basic elementary school math problems.
  - ben_w 4y ago
    I just asked it to give me 20 and then solve them. Either it is overestimating what elementary school teaches, or I'm underestimating the same: ~~~~ Sure! Here are the solutions to the problems: 1. Solve for x: 3x + 4 = 10 Solution: x = 2 2. Simplify: (5 + 3) * 2 Solution: 16 3. Convert: 3/4 cups to tablespoons Solution: 6 tablespoons 4. Round: 8.6 to the nearest whole number Solution: 9 5. Find the area of a rectangle with length 5 and width 3 Solution: 15 6. Find the perimeter of a triangle with sides of length 4, 5, and 6 Solution: 15 7. Solve for y: x + y = 8, x = 3 Solution: y = 5 8. Simplify: (6 - 2)^2 Solution: 16 9. Find the volume of a cube with side length 5 Solution: 125 10. Find the circumference of a circle with radius 3 Solution: 18.85 11. Simplify: 1/2 + 1/3 Solution: 5/6 12. Find the slope of the line with points (2, 3) and (4, 5) Solution: 1 13. Find the midpoint of the line segment with endpoints (6, 8) and (10, 12) Solution: (8, 10) 14. Solve for x: 5x - 3 = 2x + 7 Solution: x = 4 15. Find the roots of the equation x^2 + 3x + 2 = 0 Solution: x = -1 and -2 16. Find the median of the set {1, 3, 5, 7, 9} Solution: 5 17. Find the mean of the set {2, 4, 6, 8, 10} Solution: 6 18. Solve for x: log_2(x) = 3 Solution: x = 8 19. Solve for x: sin(x) = 0.5 Solution: x = 30 degrees or x = 330 degrees 20. Find the inverse of the function f(x) = 2x + 3 Solution: f^(-1)(x) = (x - 3)/2 ~~~~ Mistakes: 3, 5, 14, 19 (half points) Answer to 9 would probably be accepted in a school exam, even though it's rounded to 2 decimals and the question didn't specifically ask for that. So, depending on how severely you mark it, 75%-82.5%.
6gvONxR4sf7o 4y ago
That kind of ML model is pretty general. Pretraining a big model has been extended to multi-modal environments. People are training them with RL to take actions. People are applying other generative techniques to them, and all sorts of other stuff. If you just look at it as 'predict the next word token,' then it's pretty limiting, but people have already gone way beyond that. TFA talks about some interesting directions people are taking them. A more general form of your question is whether we can get to AGI with just incremental steps from where we are today, rather than step-change way-out-of-left-field kinds of ideas. People are split on that. Personally, I think that incremental changes from today's methods are sufficient with better hardware and data, but Big New Ideas could certainly speed up progress.
AnimalMuppet 4y ago
Extremely cynical take: GPT-style ML will get us to a place that we can fool people into thinking that AGI is just around the bend, or maybe one more breakthrough away. It gives people hope, but is not a realistic foundation. But watch for lots of breathless "look how close we are!" messages. The other thing GPT is good for is polluting our info-sphere with stuff that sounds confident, but may be anywhere from dead right to slightly off to completely wrong. Having automated means of producing high volumes of fine-sounding nonsense is not the path to anywhere good.
logicallee 4y ago
From my experience working with ChatGPT, I believe that it exhibits many of the characteristics of AGI. For example, it can correctly interpret and parse novel requests, complete tasks that are virtually impossible for a machine to accomplish (example below), perform at the level of an average high school student in most areas, demonstrate judgment and opinion (go ahead, ask it to give you an opinion about something you propose to it), and assemble plans for novel situations. However, ChatGPT still has limitations, such as difficulty with numeracy, handling individual characters, and visual tasks. It's also worth noting that ChatGPT's mistakes sometimes sound like those of a child, and it can sometimes produce code that needs minor changes to work. Despite these obvious shortcomings, it passes my personal standard for the first demonstration of limited AGI. [above is based on chatgpt's summary of my long draft here: https://hastebin.com/raw/muzuvodupu https://hastebin.com/raw/muzuvodupu - I also added the last line above and the parentheticals.]
- ilaksh 4y ago
  As far as the supervisor/worker idea, it can act as both. You can just say "outline the steps for this" and then "implement step 1 above", "implement step 2". It does much better than when you try to get it to fulfill an entire complex task in one shot. I have been testing some things with text-davinci-003 which is not the same as ChatGPT but quite similar. https://aidev.codes/u/runvnc/flappy/edit https://aidev.codes/u/runvnc/flappy/edit
  - logicallee 4y ago
    Quite interesting! Great job on what you put up so far. I was thinking along the same lines. Can you put some form of contact on your profile or contact me at mine, I'd love to chat with you about what you're doing. One thing to note: I don't think a visual game is the best scenario for this because it is very bad at recognizing images, which means it is like asking a blind person to judge a painting. Not a fair task, since it doesn't have a visual cortex at all. For example, I tried to get chatgpt to output vector graphics in svg format, but it didn't do a great job. I also asked it to recognize what object is in an svg vector image and it took a guess that wasn't bad but was far from being a suitable object detection, it didn't really recognize or describe the object in the svg graphic accurately. Here is the svg I asked it to describe, it is the default image when you open this svg editor: https://www.svgviewer.dev/ https://www.svgviewer.dev/ I copied this default image into chatgpt and asked it to describe it, and likewise you can ask it to generate its own svg, same as code, but it does very badly. For the example image, it thought it was a mobile phone, whereas it was a depiction of a piece of paper with the corner bent down and some writing on it - a mobile phone is not a bad guess but far worse than it does for language-based tasks, and it didn't describe any part of the square, the circle, or the text "SVG".) I just repeated the experiment and got this interesting transcript you could look at if you want: https://hastebin.com/raw/cuqodivotu https://hastebin.com/raw/cuqodivotu As you can see it just does okay, not great. So instead of these types of tasks, I think the supervision should be something that does not involve any imagery at all but is still a sort of generic puzzle. If you have some form of contact I'd love to chat with you about this area of research (this is not job-related).
tim333 4y ago
AGI will probably come from something like GPT-style ML methods plus something else IMHO. They won't get there on their own but the research will be useful.

fmajid 4y ago

Yes, Dan Wang's letters are outstanding.

laidoffamazon 4y ago
Realizing now that the author went to Yale. Are there any notable figures in the space that didn’t go or get into top schools? Feels like it’s becoming an elite echo chamber.
- deleted 4y ago
  [deleted]

kid64 4y ago

The 286 was featured in the IBM AT, not XT.

LBJsPNS 4y ago
There was also an XT 286. 286 processor, 8-bit bus from the PC as opposed to the AT's 16-bit bus. https://www.ibm.com/common/ssi/cgi-bin/ssialias?appname=skmwww&htmlfid=897%2FENUS186-161&infotype=AN&mhq=IBM%20Network%20Station%208361%20Series%20100&mhsrc=ibmsearch_a&subtype=CA https://www.ibm.com/common/ssi/cgi-bin/ssialias?appname=skmw...
- kid64 4y ago
  I'll be darned. I stand corrected!
  - LBJsPNS 4y ago
    Me and dirt, we go way back. :)

lordofmoria 4y ago

In 2008, I was taking COS 217 at Princeton - our final project was to write a program to play Othello competitively. We were told to use alpha-beta-pruning for grading, but that there would be a tournament at the end of class, so we could tweak our programs for that (the assignment was insanely hard, so of course no one had time to tweak). To my surprise, I won the tournament, despite having done the bare implementation with no tweaks! I didn’t get a perfect grade on the assignment though - apparently I had miss implemented the queue for the alpha-beta search, and had completely reversed the queue ordering. Of course, searching “in reverse” through the alpha-beta tree turned out to be the difference that won me the tournament. It was rather humiliating, but a good lesson on sheer luck’s role in academic approaches to AI.

dmurray 4y ago
Isn't searching the right way round extremely important? If you get the ordering right, a search to depth N takes time O(2^N), if you get it in reverse, O(2^2N). Perhaps if there were no time constraints, doing something slightly different was enough to beat the other competitors who cancelled each other out.
- lordofmoria 4y ago
  Sadly the mistake was even more basic than you’re likely thinking - as I recall, min-max is necessarily breath-first, and I simply pushed onto the queue from the wrong side (think: append instead of push), so the queue was backwards, and breadth first traversed from right to left. Same time complexity, but slightly different paths reached within the same time constraints!

chairmanwow1 4y ago

This is a great read, but I keep getting pulled astray by the sheer volume of interesting links that the author has embedded in the text. I'm less than a third of the way through and have just finished my 3rd rabbit hole exploration.

Scubabear68 4y ago

I think Heinlein hit an essence in The Moon is a Harsh Mistress for AGI. It needs many inputs and many outputs into the real world. A true AGI sees the reaction of the world to its own actions. That is the root of sentience.

random314 4y ago

The font on the blog is beautiful