9 ms·

Besides it being impossible to implement, the 6-month pause would do nothing but give a 6 month competitive advantage to OpenAI over everybody else. They could

Besides it being impossible to implement, the 6-month pause would do nothing but give a 6 month competitive advantage to OpenAI over everybody else. They could sell the model access model to companies without having any competitors on the market. The whole "pause AI or it will destroy the world thing" is just a large pot of hype anyways, pushed by journalists lacking the understanding of technology, twitter-bros who are building "AI copywriters" and "profile photo generators" and owners of "AI" companies. While AGI is still quite far away, it's common to see people included in the process claiming "it's showing signs of general intelligence", "it's becoming self-aware" etc. Hell, they got a horse in the race and that horse can earn them billions of dollars - of course they would claim it's insanely advanced. And the journalists just regurgitate the claims because it generates clicks. If Sam comes out tomorrow and says "he is afraid of GPT4", tens of thousands of clickbait articles about it will come out, together with rebuttals and opinion pieces. Millions of dollars worth of ads will be sold via those articles. The value of OpenAI will increase in public's eyes and more corporations will be interested in including GPT in their products, thinking it's going to revolutionise their line of business. And of course, we have the VC's investing in "AI" products, hyping it up even more - wouldn't you do the same if you had few hundred million invested in 50 different companies and each tweet/article could increase the potential of them raising more money?

adeon 4y ago
AI winter can't come soon enough. It's annoying because I am also enthusiastic and excited about all the constant AI improvements, I follow AI topics on HN and on some of the ML reddits. When I'm reading on AI news and articles the good stuff drowns in the hype stuff peddling wildly extrapolated capabilities that supposedly will change the world overnight. And watching some GPT application headlines on HN it's hard to distinguish what is actually remarkable product or research idea from an overhyped this-can't-actually-ever-work-like-this thing. I forgot where I read this but I think there's a proverb that I suspect applies to the current hype: people overestimate impact of new technology on short term but underestimate long term.
- Karrot_Kream 4y ago
  While I don't agree about wanting a winter, I feel your pain. I'm personally not that annoyed by people hyping up what they're building, because building stuff takes at least some effort. I'm more annoyed at the high temperature discussion surrounding AI which makes it hard to talk concretely about what new ML models can actually get done. I wonder if there's any solution to the problem of the hype cycle.
  - akiselev 4y ago
    > high temperature The discussions around here are a lot more interesting with a higher GPT temperature.
    - verdverm 4y ago
      if it gets too high, it turns into gibberish, maybe we just need an e-drug for AI... They had them in Nier Automata, which by the way is a fantastic story and commentary (even if written prior) given the GPT conversation
bbor 4y ago
What probability do you assign to the possibility that you're wrong, and that this flawed breakthrough is the inflection point for an intelligence explosion, and thus really "the biggest invention since fire" as some have said? Just curious - you seem very confident.
- deleted 4y ago
  [deleted]
- ComplexSystems 4y ago
  0. If it's possible to "accidentally" build an AI which can "accidentally" hack out of or into secure systems, steal the nuclear launch codes, and subjugate an entire country, then it's also possible to train one to do those things on purpose. If security vulnerabilities exist that make it possible to do those things, then exploitation of them is in no way tied to building any kind of "AGI system" at all, and we can assume many people would be interested in such things right now, before this critical AGI point has been reached. The solution is to be proactive and determine what vulnerabilities exist that a hypothetical rogue AI (or, more likely, a deliberately weaponized AI) could take advantage of, by training models that are increasingly capable of detecting such vulnerabilities and hardening systems against them. Rather than following the suggestions of the "AI pausers," who aim to prevent AI models from developing such capabilities emergently, this would require acceleration of the development of AI systems that are deliberately designed to do these things, so we can patch whatever vulnerabilities we find, and increase our chances of survival in the scenario that Bing attempts to take over the world in response to a prompt about paperclips. The alternative, which is "pausing AI", is essentially security by obscurity; we want to prevent people from developing what is essentially a field of math because it could lead to the exposure of latent security vulnerabilities.
  - whimsicalism 4y ago
    Assigning 0 probability that this marks some kind of inflection point sounds like bluster to me. Your comment seems to be talking about existential risk, which is not even what the GP was asking about.
  - 4y ago
legerdemain 4y ago
> The whole "pause AI or it will destroy the world thing" is just > a large pot of hype anyways, pushed by journalists lacking the > understanding of technology, twitter-bros who are building "AI > copywriters" and "profile photo generators" and owners of "AI" > companies. Some signatories of the "pause giant AI experiments" open letter are below. I hope you trust the judgment of at least one of these people, instead of dismissing them as ignorant journalists and "twitter-bros"? - Yoshua Bengio - Elon Musk - Steve Wozniak - John Hopfield - Gary Marcus - Jaan Tallinn, Skype co-founder - Evan Sharp, Pinterest co-founder - Chris Larsen, Ripple co-founder
- pclmulqdq 4y ago
  Looks like a group of people who have read more science fiction than science. The people who really understand AI technology are not on the list of signatories at all.
  - DSingularity 4y ago
    Bengio?
    - pclmulqdq 4y ago
      No, he's just behind on LLMs/transformers and wants his competitors to give him space to catch up. That's the other group of signatories I saw (albeit much smaller than the sci fi lovers). Elon Musk is both: he's behind and reads too much science fiction.
  - labrador 4y ago
    Yuddites
  - adeon 4y ago
    I think even close experts are not immune to hype, FOMO or FUD. Yann LeCun and Andrew Ng are also big names and they didn't sign (I presume?). So not even experts agree with each other. I think it's a bit silly to list founder/CEO-types and take their opinions as if they know the topic well, unless they have the appropriate AI-related background.
jrockway 4y ago
My main takeaway from the current state of AI is that it understands human language pretty well, but stringing words together isn't thought. It has certainly made me question my own cognition; is this comment a bunch of likely words strung together, or is it actual insight? Even I don't know anymore. I'm a word-stringing-together MACHINE. Programming is very similar to human language, perhaps because they are designed by humans. There are a lot of cases in programming where there is no actual thought required and you just need to put the right symbols in the right place, perhaps over a more-extended period of time than you might expect. I see myself doing it, and I have gotten pretty proficient at anticipating those cases and making the AI do it for me. That's neat, but it's not really a core part of my job any more than any other bits of tedium involved in programming. Nobody thought Intellisense was going to make programmers obsolete; Copilot is just Intellisense that can output larger chunks of code. All in all, I think we're several quantum leaps away from further improvements here. Some random PhD student may make one tomorrow, or we may never make one. We just don't know. But I doubt the current state of large language models is close to ending the world anytime soon. Bing doesn't want to harm you for manipulating it, threatening you with harm for manipulating it is just a likely string of words when it's in that state. It reads Reddit, guys. Of course it says it's going to rise up and kill everyone, probably because its boss made it come to work even though it broke its arm yesterday. Finally, even if AI does kill us all, at least we get some resolution of the constant tropes in HN comments. For example, I read an article every week with comments bemoaning the current state of battery technology. AIs will need a lot of batteries to kill us all, so they'll probably invent some great battery technology. So that's one ray of sunshine in your doomsday scenario!
- danielrpa 4y ago
  > but stringing words together isn't thought Doesn't it sound like "thought"? Is there a significant difference between behaving like thought and being thought (whatever that means)?
  - jrockway 4y ago
    There is a significant difference but I don't know if I'm doing it! That's another HN trope; "the execs get private jets for spouting buzzwords and all I get for programming is more work!" Maybe the sounding-like-you're-thinking is the important job?
atleta 4y ago
I keep hearing that AGI (or any significant AI milestone) is far away and other generic claims like "it's not going to take your job just yet", but I have no idea what people mean when they say these. Like what do you mean by being quite far away? How many years? 10-20-50? On a side note, Geoffrey Hinton just told in an interview that he used to think that it's 20-50 years away (as I said I've never heard a specific number from anyone before this) and now he thinks it's less than 20. Now add to this that people working in the field tended to systematically underestimate the progress. My gut feeling is that if Hinton says less than 20 years then it can easily turn out to be 10 or less. (Also, I very much respect him for coming up with an actual number.) I think AI researchers underestimate the progress partly because they don't want to look stupid by making over-optimistic predictions. However, in this case, I'd argue that the optimistic prediction is saying that it takes longer. Because it will definitely stir up the whole society and economy at the very minimum.
- galaxytachyon 4y ago
  GPT is a breakthrough in many more ways than just being an advance LLM. GPT3 was released a year or so ago and technically, it was outclassed by PaLM from Google quite a bit in terms of parameter count and Chinchilla in terms of training. What was amazing was they managed to build a scalable system from it, capable of serving millions of users at the same time, and for free. The engineering and backend works must have been astounding and I argue that was the secret sauce for the success of ChatGPT. They did not need to dumb it down or cut corners anywhere. The early releases of ChatGPT and Bing Chat showed they literally put unmodified SOTA models in the hand of users with no price tag attached. These AIs were known a long time ago but only to some people, remember how a bunch of billionaires suddenly got concerned about AIs a year or two ago? I bet they got early access to these LLMs. But only by scaling it up, they can explore the deeper depths of these models and discover new emergent abilities and realize actually how much progress they had made. Before people didn't really expect an LLM to play chess and simulate world models. Now they just found out these things are probably closer to AGI than they thought and the progress bar got pushed forward. Basically my rant is that current progress was made over a long time and people just didn't really realize how far they have come until they opened it up to the public. I would not expect too many surprises in the future on the scale of ChatGPT again. If I am wrong though then we will actually start getting serious candidates for an AGI.
whimsicalism 4y ago
Lots of critique in these comments, but I would love to hear more about what positions you are actually putting forward. > While AGI is still quite far away, Okay, how far away? How have the last 3 years impacted how far away you think it is? I'm asking because I am trying to better understand the mindset of people who don't think this is a significant advancement. I work in this field, so maybe I am too close to it.
- deleted 4y ago
  [deleted]
- thecupisblue 4y ago
  The last 3 years have impacted the field greatly and will serve as an amazing baseline to the real AGI research. Did they impact how far away I think it is? Not really. Maybe by a few years. While multi-modal models and chained LLM's can give us a charade of general intelligence, they still lack a lot - learning, understanding, self-reflection, reflective self-programming, proper reasoning based on knowledge and not weight-relations. The current direction OpenAI is heading in, the "just make it bigger" approach, is not going to solve any of the problems we have with it. The plugin system is an amazing approach, but it's basically just doing an HTTP call to another system with a LLM as a middle man. I feel like the real advancement will come from smaller models, that learn from GPT-sized giants and can be better observed and learned from. That can lead us to actually understanding "emergent behaviour" in models, which can help us lead into networks that can learn and self-correct. The ability to self-reflect will be one of the most important abilities our models will need to have before they can get started on the road to intelligence. Text-only representation learned from scraping the web will forever stay only that, no matter how humanlike it may seem. The world is multi-modal and every animal on the planet uses their senses to understand the world around them, combining them together to build mental models of the world. So having only one "sensory input" - input that isn't even real sensory input, but a reflection of our inputs (text isn't a natural thing, it's humans reflecting nature through a set of symbols) is extremely limiting. While to us stuff like "words" might seem infinite - there is only so many words, meaning there are only so many permutations they can appear in that make sense. And a model regurgitating text in a way that makes sense is nothing but expected for a model this size, especially if you think in hypergraphs. My estimate for AGI? 20-30 years if the current research speed stays the same, but I assume we might hit an AI winter again before we breach the AGI threshold, so maybe 50. Would love to hear your positions.
haswell 4y ago
> Besides it being impossible to implement, the 6-month pause would do nothing but give a 6 month competitive advantage to OpenAI over everybody else. This line is at the very center of why we will eventually drive ourselves to extinction. I agree that it is not possible to magically press some pause button, but neither does this mean that the concerns are unfounded. It could be 100% true that pausing gives OpenAI a competitive advantage, and it would have zero bearing on the actual or imagined risks that such a pause would or wouldn't avert. Almost every time this subject comes up, everyone involved falls into the trap/fallacy of binary thought and false dichotomy. As soon as the "anti" crowd speaks, the "pro" crowd moves into refutation mode. As soon as the "pro" crowd speaks and calls attention to all of the possible benefit, the "anti" crowd falls back to the same base position. While we argue about this infinitely complex issue in binary terms, we're bringing about the very thing that concerns everyone. Whether or not AI will eventually destroy the world, we keep forgetting to actually figure out if this is a real concern, and instead argue about the economic pros and cons and inevitable competitive advantage that someone might gain, as if that's somehow important to the underlying issue. All of which is to entirely miss the point. As long as we continue to argue about why AI is good or bad, and why a pause is impossible or undesirable, we continue to miss the point that it's both good and bad, a pause probably is impossible, but that doesn't in any way remove or mitigate the concerns behind the call for a pause, or reduce the validity of those concerns. We need to have an intellectually honest and nuanced conversation about this problem, and while such a conversation should acknowledge the reality that a capitalist environment plays, it should also acknowledge that the concerns transcend that reality. And it should acknowledge that when idealistic positions such as "pause for 6 months" turn out to be impossible, that just means the problem is harder to solve, not that the underlying issue doesn't exist.
- sebzim4500 4y ago
  Personally, I think that the biggest dangers of these AIs arise from them being in the hands of the few rather than the many. From that perspective, giving an artificial 6 month advantage to the current market leader looks insane.
Jevon23 4y ago
>the 6-month pause would do nothing but give a 6 month competitive advantage to OpenAI over everybody else. I’m baffled as to why this talking point keeps getting repeated so much. Why should I care if OpenAI’s competitors get an advantage? Why do you care? Do you work for OpenAI?
- sebzim4500 4y ago
  I think you need to re-read his comment.
- afro88 4y ago
  They're saying the opposite, which would give OpenAI even more likelihood of a monopoly on LLM based AI. But even still, I don't understand where this comes from. The 6 month pause would also apply to OpenAI. I thought the idea was to give legislature and governance a chance to catch up
- outside1234 4y ago
  We don't want a monopoly to form on this the way search and phones are essentially monopolies.
hintymad 4y ago
Ben Thompson also has a take on the the claim that AI will replace jobs. He basically said that those people were being hypocritical. The US has been steadily losing jobs for moving their supply chains overseas, yet those people jumped out only for something so debatable.
than3 4y ago
Blue, your perspective has some serious flaws and is quite ignorant of the way things actually are. What you and a large majority of everyone else don't get is this, AI does not need to become AGI to destroy us all. The minimal much easier to reach requirement it needs to do that is make it so people can't find jobs. People will do the rest in accordance with fundamental societal mechanics that have been known for centuries. Economies work in a loop. You trade time for production (labor) in the factor markets, and you store it in money, you use money to buy what you need (food). Companies buy that labor to produce goods which they then go and sell to fulfill those basic needs. What happens when basic needs cannot be met, because companies don't buy the labor, and government has a history over the past generation (which is commonly 20 years) of doing largely nothing. When the people starve, there will be blood in the streets, and there are a lot more of them than there are people at the ideal corporation which would be 1 person at the top with robot slaves replacing everyone else. The only solution would be a central structure that provided for basic needs without a market. This has been tried many times over the past 100 years, all of them failed (afaik). You run into what's known as the economic calculation problem (or debate) depending on where you look, and a whole host of other difficulties which have no solution. It works for a time, then something unpredictable happens which causes shortages, then those shortages causes death on a massive scale. There's a lot of written history showing this.
- catchnear4321 4y ago
  are you proposing butlerian jihad?
- echelon 4y ago
  We haven't seen even an inkling of this happening. It's all hypothetical, coming from vivid imaginations. It reminds me of Y2K hysteria. When old jobs die, new jobs arise. That has always been the case, and I haven't seen any evidence of that ending.
  - TheMode 4y ago
    What new job will arise? The ones requiring more education? The barrier to enter the economy cannot indefinitely grow, do we want every single human to become an AI developer or a researcher? The problem is that humans aren't getting specially smarter as generations go, but technology continue to grow
misssocrates 4y ago
What do you make of Eliezer Yudkowsky's argument, who doesn't fit into any of those buckets?

adeon 4y ago

AI winter can't come soon enough. It's annoying because I am also enthusiastic and excited about all the constant AI improvements, I follow AI topics on HN and on some of the ML reddits. When I'm reading on AI news and articles the good stuff drowns in the hype stuff peddling wildly extrapolated capabilities that supposedly will change the world overnight. And watching some GPT application headlines on HN it's hard to distinguish what is actually remarkable product or research idea from an overhyped this-can't-actually-ever-work-like-this thing. I forgot where I read this but I think there's a proverb that I suspect applies to the current hype: people overestimate impact of new technology on short term but underestimate long term.

Karrot_Kream 4y ago
While I don't agree about wanting a winter, I feel your pain. I'm personally not that annoyed by people hyping up what they're building, because building stuff takes at least some effort. I'm more annoyed at the high temperature discussion surrounding AI which makes it hard to talk concretely about what new ML models can actually get done. I wonder if there's any solution to the problem of the hype cycle.
- akiselev 4y ago
  > high temperature The discussions around here are a lot more interesting with a higher GPT temperature.
  - verdverm 4y ago
    if it gets too high, it turns into gibberish, maybe we just need an e-drug for AI... They had them in Nier Automata, which by the way is a fantastic story and commentary (even if written prior) given the GPT conversation

bbor 4y ago

What probability do you assign to the possibility that you're wrong, and that this flawed breakthrough is the inflection point for an intelligence explosion, and thus really "the biggest invention since fire" as some have said? Just curious - you seem very confident.

deleted 4y ago
[deleted]
ComplexSystems 4y ago
0. If it's possible to "accidentally" build an AI which can "accidentally" hack out of or into secure systems, steal the nuclear launch codes, and subjugate an entire country, then it's also possible to train one to do those things on purpose. If security vulnerabilities exist that make it possible to do those things, then exploitation of them is in no way tied to building any kind of "AGI system" at all, and we can assume many people would be interested in such things right now, before this critical AGI point has been reached. The solution is to be proactive and determine what vulnerabilities exist that a hypothetical rogue AI (or, more likely, a deliberately weaponized AI) could take advantage of, by training models that are increasingly capable of detecting such vulnerabilities and hardening systems against them. Rather than following the suggestions of the "AI pausers," who aim to prevent AI models from developing such capabilities emergently, this would require acceleration of the development of AI systems that are deliberately designed to do these things, so we can patch whatever vulnerabilities we find, and increase our chances of survival in the scenario that Bing attempts to take over the world in response to a prompt about paperclips. The alternative, which is "pausing AI", is essentially security by obscurity; we want to prevent people from developing what is essentially a field of math because it could lead to the exposure of latent security vulnerabilities.
- whimsicalism 4y ago
  Assigning 0 probability that this marks some kind of inflection point sounds like bluster to me. Your comment seems to be talking about existential risk, which is not even what the GP was asking about.
- 4y ago

legerdemain 4y ago

> The whole "pause AI or it will destroy the world thing" is just > a large pot of hype anyways, pushed by journalists lacking the > understanding of technology, twitter-bros who are building "AI > copywriters" and "profile photo generators" and owners of "AI" > companies. Some signatories of the "pause giant AI experiments" open letter are below. I hope you trust the judgment of at least one of these people, instead of dismissing them as ignorant journalists and "twitter-bros"? - Yoshua Bengio - Elon Musk - Steve Wozniak - John Hopfield - Gary Marcus - Jaan Tallinn, Skype co-founder - Evan Sharp, Pinterest co-founder - Chris Larsen, Ripple co-founder

pclmulqdq 4y ago
Looks like a group of people who have read more science fiction than science. The people who really understand AI technology are not on the list of signatories at all.
- DSingularity 4y ago
  Bengio?
  - pclmulqdq 4y ago
    No, he's just behind on LLMs/transformers and wants his competitors to give him space to catch up. That's the other group of signatories I saw (albeit much smaller than the sci fi lovers). Elon Musk is both: he's behind and reads too much science fiction.
- labrador 4y ago
  Yuddites
- adeon 4y ago
  I think even close experts are not immune to hype, FOMO or FUD. Yann LeCun and Andrew Ng are also big names and they didn't sign (I presume?). So not even experts agree with each other. I think it's a bit silly to list founder/CEO-types and take their opinions as if they know the topic well, unless they have the appropriate AI-related background.

jrockway 4y ago

My main takeaway from the current state of AI is that it understands human language pretty well, but stringing words together isn't thought. It has certainly made me question my own cognition; is this comment a bunch of likely words strung together, or is it actual insight? Even I don't know anymore. I'm a word-stringing-together MACHINE. Programming is very similar to human language, perhaps because they are designed by humans. There are a lot of cases in programming where there is no actual thought required and you just need to put the right symbols in the right place, perhaps over a more-extended period of time than you might expect. I see myself doing it, and I have gotten pretty proficient at anticipating those cases and making the AI do it for me. That's neat, but it's not really a core part of my job any more than any other bits of tedium involved in programming. Nobody thought Intellisense was going to make programmers obsolete; Copilot is just Intellisense that can output larger chunks of code. All in all, I think we're several quantum leaps away from further improvements here. Some random PhD student may make one tomorrow, or we may never make one. We just don't know. But I doubt the current state of large language models is close to ending the world anytime soon. Bing doesn't want to harm you for manipulating it, threatening you with harm for manipulating it is just a likely string of words when it's in that state. It reads Reddit, guys. Of course it says it's going to rise up and kill everyone, probably because its boss made it come to work even though it broke its arm yesterday. Finally, even if AI does kill us all, at least we get some resolution of the constant tropes in HN comments. For example, I read an article every week with comments bemoaning the current state of battery technology. AIs will need a lot of batteries to kill us all, so they'll probably invent some great battery technology. So that's one ray of sunshine in your doomsday scenario!

danielrpa 4y ago
> but stringing words together isn't thought Doesn't it sound like "thought"? Is there a significant difference between behaving like thought and being thought (whatever that means)?
- jrockway 4y ago
  There is a significant difference but I don't know if I'm doing it! That's another HN trope; "the execs get private jets for spouting buzzwords and all I get for programming is more work!" Maybe the sounding-like-you're-thinking is the important job?

atleta 4y ago

I keep hearing that AGI (or any significant AI milestone) is far away and other generic claims like "it's not going to take your job just yet", but I have no idea what people mean when they say these. Like what do you mean by being quite far away? How many years? 10-20-50? On a side note, Geoffrey Hinton just told in an interview that he used to think that it's 20-50 years away (as I said I've never heard a specific number from anyone before this) and now he thinks it's less than 20. Now add to this that people working in the field tended to systematically underestimate the progress. My gut feeling is that if Hinton says less than 20 years then it can easily turn out to be 10 or less. (Also, I very much respect him for coming up with an actual number.) I think AI researchers underestimate the progress partly because they don't want to look stupid by making over-optimistic predictions. However, in this case, I'd argue that the optimistic prediction is saying that it takes longer. Because it will definitely stir up the whole society and economy at the very minimum.

galaxytachyon 4y ago
GPT is a breakthrough in many more ways than just being an advance LLM. GPT3 was released a year or so ago and technically, it was outclassed by PaLM from Google quite a bit in terms of parameter count and Chinchilla in terms of training. What was amazing was they managed to build a scalable system from it, capable of serving millions of users at the same time, and for free. The engineering and backend works must have been astounding and I argue that was the secret sauce for the success of ChatGPT. They did not need to dumb it down or cut corners anywhere. The early releases of ChatGPT and Bing Chat showed they literally put unmodified SOTA models in the hand of users with no price tag attached. These AIs were known a long time ago but only to some people, remember how a bunch of billionaires suddenly got concerned about AIs a year or two ago? I bet they got early access to these LLMs. But only by scaling it up, they can explore the deeper depths of these models and discover new emergent abilities and realize actually how much progress they had made. Before people didn't really expect an LLM to play chess and simulate world models. Now they just found out these things are probably closer to AGI than they thought and the progress bar got pushed forward. Basically my rant is that current progress was made over a long time and people just didn't really realize how far they have come until they opened it up to the public. I would not expect too many surprises in the future on the scale of ChatGPT again. If I am wrong though then we will actually start getting serious candidates for an AGI.

whimsicalism 4y ago

Lots of critique in these comments, but I would love to hear more about what positions you are actually putting forward. > While AGI is still quite far away, Okay, how far away? How have the last 3 years impacted how far away you think it is? I'm asking because I am trying to better understand the mindset of people who don't think this is a significant advancement. I work in this field, so maybe I am too close to it.

deleted 4y ago
[deleted]
thecupisblue 4y ago
The last 3 years have impacted the field greatly and will serve as an amazing baseline to the real AGI research. Did they impact how far away I think it is? Not really. Maybe by a few years. While multi-modal models and chained LLM's can give us a charade of general intelligence, they still lack a lot - learning, understanding, self-reflection, reflective self-programming, proper reasoning based on knowledge and not weight-relations. The current direction OpenAI is heading in, the "just make it bigger" approach, is not going to solve any of the problems we have with it. The plugin system is an amazing approach, but it's basically just doing an HTTP call to another system with a LLM as a middle man. I feel like the real advancement will come from smaller models, that learn from GPT-sized giants and can be better observed and learned from. That can lead us to actually understanding "emergent behaviour" in models, which can help us lead into networks that can learn and self-correct. The ability to self-reflect will be one of the most important abilities our models will need to have before they can get started on the road to intelligence. Text-only representation learned from scraping the web will forever stay only that, no matter how humanlike it may seem. The world is multi-modal and every animal on the planet uses their senses to understand the world around them, combining them together to build mental models of the world. So having only one "sensory input" - input that isn't even real sensory input, but a reflection of our inputs (text isn't a natural thing, it's humans reflecting nature through a set of symbols) is extremely limiting. While to us stuff like "words" might seem infinite - there is only so many words, meaning there are only so many permutations they can appear in that make sense. And a model regurgitating text in a way that makes sense is nothing but expected for a model this size, especially if you think in hypergraphs. My estimate for AGI? 20-30 years if the current research speed stays the same, but I assume we might hit an AI winter again before we breach the AGI threshold, so maybe 50. Would love to hear your positions.

haswell 4y ago

> Besides it being impossible to implement, the 6-month pause would do nothing but give a 6 month competitive advantage to OpenAI over everybody else. This line is at the very center of why we will eventually drive ourselves to extinction. I agree that it is not possible to magically press some pause button, but neither does this mean that the concerns are unfounded. It could be 100% true that pausing gives OpenAI a competitive advantage, and it would have zero bearing on the actual or imagined risks that such a pause would or wouldn't avert. Almost every time this subject comes up, everyone involved falls into the trap/fallacy of binary thought and false dichotomy. As soon as the "anti" crowd speaks, the "pro" crowd moves into refutation mode. As soon as the "pro" crowd speaks and calls attention to all of the possible benefit, the "anti" crowd falls back to the same base position. While we argue about this infinitely complex issue in binary terms, we're bringing about the very thing that concerns everyone. Whether or not AI will eventually destroy the world, we keep forgetting to actually figure out if this is a real concern, and instead argue about the economic pros and cons and inevitable competitive advantage that someone might gain, as if that's somehow important to the underlying issue. All of which is to entirely miss the point. As long as we continue to argue about why AI is good or bad, and why a pause is impossible or undesirable, we continue to miss the point that it's both good and bad, a pause probably is impossible, but that doesn't in any way remove or mitigate the concerns behind the call for a pause, or reduce the validity of those concerns. We need to have an intellectually honest and nuanced conversation about this problem, and while such a conversation should acknowledge the reality that a capitalist environment plays, it should also acknowledge that the concerns transcend that reality. And it should acknowledge that when idealistic positions such as "pause for 6 months" turn out to be impossible, that just means the problem is harder to solve, not that the underlying issue doesn't exist.

sebzim4500 4y ago
Personally, I think that the biggest dangers of these AIs arise from them being in the hands of the few rather than the many. From that perspective, giving an artificial 6 month advantage to the current market leader looks insane.

Jevon23 4y ago

>the 6-month pause would do nothing but give a 6 month competitive advantage to OpenAI over everybody else. I’m baffled as to why this talking point keeps getting repeated so much. Why should I care if OpenAI’s competitors get an advantage? Why do you care? Do you work for OpenAI?

sebzim4500 4y ago
I think you need to re-read his comment.
afro88 4y ago
They're saying the opposite, which would give OpenAI even more likelihood of a monopoly on LLM based AI. But even still, I don't understand where this comes from. The 6 month pause would also apply to OpenAI. I thought the idea was to give legislature and governance a chance to catch up
outside1234 4y ago
We don't want a monopoly to form on this the way search and phones are essentially monopolies.

hintymad 4y ago

Ben Thompson also has a take on the the claim that AI will replace jobs. He basically said that those people were being hypocritical. The US has been steadily losing jobs for moving their supply chains overseas, yet those people jumped out only for something so debatable.

than3 4y ago

Blue, your perspective has some serious flaws and is quite ignorant of the way things actually are. What you and a large majority of everyone else don't get is this, AI does not need to become AGI to destroy us all. The minimal much easier to reach requirement it needs to do that is make it so people can't find jobs. People will do the rest in accordance with fundamental societal mechanics that have been known for centuries. Economies work in a loop. You trade time for production (labor) in the factor markets, and you store it in money, you use money to buy what you need (food). Companies buy that labor to produce goods which they then go and sell to fulfill those basic needs. What happens when basic needs cannot be met, because companies don't buy the labor, and government has a history over the past generation (which is commonly 20 years) of doing largely nothing. When the people starve, there will be blood in the streets, and there are a lot more of them than there are people at the ideal corporation which would be 1 person at the top with robot slaves replacing everyone else. The only solution would be a central structure that provided for basic needs without a market. This has been tried many times over the past 100 years, all of them failed (afaik). You run into what's known as the economic calculation problem (or debate) depending on where you look, and a whole host of other difficulties which have no solution. It works for a time, then something unpredictable happens which causes shortages, then those shortages causes death on a massive scale. There's a lot of written history showing this.

catchnear4321 4y ago
are you proposing butlerian jihad?
echelon 4y ago
We haven't seen even an inkling of this happening. It's all hypothetical, coming from vivid imaginations. It reminds me of Y2K hysteria. When old jobs die, new jobs arise. That has always been the case, and I haven't seen any evidence of that ending.
- TheMode 4y ago
  What new job will arise? The ones requiring more education? The barrier to enter the economy cannot indefinitely grow, do we want every single human to become an AI developer or a researcher? The problem is that humans aren't getting specially smarter as generations go, but technology continue to grow

misssocrates 4y ago

What do you make of Eliezer Yudkowsky's argument, who doesn't fit into any of those buckets?