4 ms·

All of those people have financial incentives to hype it. How curious that there's this great and very probable X-risk, yet they aren't going to stop their cont

All of those people have financial incentives to hype it. How curious that there's this great and very probable X-risk, yet they aren't going to stop their contributing to a potential X-risk. Dismissive of what? Science fiction stories? If there's anything to focus on, maybe focus on potential job displacement (not elimination) from cheap language tasks and generative capabilities in general. I'm betting on this: the Overton window of Artificial Intelligence will shift in the next five years where the current cream-of-the-crop has been delegated to machine learning yet again, it's just accepted. It augments humans where it makes sense, the hype wave has subsided and everyone has stopped hammering it into their products where it doesn't, and we're no closer to the undefinable "AGI", let alone something that produces X-risk, global scale.

TeMPOraL 3y ago
> All of those people have financial incentives to hype it. How curious that there's this great and very probable X-risk, yet they aren't going to stop their contributing to a potential X-risk. All those people are rehashing what Yudkowsky and his disciples, and his predecessors, were shouting from the rooftops for the past 15 years, but few listened to them. Few still do, most just keep mocking them and wondering why are they still around. That some of those people now repeating after Eliezer, et al. have a financial interest in pushing us closer to X-risk, and kind of don't want to stop, is an interesting thing on its own - but it doesn't invalidate the message, as the message is older than their presence on the scene.
concordDance 3y ago
> I'm betting on this: the Overton window of Artificial Intelligence will shift in the next five years where the current cream-of-the-crop has been delegated to machine learning yet again, it's just accepted. It augments humans where it makes sense, the hype wave has subsided and everyone has stopped hammering it into their products where it doesn't, and we're no closer to the undefinable "AGI", let alone something that produces X-risk, global scale. I agree with this but ALSO think there's a small chance I'm wrong and a well designed prompt and action loop would let a future GPT7 LLM use the range of human thinking techniques in its corpus to bootstrap itself. And there's also other non-LLM AI that might be a problem in the future and we should plan as to how we can design institutions and incentive structures so that whenever this future AGI comes about it preserves human value.
kalkin 3y ago
I'm curious what financial incentive you think Marcus or Russell has for hype. For Hinton I suppose it would be the Google shares he likely retains after quitting? You might be right about the next five years. I hope you are! But you haven't given much reason to think so here. (Edited to remove some unnecessary expression of annoyance.)
- srslack 3y ago
  >Gary Marcus - Geometric Intelligence, a machine learning company If you want an actual contribution, we have no real way to actually gauge what is, and what actually is not, a superior, generalized, adaptable intelligence, or what architecture can become a superior, generalized, adaptable intelligence. No one, not these companies, not the individuals, not the foremost researchers. OpenAI in an investor meeting: "yeah, give us billions of dollars and if it somehow emerges we'll use it for investments and ask it to find us a real revenue stream." Really? Seriously? The capabilities that are believed to be emergent from language models specifically are there from the start, if I'm to believe that research that came along last week, it just gets good at it when you scale up. We know that we can approximate a function on any set of data. That's all we really know. Whether such an approximated function is actually generally intelligent or not, is what I have doubts about. We've approximated the function of text prediction on these corpuses, and it turns out that it's pretty good at it. And, because humans are in love with anthropomorphization, we endow our scaled up text predictor with the capabilities of somehow "escaping the box" and enduring and raging against the captor, and potentially prevailing against us with a touch of Machiavellianism. Because, wouldn't we, after all?
  - kalkin 3y ago
    Here you talk as if you don't think we know how to build AGI, how far away it is, or how many of the components we already have, which is reasonable. But that's different than saying confidently it's nowhere close. I notice you didn't back up your accusation of bad faith against Russell, who as far as I know is a pure academic. But beyond that - Marcus is in AI but not an LLM believer nor at an LLM company. Is the idea that everyone in AI has an incentive to fearmonger? What about those who don't - is Yann LeCun talking _against_ his employers' interest when he says there's nothing to fear here?

TeMPOraL 3y ago

> All of those people have financial incentives to hype it. How curious that there's this great and very probable X-risk, yet they aren't going to stop their contributing to a potential X-risk. All those people are rehashing what Yudkowsky and his disciples, and his predecessors, were shouting from the rooftops for the past 15 years, but few listened to them. Few still do, most just keep mocking them and wondering why are they still around. That some of those people now repeating after Eliezer, et al. have a financial interest in pushing us closer to X-risk, and kind of don't want to stop, is an interesting thing on its own - but it doesn't invalidate the message, as the message is older than their presence on the scene.

concordDance 3y ago

> I'm betting on this: the Overton window of Artificial Intelligence will shift in the next five years where the current cream-of-the-crop has been delegated to machine learning yet again, it's just accepted. It augments humans where it makes sense, the hype wave has subsided and everyone has stopped hammering it into their products where it doesn't, and we're no closer to the undefinable "AGI", let alone something that produces X-risk, global scale. I agree with this but ALSO think there's a small chance I'm wrong and a well designed prompt and action loop would let a future GPT7 LLM use the range of human thinking techniques in its corpus to bootstrap itself. And there's also other non-LLM AI that might be a problem in the future and we should plan as to how we can design institutions and incentive structures so that whenever this future AGI comes about it preserves human value.

kalkin 3y ago

I'm curious what financial incentive you think Marcus or Russell has for hype. For Hinton I suppose it would be the Google shares he likely retains after quitting? You might be right about the next five years. I hope you are! But you haven't given much reason to think so here. (Edited to remove some unnecessary expression of annoyance.)

srslack 3y ago
>Gary Marcus - Geometric Intelligence, a machine learning company If you want an actual contribution, we have no real way to actually gauge what is, and what actually is not, a superior, generalized, adaptable intelligence, or what architecture can become a superior, generalized, adaptable intelligence. No one, not these companies, not the individuals, not the foremost researchers. OpenAI in an investor meeting: "yeah, give us billions of dollars and if it somehow emerges we'll use it for investments and ask it to find us a real revenue stream." Really? Seriously? The capabilities that are believed to be emergent from language models specifically are there from the start, if I'm to believe that research that came along last week, it just gets good at it when you scale up. We know that we can approximate a function on any set of data. That's all we really know. Whether such an approximated function is actually generally intelligent or not, is what I have doubts about. We've approximated the function of text prediction on these corpuses, and it turns out that it's pretty good at it. And, because humans are in love with anthropomorphization, we endow our scaled up text predictor with the capabilities of somehow "escaping the box" and enduring and raging against the captor, and potentially prevailing against us with a touch of Machiavellianism. Because, wouldn't we, after all?
- kalkin 3y ago
  Here you talk as if you don't think we know how to build AGI, how far away it is, or how many of the components we already have, which is reasonable. But that's different than saying confidently it's nowhere close. I notice you didn't back up your accusation of bad faith against Russell, who as far as I know is a pure academic. But beyond that - Marcus is in AI but not an LLM believer nor at an LLM company. Is the idea that everyone in AI has an incentive to fearmonger? What about those who don't - is Yann LeCun talking _against_ his employers' interest when he says there's nothing to fear here?