5 ms·

I don't agree with all of the OP's arguments, but wow, what a great little piece of writing! As the OP points out, the "accelerators vs doomers" debate in AI h

by cs702 3y ago

I don't agree with all of the OP's arguments, but wow, what a great little piece of writing! As the OP points out, the "accelerators vs doomers" debate in AI has more than a few similarities with the medieval debates about the nature of angels.

ssss11 3y ago
You sound like you have some knowledge to share and I know nothing about the medieval debates about the nature of angels! Could you elaborate please?
- dllthomas 3y ago
  I believe parent is referencing https://en.wikipedia.org/wiki/How_many_angels_can_dance_on_the_head_of_a_pin%3F https://en.wikipedia.org/wiki/How_many_angels_can_dance_on_t...
  - mmcdermott 3y ago
    That same Wikipedia article casts some doubt about whether the question "How many angels can dance on the head of a pin?" was really a question of serious discussion. From the same entry: > However, evidence that the question was widely debated in medieval scholarship is lacking.[5] One theory is that it is an early modern fabrication,[a] used to discredit scholastic philosophy at a time when it still played a significant role in university education.
    - hoerensagen 3y ago
      The answer is: One if it's the gavotte
    - skeaker 3y ago
      Sure, but the point of the phrase is that the question itself is a waste of time.
    - empath-nirvana 3y ago
      It's really just short hand for rationalist debate in general, which is what the scholastics were engaged in. Once you decide you _know_ certain things, then you can end up with all kinds of frankly nutty beliefs based on those priors, following a rock solid chain of rational logic all along, as long as your priors are _wrong_. Scholastics ended up having debates about the nature of the universal intellect or angels or whatever, and rationalists today argue about super human AI. That's really the problem with rationalist discourse in general. A lot of them start with what they want to argue, and then use whatever assumptions they need to start building a chain of rationalist logic to support that outcome. Clearly a lot of "effective altruists" for example, want to argue that the most altruistic thing they could possibly be doing is to earn as much money as they possibly can and horde as much wealth as they possibly can, so they'll come up with a tower of logic based on far-fetched ideas like making humans an interplanetary species or hyperintelligent AIs, or life extension or whatever so they can come up with absurd arguments like: If we don't end up an interplanetary species, billions and billions billions of people will never be born so that's obviously the most important thing anybody could ever be working on, so who cares about that kid starving in Africa right now. He's not going to be a rocket scientist, what good is he? One thing most philosophers learned at some point is that you need to temper rationalism with a lot of humility because every chain of logic has all kinds of places where it could be wrong, and any of those being wrong is catastrophic to the outcome of your logic.
      - JohnFen 3y ago
        > Once you decide you _know_ certain things, then you can end up with all kinds of frankly nutty beliefs based on those priors, following a rock solid chain of rational logic all along, as long as your priors are _wrong_. This mechanism is exactly why the more intelligent a person is, the more likely they are to believe very weird things. They can more easily assemble a chain of rational logic to lead them to whatever it is that they want to believe.
        lazide 3y ago
        If we’re rationalizing animals (instead of rational), then it follows that the more power/ability someone has to be rational, the more power they have to retcon (essentially) what they want to do as okay. (Rationalize it) Very much a double edged sword.
    - aprilthird2021 3y ago
      The point is that religions at the time had a logical framework which the scholars liked to interrogate and play with the logic of even if that served no real world purpose. Likewise, fighting about doom vs accel when current day Gen AI is nowhere close to that kind of stuff (and hasn't shown it can ever be) is kind of pointless
gumby 3y ago
> wow, what a great little piece of writing! If you like this essay from The Economist, note that this is the standard level of quality for that magazine (or, as they call themselves for historical reasons, "newspaper"). I've been a subscriber since 1985.
- cs702 3y ago
  Long-time occasional reader. The level of quality is excellent, I agree.
nradov 3y ago
Brief in the imminent arrival of super intelligent AGI that will transform society is essentially a new secular religion. The technological cognoscenti who believe in dismiss the doubters who insist on evidence as fools. "Surely I come quickly. Amen."
- vlovich123 3y ago
  I would say a definition for GAI is a system that can improve its own ability to adapt to new problems. That’s a more concrete formulation than I’ve typically seen. Currently humans are still in the loop, but we already have AI enabling advancements in their own functioning at a very primitive level. Extrapolating from previous growth is a form of belief without evidence since past performance not indicative of future results. But that’s generally true of all prognostication and I’m not sure what kind of evidence you’d be looking for aside from past performance. The doubters are dismissed as naive thinking that something is outside our ability to achieve something, but that’s only if you keep moving goalposts and treat it like Zeno’s paradox. Like yes, there are weaknesses to our current techniques. At the same time we’ve also demonstrated an uncanny ability to step around them and reach new heights. For example, our ability to beat Go took less time than it took to develop techniques to beat humans at chess. Automation now outcompetes humans at many many things that seemed impossible before. Techniques / solutions will also be combined to solve even harder problems (eg now LLMs are being researched to take over executive command control operations of robots for example instead of using classical control systems algorithms that were hand built and hand tuned)
- ben_w 3y ago
  Automation has been radically changing our societies since before Marx wrote down some thoughts and called it communism. Things which used to be considered AI before we solved them, e.g. automated optimisation of things like code compilation or CPU layouts, have improved our capacity to automate design and testing of what is now called AI. Could stop at any point. I'll be very surprised if someone makes a CPU with more than one transistor per atom. But even if development stops right now, our qualification systems haven't caught up (and IMO can't catch up) with LLMs. Might need to replace them with mandatory 5 years internships to get people beyond what is now the "junior" stage in many professions — junior being approximately the level which the better existing LLMs can respond at. "Transform society" covers a lot more than anyone's idea of the singularity.

by cs702 3y ago

ssss11 3y ago

You sound like you have some knowledge to share and I know nothing about the medieval debates about the nature of angels! Could you elaborate please?

dllthomas 3y ago
I believe parent is referencing https://en.wikipedia.org/wiki/How_many_angels_can_dance_on_the_head_of_a_pin%3F https://en.wikipedia.org/wiki/How_many_angels_can_dance_on_t...
- mmcdermott 3y ago
  That same Wikipedia article casts some doubt about whether the question "How many angels can dance on the head of a pin?" was really a question of serious discussion. From the same entry: > However, evidence that the question was widely debated in medieval scholarship is lacking.[5] One theory is that it is an early modern fabrication,[a] used to discredit scholastic philosophy at a time when it still played a significant role in university education.
  - hoerensagen 3y ago
    The answer is: One if it's the gavotte
  - skeaker 3y ago
    Sure, but the point of the phrase is that the question itself is a waste of time.
  - empath-nirvana 3y ago
    It's really just short hand for rationalist debate in general, which is what the scholastics were engaged in. Once you decide you _know_ certain things, then you can end up with all kinds of frankly nutty beliefs based on those priors, following a rock solid chain of rational logic all along, as long as your priors are _wrong_. Scholastics ended up having debates about the nature of the universal intellect or angels or whatever, and rationalists today argue about super human AI. That's really the problem with rationalist discourse in general. A lot of them start with what they want to argue, and then use whatever assumptions they need to start building a chain of rationalist logic to support that outcome. Clearly a lot of "effective altruists" for example, want to argue that the most altruistic thing they could possibly be doing is to earn as much money as they possibly can and horde as much wealth as they possibly can, so they'll come up with a tower of logic based on far-fetched ideas like making humans an interplanetary species or hyperintelligent AIs, or life extension or whatever so they can come up with absurd arguments like: If we don't end up an interplanetary species, billions and billions billions of people will never be born so that's obviously the most important thing anybody could ever be working on, so who cares about that kid starving in Africa right now. He's not going to be a rocket scientist, what good is he? One thing most philosophers learned at some point is that you need to temper rationalism with a lot of humility because every chain of logic has all kinds of places where it could be wrong, and any of those being wrong is catastrophic to the outcome of your logic.
    - JohnFen 3y ago
      > Once you decide you _know_ certain things, then you can end up with all kinds of frankly nutty beliefs based on those priors, following a rock solid chain of rational logic all along, as long as your priors are _wrong_. This mechanism is exactly why the more intelligent a person is, the more likely they are to believe very weird things. They can more easily assemble a chain of rational logic to lead them to whatever it is that they want to believe.
      - lazide 3y ago
        If we’re rationalizing animals (instead of rational), then it follows that the more power/ability someone has to be rational, the more power they have to retcon (essentially) what they want to do as okay. (Rationalize it) Very much a double edged sword.
  - aprilthird2021 3y ago
    The point is that religions at the time had a logical framework which the scholars liked to interrogate and play with the logic of even if that served no real world purpose. Likewise, fighting about doom vs accel when current day Gen AI is nowhere close to that kind of stuff (and hasn't shown it can ever be) is kind of pointless

gumby 3y ago

> wow, what a great little piece of writing! If you like this essay from The Economist, note that this is the standard level of quality for that magazine (or, as they call themselves for historical reasons, "newspaper"). I've been a subscriber since 1985.

cs702 3y ago
Long-time occasional reader. The level of quality is excellent, I agree.

nradov 3y ago

Brief in the imminent arrival of super intelligent AGI that will transform society is essentially a new secular religion. The technological cognoscenti who believe in dismiss the doubters who insist on evidence as fools. "Surely I come quickly. Amen."

vlovich123 3y ago
I would say a definition for GAI is a system that can improve its own ability to adapt to new problems. That’s a more concrete formulation than I’ve typically seen. Currently humans are still in the loop, but we already have AI enabling advancements in their own functioning at a very primitive level. Extrapolating from previous growth is a form of belief without evidence since past performance not indicative of future results. But that’s generally true of all prognostication and I’m not sure what kind of evidence you’d be looking for aside from past performance. The doubters are dismissed as naive thinking that something is outside our ability to achieve something, but that’s only if you keep moving goalposts and treat it like Zeno’s paradox. Like yes, there are weaknesses to our current techniques. At the same time we’ve also demonstrated an uncanny ability to step around them and reach new heights. For example, our ability to beat Go took less time than it took to develop techniques to beat humans at chess. Automation now outcompetes humans at many many things that seemed impossible before. Techniques / solutions will also be combined to solve even harder problems (eg now LLMs are being researched to take over executive command control operations of robots for example instead of using classical control systems algorithms that were hand built and hand tuned)
ben_w 3y ago
Automation has been radically changing our societies since before Marx wrote down some thoughts and called it communism. Things which used to be considered AI before we solved them, e.g. automated optimisation of things like code compilation or CPU layouts, have improved our capacity to automate design and testing of what is now called AI. Could stop at any point. I'll be very surprised if someone makes a CPU with more than one transistor per atom. But even if development stops right now, our qualification systems haven't caught up (and IMO can't catch up) with LLMs. Might need to replace them with mandatory 5 years internships to get people beyond what is now the "junior" stage in many professions — junior being approximately the level which the better existing LLMs can respond at. "Transform society" covers a lot more than anyone's idea of the singularity.