3 ms·

> He sees these scientists as strong men, willing to wrestle with the tragedy of existence by increasing the amount of happiness in the world. That struggle mak

> He sees these scientists as strong men, willing to wrestle with the tragedy of existence by increasing the amount of happiness in the world. That struggle makes them optimists. Loved that.

shermantanktop 1mo ago
He points out the value of introspection, which the literary culture has in spades, but the scientific culture utterly lacks. If we get the worst outcomes from AGI, the scientists who worked so hard to enable it are unlikely to reflect deeply about it. If they had that capacity, they might do it ahead of time. And some do…but not enough to stop the inevitable tide. Ironic, given the role that chance and statistical prediction plays in the mechanisms of AI. Free will is in fact a thing, and moral choices have moral consequences.
- linkregister 1mo ago
  The scientific method is inherently introspective. Studies seek to affirm or falsify the null hypothesis. Those studies are subject to peer review. Verification and falsifiability are baked into every step. There are numerous examples of fraud in science. When this occurs it is a scandal. There will certainly continue to be fraudulent papers being published. Hopefully these will be caught and retracted. This comment is probably not talking about peer review and verification, however. The notion is likely that scientists are progress-oriented and not thinking whether an advancement should be done, not whether it can be done. However, for virtually every scientific advancement, the leading scientists write extensively about the implications of the technology! Specifically to the commenter's example of AI research, it's trivial to find long essays by the leading AI researchers, company leaders, and venture capitalists. It's more uncommon for an accomplished AI researcher to not have a series of essays about their work. Contrary to the assertion that AGI scientists are not introspecting, their introspection dominates discussion on this website.
  - shermantanktop 1mo ago
    The only important question in this space is how to balance the speed of development of this technology with safety. And globally, as a human race, our answer is: speed. Yes, lots of AI scientists spend lots of time and energy on safety. But all the incentives (economic, reputational, professional, geopolitical) are on the side of speed. So sure, there are introspective AI scientists. I fear their role in the history of this period will be a footnote about cautionary voices.
- AlexandrB 1mo ago
  > He points out the value of introspection, which the literary culture has in spades, but the scientific culture utterly lacks. Maybe this was true at one point but I don't think modern literary culture has much capacity to be introspective. The politicization of academia hit the liberal arts harder than anywhere else[1], and when it comes to politically charged topics there's little room for disagreement left. Introspection is not possible if questioning the status quo can be a career-ender[2]. [1] https://www.independent.org/tir/2022-23-winter/the-hyperpoliticization-of-higher-ed/ https://www.independent.org/tir/2022-23-winter/the-hyperpoli... [2] https://www.fire.org/research-learn/sanctioned-scholars-price-speaking-freely-todays-academy https://www.fire.org/research-learn/sanctioned-scholars-pric...
- JoshTriplett 1mo ago
  > He points out the value of introspection, which the literary culture has in spades, but the scientific culture utterly lacks. I think you're drastically overgeneralizing. There exist many scientists who think about the impact and ethics of their work, and choose what to work on accordingly, as well as trying to discourage or mitigate harms. There aren't necessarily enough, but I wonder how much of that is actually lack of introspection, rather than lack of being in a position where their reflections are able to amount to much. I wish everyone were empowered to follow their ethics without worrying about their next paycheck. > If we get the worst outcomes from AGI, the scientists who worked so hard to enable it are unlikely to reflect deeply about it. If we get the worst outcomes from AGI, which are also the modal outcomes from AGI if we don't thread the needle, there won't be anyone left to reflect.

> He sees these scientists as strong men, willing to wrestle with the tragedy of existence by increasing the amount of happiness in the world. That struggle makes them optimists. Loved that.

shermantanktop 1mo ago

He points out the value of introspection, which the literary culture has in spades, but the scientific culture utterly lacks. If we get the worst outcomes from AGI, the scientists who worked so hard to enable it are unlikely to reflect deeply about it. If they had that capacity, they might do it ahead of time. And some do…but not enough to stop the inevitable tide. Ironic, given the role that chance and statistical prediction plays in the mechanisms of AI. Free will is in fact a thing, and moral choices have moral consequences.

linkregister 1mo ago
The scientific method is inherently introspective. Studies seek to affirm or falsify the null hypothesis. Those studies are subject to peer review. Verification and falsifiability are baked into every step. There are numerous examples of fraud in science. When this occurs it is a scandal. There will certainly continue to be fraudulent papers being published. Hopefully these will be caught and retracted. This comment is probably not talking about peer review and verification, however. The notion is likely that scientists are progress-oriented and not thinking whether an advancement should be done, not whether it can be done. However, for virtually every scientific advancement, the leading scientists write extensively about the implications of the technology! Specifically to the commenter's example of AI research, it's trivial to find long essays by the leading AI researchers, company leaders, and venture capitalists. It's more uncommon for an accomplished AI researcher to not have a series of essays about their work. Contrary to the assertion that AGI scientists are not introspecting, their introspection dominates discussion on this website.
- shermantanktop 1mo ago
  The only important question in this space is how to balance the speed of development of this technology with safety. And globally, as a human race, our answer is: speed. Yes, lots of AI scientists spend lots of time and energy on safety. But all the incentives (economic, reputational, professional, geopolitical) are on the side of speed. So sure, there are introspective AI scientists. I fear their role in the history of this period will be a footnote about cautionary voices.
AlexandrB 1mo ago
> He points out the value of introspection, which the literary culture has in spades, but the scientific culture utterly lacks. Maybe this was true at one point but I don't think modern literary culture has much capacity to be introspective. The politicization of academia hit the liberal arts harder than anywhere else[1], and when it comes to politically charged topics there's little room for disagreement left. Introspection is not possible if questioning the status quo can be a career-ender[2]. [1] https://www.independent.org/tir/2022-23-winter/the-hyperpoliticization-of-higher-ed/ https://www.independent.org/tir/2022-23-winter/the-hyperpoli... [2] https://www.fire.org/research-learn/sanctioned-scholars-price-speaking-freely-todays-academy https://www.fire.org/research-learn/sanctioned-scholars-pric...
JoshTriplett 1mo ago
> He points out the value of introspection, which the literary culture has in spades, but the scientific culture utterly lacks. I think you're drastically overgeneralizing. There exist many scientists who think about the impact and ethics of their work, and choose what to work on accordingly, as well as trying to discourage or mitigate harms. There aren't necessarily enough, but I wonder how much of that is actually lack of introspection, rather than lack of being in a position where their reflections are able to amount to much. I wish everyone were empowered to follow their ethics without worrying about their next paycheck. > If we get the worst outcomes from AGI, the scientists who worked so hard to enable it are unlikely to reflect deeply about it. If we get the worst outcomes from AGI, which are also the modal outcomes from AGI if we don't thread the needle, there won't be anyone left to reflect.