5 ms·

Either Nassim is missing the point or you are misquoting him. I think this is more about should experts be in charge of their own domain.

dist-epoch 3y ago
I'm not misquoting him. Nassim is arguing that RISK is a separate discipline, separate from the domain where risk applies. That a person building AI is not the correct choice for estimating AI risk. You don't ask gun making companies to make policies regarding risk of gun owning in society.
- petermcneeley 3y ago
  Perhaps that is Nassim's argument because he is an expert in risk. The actual issue with experts monitoring experts is the bias induced by self interest (see previous sentence). Nassim sort of gets this when it comes to his discussion of "skin in the game". I dont want to malign Nassim too much but he seems like the kind of person that "sorta gets it" when it comes to everything that matters.
  - dist-epoch 3y ago
    AI researchers also have a vested interest in saying their technology is safe.
    - petermcneeley 3y ago
      Does Douglas Hofstadter?
- jll29 3y ago
  I agree with this, and with Nassim Taleb's point. Having worked in the medical domain in the past, paramedics and medics that should know better were taking extremely high health-related risks (riding a motor bike => crashing and burning to death, smoking => dying from lung canceer, speeding onto a crossing => dying in an ambulance crash before arriving at the 999 call site etc.). So risk is indeed its own discipline, separate from the domain where risk applies, even if we are talking about the life-rescuing domain of medicine: a person rescuing another is not automatically an expert at reducing their own (health/life) risk exposure. While neural network research results are published in NeurIPS, ICLR, ICML, ECML/PKDD, JMLR etc., risk results tend to get published in the risk community at conferences like SRA [1] (Europe: SRA-E) and the likes. I'm not a fan of this academic segregation, merely describing what is going on (in my own teaching, for instance, I include risk/ethics consideration along the way with teaching the technical side, to avoid ignorance caused by over-compartmentalization). [1] Annual Meeting of the Society of Risk Analysis, https://www.sra.org/events-webinars/annual-meeting/ https://www.sra.org/events-webinars/annual-meeting/
  - petermcneeley 3y ago
    But then how does that jive with skin in the game? In his Nassim's teaching he brings up the example of the idea that the Roman engineers that built/designed the bridges were forced to have their families live under the bridges. It sounds like those engineers directly involved in the practice do understand the risk of their domains it is merely their incentives that need to be aligned.
    - dist-epoch 3y ago
      > But then how does that jive with skin in the game Simple. Nassim says there are 4 quadrants, one axis Mediocristan-Extremistan, the other Simple-Complex payoff. Building a bridge is Mediocristan/Simple payoff, a well understood problem with no black swans. So it's easy to compute risk. Other stuff is Extremistan/Complex payoff - financial trading, pandemics, AI. And he argues that you need RISK professionals for this quadrant, because people working here (traders, virologists, AI builders) do not understand how to compute the risk correctly. https://www.researchgate.net/profile/Gaute-Bjorklund-Wangen/publication/318853336/figure/fig4/AS:931308163301378@1599052577229/Talebs-four-quadrants-for-Risk-Classification-Based-on-Taleb143.ppm https://www.researchgate.net/profile/Gaute-Bjorklund-Wangen/...
      - petermcneeley 3y ago
        You think the virologists do not understand the risks? And what are the biases of the risk professionals? What are their motives?
- ke88y 3y ago
  > Nassim is arguing that RISK is a separate discipline, separate from the domain where risk applies. That a person building AI is not the correct choice for estimating AI risk. I think this is often fair. It's actually one of my primary criticisms of the NeurIPS/ICML survey. FWIW, people working on AI Safety -- like, actually working on it, not philosophizing about it -- are some of the most incredulous and annoyed about the "AGI => extinction" crowd.

dist-epoch 3y ago

I'm not misquoting him. Nassim is arguing that RISK is a separate discipline, separate from the domain where risk applies. That a person building AI is not the correct choice for estimating AI risk. You don't ask gun making companies to make policies regarding risk of gun owning in society.

petermcneeley 3y ago
Perhaps that is Nassim's argument because he is an expert in risk. The actual issue with experts monitoring experts is the bias induced by self interest (see previous sentence). Nassim sort of gets this when it comes to his discussion of "skin in the game". I dont want to malign Nassim too much but he seems like the kind of person that "sorta gets it" when it comes to everything that matters.
- dist-epoch 3y ago
  AI researchers also have a vested interest in saying their technology is safe.
  - petermcneeley 3y ago
    Does Douglas Hofstadter?
jll29 3y ago
I agree with this, and with Nassim Taleb's point. Having worked in the medical domain in the past, paramedics and medics that should know better were taking extremely high health-related risks (riding a motor bike => crashing and burning to death, smoking => dying from lung canceer, speeding onto a crossing => dying in an ambulance crash before arriving at the 999 call site etc.). So risk is indeed its own discipline, separate from the domain where risk applies, even if we are talking about the life-rescuing domain of medicine: a person rescuing another is not automatically an expert at reducing their own (health/life) risk exposure. While neural network research results are published in NeurIPS, ICLR, ICML, ECML/PKDD, JMLR etc., risk results tend to get published in the risk community at conferences like SRA [1] (Europe: SRA-E) and the likes. I'm not a fan of this academic segregation, merely describing what is going on (in my own teaching, for instance, I include risk/ethics consideration along the way with teaching the technical side, to avoid ignorance caused by over-compartmentalization). [1] Annual Meeting of the Society of Risk Analysis, https://www.sra.org/events-webinars/annual-meeting/ https://www.sra.org/events-webinars/annual-meeting/
- petermcneeley 3y ago
  But then how does that jive with skin in the game? In his Nassim's teaching he brings up the example of the idea that the Roman engineers that built/designed the bridges were forced to have their families live under the bridges. It sounds like those engineers directly involved in the practice do understand the risk of their domains it is merely their incentives that need to be aligned.
  - dist-epoch 3y ago
    > But then how does that jive with skin in the game Simple. Nassim says there are 4 quadrants, one axis Mediocristan-Extremistan, the other Simple-Complex payoff. Building a bridge is Mediocristan/Simple payoff, a well understood problem with no black swans. So it's easy to compute risk. Other stuff is Extremistan/Complex payoff - financial trading, pandemics, AI. And he argues that you need RISK professionals for this quadrant, because people working here (traders, virologists, AI builders) do not understand how to compute the risk correctly. https://www.researchgate.net/profile/Gaute-Bjorklund-Wangen/publication/318853336/figure/fig4/AS:931308163301378@1599052577229/Talebs-four-quadrants-for-Risk-Classification-Based-on-Taleb143.ppm https://www.researchgate.net/profile/Gaute-Bjorklund-Wangen/...
    - petermcneeley 3y ago
      You think the virologists do not understand the risks? And what are the biases of the risk professionals? What are their motives?
ke88y 3y ago
> Nassim is arguing that RISK is a separate discipline, separate from the domain where risk applies. That a person building AI is not the correct choice for estimating AI risk. I think this is often fair. It's actually one of my primary criticisms of the NeurIPS/ICML survey. FWIW, people working on AI Safety -- like, actually working on it, not philosophizing about it -- are some of the most incredulous and annoyed about the "AGI => extinction" crowd.