3 ms·

This makes sense where you're coming from. I still think you're doing a bit of disservice to some of the uncertainty estimation methods for DL. Batch normalizat

by joxel 4y ago

This makes sense where you're coming from. I still think you're doing a bit of disservice to some of the uncertainty estimation methods for DL. Batch normalization, ensembling, dropout etc. have shown to be very close approximations for Bayesian methods which, as far as I know, are probably the closest we get to mathematically modeling "real-world probability". If you have information that goes against this I'd love to hear it because I'm just relating my understanding of the situation at this time.

godelski 4y ago
> If you have information that goes against this I'd love to hear it because I'm just relating my understanding of the situation at this time. I do and I wrote it. My example is not one that is specific to DL or any type of architecture. My example works for any arbitrary statistical model. What you're missing is a nuanced difference between word definitions. Likelihood and probability are not the same thing. (You're also clearly misreading those papers and have not dug much into DL because there's tons of models that work on modeling probabilities directly, and this is specifically the area I work in)
- joxel 4y ago
  Do you have some papers of DL that models probabilities directly? You're not just talking about Bayesian Neural Networks right? I've had ideas for having a neural network predict both its target and its estimation of error (by using the actual residual in the loss function to compare to predicted error estimation). Is that similar to what you're talking about?
  - godelski 4y ago
    A few thousand in fact. Look for Ian Goodfellow's taxonomy. I'd have more patience but you're demonstrating a clear lack of domain knowledge but spoke with extremely high confidence earlier.
    - joxel 4y ago
      Has anyone ever told you that you’re an asshole? Imagine telling someone you don’t want to talk to them because of hubris while talking to them like you currently are. I also work in DL, I’m an early career employee and have some experience and am interested to learn. I also have publications involving combinations of uncertainty methods and PINNs and show that certain applications of uncertainty modeling do work in those environments for practical solutions. The way you are describing things is just a little different than what I’m used to, and you decided to use that to insult and belittle me.

by joxel 4y ago

godelski 4y ago

> If you have information that goes against this I'd love to hear it because I'm just relating my understanding of the situation at this time. I do and I wrote it. My example is not one that is specific to DL or any type of architecture. My example works for any arbitrary statistical model. What you're missing is a nuanced difference between word definitions. Likelihood and probability are not the same thing. (You're also clearly misreading those papers and have not dug much into DL because there's tons of models that work on modeling probabilities directly, and this is specifically the area I work in)

joxel 4y ago
Do you have some papers of DL that models probabilities directly? You're not just talking about Bayesian Neural Networks right? I've had ideas for having a neural network predict both its target and its estimation of error (by using the actual residual in the loss function to compare to predicted error estimation). Is that similar to what you're talking about?
- godelski 4y ago
  A few thousand in fact. Look for Ian Goodfellow's taxonomy. I'd have more patience but you're demonstrating a clear lack of domain knowledge but spoke with extremely high confidence earlier.
  - joxel 4y ago
    Has anyone ever told you that you’re an asshole? Imagine telling someone you don’t want to talk to them because of hubris while talking to them like you currently are. I also work in DL, I’m an early career employee and have some experience and am interested to learn. I also have publications involving combinations of uncertainty methods and PINNs and show that certain applications of uncertainty modeling do work in those environments for practical solutions. The way you are describing things is just a little different than what I’m used to, and you decided to use that to insult and belittle me.