4 ms·

I have limited knowledge of machine learning so forgive me if I am wrong. I recall that some classification techniques provide you a confidence score of the cl

I have limited knowledge of machine learning so forgive me if I am wrong. I recall that some classification techniques provide you a confidence score of the classification... so can't you just treat a classification score lower than a certain threshold as "I don't know"?

warent 8y ago
This question was answered in the comments section of the article. Basically Softmax will only give you values close to 1 or close to 0, so there's no room for your own interpretation or threshold. https://medium.com/@paraschopra/softmax-amplifies-small-differences-so-probabilities-that-come-out-of-it-get-skewed-towards-either-cf1c8c50fab9 https://medium.com/@paraschopra/softmax-amplifies-small-diff...
- anonymousDan 8y ago
  Do you need to do softmax? Why not just look at the underlying amplitudes of the output before applying softmax? Or is it that they won't be normalized?
  - paraschopra 8y ago
    Interesting suggestion, honestly I'm not sure. Although this should be easy to test empirically. I feel if this were to work, it may depend on architecture of the network and what intermediate functions are used (ReLu, sigmoid and tanh). But your suggestion is easy to test empirically and I'll try it. There was another method proposed recently for approximating uncertainties: using dropout during evaluation (which effectively makes different runs on the same image as samples). But I came across a discussion on the efficacy of the technique and I wasn't sure. https://www.reddit.com/r/MachineLearning/comments/7bm4b2/d_what_is_the_current_state_of_dropout_as/ https://www.reddit.com/r/MachineLearning/comments/7bm4b2/d_w... My next project is to implement dropout and see how it compares with the current approach. I'll also try your suggestion.
  - cultus 8y ago
    Yes, they wouldn't be normalized so you can't interpret it as a probability. Softmax is the function that normalizes it. However, even that you can't interpret as a probability of some outcome, because you do not take into account the prior. For that, you need actual Bayesian techniques.
    - anonymousDan 8y ago
      Can you elaborate on what you mean by not taking into account the prior? What is the output of aoftmax if not a probability?
      - cultus 8y ago
        Well, it's a probability of the output given the fixed, fitted model parameters. This is the likelihood. This is not the probability of getting that output, since we don't know what the parameters should really be. See the inverse probability fallacy [0]. This confusion is literally the main reason behind the replication crisis of scientific research. In ML, you typically have a model f(w;u), and you are trying to fit the parameters w to some data, given some hyperparameters u that represent some kind of prior information. In frequentist techniques, these could be regularization parameters. In frequentist techniques, the hyperparameters the fitted parameters are assumed fixed, and not random. In contrast, Bayesian methods assume it is random as well. This extra uncertainty is neglected by the predictions of frequentist techniques. Nonetheless, the uncertainty is real. [0] https://en.wikipedia.org/wiki/Confusion_of_the_inverse https://en.wikipedia.org/wiki/Confusion_of_the_inverse
        anonymousDan 8y ago
        Thanks!
- jeremysalwen 8y ago
  I think that post is quite misleading. 90 and 100 are extreme values representing extreme confidence. Your model could just as easily learn [0.09, 0.10, 0.05] on the final layer, and the probabilities would then be [0.337, 0.340, 0.323]. And look, now softmax has the opposite problem! The final activation layers differed by a factor of 2, but softmax output almost identical probabilities! Except this should not be surprising, because softmax is not scale invariant. Values on the unit scale is what you would normally expect, so if you feed it values 100x larger, it's going to be way more confident. If you feed it values 100x smaller, it's going to be way less confident.
  - paraschopra 8y ago
    This is interesting. In theory, what you are suggesting can be true but in practice you do think the last layer can output 0.1, 0.09, etc especially when last layer is typically not pushed through sigmoid.
    - rjtavares 8y ago
      Isn't that mostly true with highly curated datasets with clearly defined categories?
conjectures 8y ago
No. Probabilities can be random variables themselves. Say p is the probability an image is of a cat. p could be low, e.g. 0.05. However, perhaps p has the 0.9 credible interval (0.01, 0.1). The width of that interval expresses how confident you are of the point estimate. The correct translation of "I don't know" is a wide credible interval.
- dwohnitmok 8y ago
  I disagree with this characterization, especially in a Bayesian context (which credible interval comes from). Parameters into a model can be modeled as random variables. Probabilities themselves cannot be, unless they are themselves parameters (or happen to coincide with a parameter) into another part of a statistical model. A statistical estimation of an event (in the sense of probability theory) must bottom out at a single number. Let's take a coin toss as an example. It does not make sense to say I think that the coin has a 0.05 chance of flipping heads with a 0.9 credible interval of (0.01, 0.1). It does not make sense even to have a probability distribution on coming up heads (strictly speaking). Coming up heads is an event and events do not distributions associated with them, but rather single probabilities. On the other hand, we can model the outcomes of a coin with a Bernoulli distribution with parameter p where the probability of heads is p and that of tails is 1 - p. Notice here that p is a parameter that just happens to coincide with our probability. It then makes sense to talk of p as a random variable and having a distribution, credible interval, etc. Note, however, that our estimation of the event of the coin coming up heads is still a single number at the end of the day. Our intermediate calculations might just involve integrating over the space of intermediate values of p, which means that p no longer coincides with the probability of heads (indeed it doesn't really make sense because p is no longer a single number). But how do we express the intuition of "confidence in my probability estimate?" After all it seems like there's a difference between me saying this coin has a 50% chance of heads when I know it's fair and me saying a 50% chance of heads when you have me an unfamiliar coin I've never inspected (even though reasonable statistical models for either case might both say it's 50%). In Bayesian statistics this is encoded entirely by the function that updates your posterior probability. If you are very "confident" your posterior probability will remain similar to your prior probability no matter what evidence you see (I know for sure this coin is fair so even if it comes up tails 1000 times I still think the next time will be 50% chance of heads). If you are not very confident your posterior probability might change more dramatically (hmm... I saw a string of tails, I'm pretty sure the next coin flip probably won't be heads again and I'll modify my probability accordingly).
nafizh 8y ago
I have to disagree with the top reply. You definitely can interpret low probability as I don't know. Check out this blog post- http://henripal.github.io/blog/langevin http://henripal.github.io/blog/langevin And these papers- 1. https://arxiv.org/abs/1806.02855 https://arxiv.org/abs/1806.02855 2. http://arxiv.org/abs/1811.11145 http://arxiv.org/abs/1811.11145

warent 8y ago

This question was answered in the comments section of the article. Basically Softmax will only give you values close to 1 or close to 0, so there's no room for your own interpretation or threshold. https://medium.com/@paraschopra/softmax-amplifies-small-differences-so-probabilities-that-come-out-of-it-get-skewed-towards-either-cf1c8c50fab9 https://medium.com/@paraschopra/softmax-amplifies-small-diff...

anonymousDan 8y ago
Do you need to do softmax? Why not just look at the underlying amplitudes of the output before applying softmax? Or is it that they won't be normalized?
- paraschopra 8y ago
  Interesting suggestion, honestly I'm not sure. Although this should be easy to test empirically. I feel if this were to work, it may depend on architecture of the network and what intermediate functions are used (ReLu, sigmoid and tanh). But your suggestion is easy to test empirically and I'll try it. There was another method proposed recently for approximating uncertainties: using dropout during evaluation (which effectively makes different runs on the same image as samples). But I came across a discussion on the efficacy of the technique and I wasn't sure. https://www.reddit.com/r/MachineLearning/comments/7bm4b2/d_what_is_the_current_state_of_dropout_as/ https://www.reddit.com/r/MachineLearning/comments/7bm4b2/d_w... My next project is to implement dropout and see how it compares with the current approach. I'll also try your suggestion.
- cultus 8y ago
  Yes, they wouldn't be normalized so you can't interpret it as a probability. Softmax is the function that normalizes it. However, even that you can't interpret as a probability of some outcome, because you do not take into account the prior. For that, you need actual Bayesian techniques.
  - anonymousDan 8y ago
    Can you elaborate on what you mean by not taking into account the prior? What is the output of aoftmax if not a probability?
    - cultus 8y ago
      Well, it's a probability of the output given the fixed, fitted model parameters. This is the likelihood. This is not the probability of getting that output, since we don't know what the parameters should really be. See the inverse probability fallacy [0]. This confusion is literally the main reason behind the replication crisis of scientific research. In ML, you typically have a model f(w;u), and you are trying to fit the parameters w to some data, given some hyperparameters u that represent some kind of prior information. In frequentist techniques, these could be regularization parameters. In frequentist techniques, the hyperparameters the fitted parameters are assumed fixed, and not random. In contrast, Bayesian methods assume it is random as well. This extra uncertainty is neglected by the predictions of frequentist techniques. Nonetheless, the uncertainty is real. [0] https://en.wikipedia.org/wiki/Confusion_of_the_inverse https://en.wikipedia.org/wiki/Confusion_of_the_inverse
      - anonymousDan 8y ago
        Thanks!
jeremysalwen 8y ago
I think that post is quite misleading. 90 and 100 are extreme values representing extreme confidence. Your model could just as easily learn [0.09, 0.10, 0.05] on the final layer, and the probabilities would then be [0.337, 0.340, 0.323]. And look, now softmax has the opposite problem! The final activation layers differed by a factor of 2, but softmax output almost identical probabilities! Except this should not be surprising, because softmax is not scale invariant. Values on the unit scale is what you would normally expect, so if you feed it values 100x larger, it's going to be way more confident. If you feed it values 100x smaller, it's going to be way less confident.
- paraschopra 8y ago
  This is interesting. In theory, what you are suggesting can be true but in practice you do think the last layer can output 0.1, 0.09, etc especially when last layer is typically not pushed through sigmoid.
  - rjtavares 8y ago
    Isn't that mostly true with highly curated datasets with clearly defined categories?

conjectures 8y ago

No. Probabilities can be random variables themselves. Say p is the probability an image is of a cat. p could be low, e.g. 0.05. However, perhaps p has the 0.9 credible interval (0.01, 0.1). The width of that interval expresses how confident you are of the point estimate. The correct translation of "I don't know" is a wide credible interval.

dwohnitmok 8y ago
I disagree with this characterization, especially in a Bayesian context (which credible interval comes from). Parameters into a model can be modeled as random variables. Probabilities themselves cannot be, unless they are themselves parameters (or happen to coincide with a parameter) into another part of a statistical model. A statistical estimation of an event (in the sense of probability theory) must bottom out at a single number. Let's take a coin toss as an example. It does not make sense to say I think that the coin has a 0.05 chance of flipping heads with a 0.9 credible interval of (0.01, 0.1). It does not make sense even to have a probability distribution on coming up heads (strictly speaking). Coming up heads is an event and events do not distributions associated with them, but rather single probabilities. On the other hand, we can model the outcomes of a coin with a Bernoulli distribution with parameter p where the probability of heads is p and that of tails is 1 - p. Notice here that p is a parameter that just happens to coincide with our probability. It then makes sense to talk of p as a random variable and having a distribution, credible interval, etc. Note, however, that our estimation of the event of the coin coming up heads is still a single number at the end of the day. Our intermediate calculations might just involve integrating over the space of intermediate values of p, which means that p no longer coincides with the probability of heads (indeed it doesn't really make sense because p is no longer a single number). But how do we express the intuition of "confidence in my probability estimate?" After all it seems like there's a difference between me saying this coin has a 50% chance of heads when I know it's fair and me saying a 50% chance of heads when you have me an unfamiliar coin I've never inspected (even though reasonable statistical models for either case might both say it's 50%). In Bayesian statistics this is encoded entirely by the function that updates your posterior probability. If you are very "confident" your posterior probability will remain similar to your prior probability no matter what evidence you see (I know for sure this coin is fair so even if it comes up tails 1000 times I still think the next time will be 50% chance of heads). If you are not very confident your posterior probability might change more dramatically (hmm... I saw a string of tails, I'm pretty sure the next coin flip probably won't be heads again and I'll modify my probability accordingly).

nafizh 8y ago

I have to disagree with the top reply. You definitely can interpret low probability as I don't know. Check out this blog post- http://henripal.github.io/blog/langevin http://henripal.github.io/blog/langevin And these papers- 1. https://arxiv.org/abs/1806.02855 https://arxiv.org/abs/1806.02855 2. http://arxiv.org/abs/1811.11145 http://arxiv.org/abs/1811.11145