4 ms·

This would be really nice, but the only solution I know here involves working with numbers that represent a function of p rather than p itself. And what you do

This would be really nice, but the only solution I know here involves working with numbers that represent a function of p rather than p itself. And what you do depends on what you're computing. In general, you'll find lots of useful math functions that have specialized routines around zero and one, like numpy's log1p function, logaddexp, expm1, and company. Log odds ratios are also good representations that have this kind of accuracy between zero and one. They're good when you need them. (e.g. 100:1 odds are log(100/1) and 1:100 odds are log(1/100))

runarberg 5y ago
I see. So basically you just know a bunch of log rules and then map your distribution into a log-distribution. And I guess this becomes second nature to a seasoned statistician. So a dedicated numeric type for probability value is nothing more than a nice-to-have.
- 6gvONxR4sf7o 5y ago
  Kinda, but to make it more explicit: with floating point numbers you don't even need the explicit log distribution because it's kinda baked in already, with the number split between an absolute and an exponential part. So floats capture an exponential scale of precision naturally, allowing you to write 1e-200 and 1 and 1e200 just as easily, with a common relative precision around each. You use the log scale as a mental model and then just keep floating point issues in mind while you work. The handy functions are more about making sure that cancellations don't happen by providing a few primitive operations that cover lots of the uses. Like the handy log1p which computes log(1+x). If you did this with the log function, you'll compute log(1+1e-200)=log(1)=0. If you use log1p, you get log1p(1e-200)=1e-200. You avoid the loss of info that comes with adding something close to zero to something that isn't close to zero, which is the real trick. And then if you need something close to one, you probable just use the converse: instead of using p=1-eps, you just work with eps itself.

runarberg 5y ago

I see. So basically you just know a bunch of log rules and then map your distribution into a log-distribution. And I guess this becomes second nature to a seasoned statistician. So a dedicated numeric type for probability value is nothing more than a nice-to-have.

6gvONxR4sf7o 5y ago
Kinda, but to make it more explicit: with floating point numbers you don't even need the explicit log distribution because it's kinda baked in already, with the number split between an absolute and an exponential part. So floats capture an exponential scale of precision naturally, allowing you to write 1e-200 and 1 and 1e200 just as easily, with a common relative precision around each. You use the log scale as a mental model and then just keep floating point issues in mind while you work. The handy functions are more about making sure that cancellations don't happen by providing a few primitive operations that cover lots of the uses. Like the handy log1p which computes log(1+x). If you did this with the log function, you'll compute log(1+1e-200)=log(1)=0. If you use log1p, you get log1p(1e-200)=1e-200. You avoid the loss of info that comes with adding something close to zero to something that isn't close to zero, which is the real trick. And then if you need something close to one, you probable just use the converse: instead of using p=1-eps, you just work with eps itself.