3 ms·

> This latter property is particularly useful for enhancing the precision of information processing. For example, information represented by an individual neuro

> This latter property is particularly useful for enhancing the precision of information processing. For example, information represented by an individual neuron may be noisy (say, with a precision of 1 in 100). By taking the average of input from 100 neurons carrying the same information, the common downstream partner neuron can represent the information with much higher precision (about 1 in 1,000 in this case). What does precision mean here, mathematically? Precision as in precision/recall doesn't seem to work, as precision in that context means TP / (TP + FP), and going from 1/100 to 1/1000 would mean worse precision. Does it mean something like floating point precision, where increasing bits can increase information represented, i.e. 2 bits represent max of 11=3, 3 bits represent maximum of 111=7 and so on? I can't quite this to work...

eli_gottlieb 4y ago
They mean signal-to-noise ratio. Let's say you want to estimate some quantity, so you put a Gaussian prior over it. You then have a bunch of different sensors subject to white (Gaussian) noise, and some link-function connecting the latent value to the mean of the sensor reading with fixed precision. Gaussian with Gaussian is a conjugate prior-likelihood pair, so you can calculate the posterior parameters in closed form. Your posterior mean is going to be a precision-weighted average of your sample mean and your prior mean, while your posterior precision (signal-to-noise) is going to be proportional to the number of sensors you used.
- saeranv 4y ago
  Thank you, that's a very clever explaination. Even though I can't grok the mathematics off the top of my head (I would have to look up the conjugate gaussian-gaussian formula and work this out) I think converting this to a posterior probability calculation is much more intuitive, then the article's explaination. So, my crude understanding is that therefore the signal and noise is the mean and variance of the Gaussian, respectively. The posterior uncertainty in this is case is thus a function of a weighted average of the sample data, weighted by the priors, normalized by the data, which results in a higher signal-to-noise ratio, or lower uncertainty.
  - eli_gottlieb 4y ago
    >So, my crude understanding is that therefore the signal and noise is the mean and variance of the Gaussian, respectively. The posterior uncertainty in this is case is thus a function of a weighted average of the sample data, weighted by the priors, normalized by the data, which results in a higher signal-to-noise ratio, or lower uncertainty. The variance of the Gaussian is proportional to a noise-to-signal ratio, so you have to turn it upside-down to get the signal-to-noise we usually work with. But yeah. https://en.wikipedia.org/wiki/Signal-to-noise_ratio https://en.wikipedia.org/wiki/Signal-to-noise_ratio

eli_gottlieb 4y ago

They mean signal-to-noise ratio. Let's say you want to estimate some quantity, so you put a Gaussian prior over it. You then have a bunch of different sensors subject to white (Gaussian) noise, and some link-function connecting the latent value to the mean of the sensor reading with fixed precision. Gaussian with Gaussian is a conjugate prior-likelihood pair, so you can calculate the posterior parameters in closed form. Your posterior mean is going to be a precision-weighted average of your sample mean and your prior mean, while your posterior precision (signal-to-noise) is going to be proportional to the number of sensors you used.

saeranv 4y ago
Thank you, that's a very clever explaination. Even though I can't grok the mathematics off the top of my head (I would have to look up the conjugate gaussian-gaussian formula and work this out) I think converting this to a posterior probability calculation is much more intuitive, then the article's explaination. So, my crude understanding is that therefore the signal and noise is the mean and variance of the Gaussian, respectively. The posterior uncertainty in this is case is thus a function of a weighted average of the sample data, weighted by the priors, normalized by the data, which results in a higher signal-to-noise ratio, or lower uncertainty.
- eli_gottlieb 4y ago
  >So, my crude understanding is that therefore the signal and noise is the mean and variance of the Gaussian, respectively. The posterior uncertainty in this is case is thus a function of a weighted average of the sample data, weighted by the priors, normalized by the data, which results in a higher signal-to-noise ratio, or lower uncertainty. The variance of the Gaussian is proportional to a noise-to-signal ratio, so you have to turn it upside-down to get the signal-to-noise we usually work with. But yeah. https://en.wikipedia.org/wiki/Signal-to-noise_ratio https://en.wikipedia.org/wiki/Signal-to-noise_ratio