4 ms·

Out of curiosity: how deep does that need go? Low level hardware analysis? Chip manufacturing facility audits? How would you even approach this?

by blhack 13y ago

noonespecial 13y ago
Recalling the The Pentium FDIV bug, I'd say yes. Chip level.
lutusp 13y ago
> Out of curiosity: how deep does that need go? Deep enough that the meaning of the result cannot reasonably be called into question, just like in science. > How would you even approach this? As scientists. Someone once described science as ... not so much knowing as knowing that we know. Same idea. Many results that seemed certain when published, evaporated once the mathematics were examined closely. This happens even when the entire process -- data, equations, conclusions -- are all included in the publication. Given that history, imagine how much more cautious people want to be when part of the process of arriving at a critical conclusion is concealed perpetually.
- mkingston 13y ago
  "Deep enough that the meaning of the result cannot reasonably be called into question, just like in science." I think what is really being asked is: "What level of confidence is required?". I would assume you trust the hardware and only scrutinise the software? In contrast: are results tested on a plurality of machines with intentionally, measurably different hardware and the absence of discrepancies implies that the hardware is not a factor with regard to the problem being solved? Presumedly you'd need some sort of standardised test suite to pass on your hardware to really confidently declare the hardware to not be a factor. I doubt this is the case, but like the previous poster, I'm quite curious as to the specific level of scrutiny applied here.
  - lutusp 13y ago
    > I think what is really being asked is: "What level of confidence is required?". That's something that varies from field to field. In physics, a new discovery requires a five-sigma result, meaning a result that has a one in 3.5 million probability of arising by chance rather than the legitimate detection of a new particle or phenomenon. http://www.select-statistics.co.uk/blog/webmaster/one-sigma-or-five http://www.select-statistics.co.uk/blog/webmaster/one-sigma-... In psychology, by contrast, a p-value of 0.05 (i.e. 2 sigma) is accepted as statistically significant, which by itself may explain the sad state of the field. http://en.wikipedia.org/wiki/P-value http://en.wikipedia.org/wiki/P-value > I would assume you trust the hardware and only scrutinise the software? No, a scientist doesn't trust either hardware or software, but hardware can be checked and validated. In the case of Mathematica, the software cannot be checked and validated, it must be trusted. And for a scientist, trust cannot be distinguished from faith.
    - mkingston 13y ago
      Thanks for your reply. "No, a scientist doesn't trust either hardware or software, but hardware can be checked and validated." Do you know if or how this is done?

Out of curiosity: how deep does that need go? Low level hardware analysis? Chip manufacturing facility audits? How would you even approach this?

by blhack 13y ago

Out of curiosity: how deep does that need go? Low level hardware analysis? Chip manufacturing facility audits? How would you even approach this?

noonespecial 13y ago

Recalling the The Pentium FDIV bug, I'd say yes. Chip level.

lutusp 13y ago

> Out of curiosity: how deep does that need go? Deep enough that the meaning of the result cannot reasonably be called into question, just like in science. > How would you even approach this? As scientists. Someone once described science as ... not so much knowing as knowing that we know. Same idea. Many results that seemed certain when published, evaporated once the mathematics were examined closely. This happens even when the entire process -- data, equations, conclusions -- are all included in the publication. Given that history, imagine how much more cautious people want to be when part of the process of arriving at a critical conclusion is concealed perpetually.

mkingston 13y ago
"Deep enough that the meaning of the result cannot reasonably be called into question, just like in science." I think what is really being asked is: "What level of confidence is required?". I would assume you trust the hardware and only scrutinise the software? In contrast: are results tested on a plurality of machines with intentionally, measurably different hardware and the absence of discrepancies implies that the hardware is not a factor with regard to the problem being solved? Presumedly you'd need some sort of standardised test suite to pass on your hardware to really confidently declare the hardware to not be a factor. I doubt this is the case, but like the previous poster, I'm quite curious as to the specific level of scrutiny applied here.
- lutusp 13y ago
  > I think what is really being asked is: "What level of confidence is required?". That's something that varies from field to field. In physics, a new discovery requires a five-sigma result, meaning a result that has a one in 3.5 million probability of arising by chance rather than the legitimate detection of a new particle or phenomenon. http://www.select-statistics.co.uk/blog/webmaster/one-sigma-or-five http://www.select-statistics.co.uk/blog/webmaster/one-sigma-... In psychology, by contrast, a p-value of 0.05 (i.e. 2 sigma) is accepted as statistically significant, which by itself may explain the sad state of the field. http://en.wikipedia.org/wiki/P-value http://en.wikipedia.org/wiki/P-value > I would assume you trust the hardware and only scrutinise the software? No, a scientist doesn't trust either hardware or software, but hardware can be checked and validated. In the case of Mathematica, the software cannot be checked and validated, it must be trusted. And for a scientist, trust cannot be distinguished from faith.
  - mkingston 13y ago
    Thanks for your reply. "No, a scientist doesn't trust either hardware or software, but hardware can be checked and validated." Do you know if or how this is done?