4 ms·

> You have an exhaustive test suite for your important numeric libraries, right? > You have a benchmarks suite to protect against performance degradation, righ

> You have an exhaustive test suite for your important numeric libraries, right? > You have a benchmarks suite to protect against performance degradation, right? In theory, in a perfect world, yes. In practice, people use open source libraries like Eigen because they are the best, not because of their extensive test coverage. Doing things which might break these libraries will fork the community.

still_grokking 4y ago
How a lib where you don't even know whether it computes correct results can be called "best"? Only because something spits out numbers really fast doesn't say anything about the quality of this something. Especially when considering that optimizing for speed comes most times with very hacky code! This, plus the fact that this code is usually written by laymen (scientist aren't software engineers!) makes such things very questionable. After looking into scientific computing and the usual software "quality" there I lost quite some trust in anything that comes out of there. This was actually very disillusionary and extremely sad to find out. This could even destroy the broader trust in science in general (as everything today in this field depends on statistics calculated by computers). This would be an catastrophic outcome! But most people would currently even deny the existence of an issue… Related (together with the parent): https://news.ycombinator.com/item?id=34224186 https://news.ycombinator.com/item?id=34224186 Also related as people (scientists!) got blinded by the promise of speed without considering correctness: https://yuri.is/not-julia/ https://yuri.is/not-julia/ A house of cards… :-(
- snovv_crash 4y ago
  This is normal for open source. You get what you pay for, and nobody is paying for test suites for potential future users to freeload off of. Why would numerical libraries be any different from OpenSSL pre-heartbleed? And how would you fix this issue without spending other people's time and money on things they don't want to spend it on? This is why subtle compiler rule changes are so potentially devastating.

still_grokking 4y ago

How a lib where you don't even know whether it computes correct results can be called "best"? Only because something spits out numbers really fast doesn't say anything about the quality of this something. Especially when considering that optimizing for speed comes most times with very hacky code! This, plus the fact that this code is usually written by laymen (scientist aren't software engineers!) makes such things very questionable. After looking into scientific computing and the usual software "quality" there I lost quite some trust in anything that comes out of there. This was actually very disillusionary and extremely sad to find out. This could even destroy the broader trust in science in general (as everything today in this field depends on statistics calculated by computers). This would be an catastrophic outcome! But most people would currently even deny the existence of an issue… Related (together with the parent): https://news.ycombinator.com/item?id=34224186 https://news.ycombinator.com/item?id=34224186 Also related as people (scientists!) got blinded by the promise of speed without considering correctness: https://yuri.is/not-julia/ https://yuri.is/not-julia/ A house of cards… :-(

snovv_crash 4y ago
This is normal for open source. You get what you pay for, and nobody is paying for test suites for potential future users to freeload off of. Why would numerical libraries be any different from OpenSSL pre-heartbleed? And how would you fix this issue without spending other people's time and money on things they don't want to spend it on? This is why subtle compiler rule changes are so potentially devastating.