5 ms·

While I totally hear you regarding the pain of python modules (particularly on Windows), the point of python 'distributions' like anaconda and canopy is to brin

While I totally hear you regarding the pain of python modules (particularly on Windows), the point of python 'distributions' like anaconda and canopy is to bring the kitchen sink along, kind of like mathematica. The problem with Mathematica from a science point of view is that, being closed source, means you can't independently ensure the calculations are happening correctly. To be replicable, science involving data needs to use open source tools.

chillydawg 9y ago
I'm not sure I agree with that. If you publish your methods and make your data available, then a replicator has all they need. They'd have to reimplement the data pipeline anyway, otherwise it's not a replicated result it's just someone running your notebook with potentially the same bugs in it again.
starpilot 9y ago
> The problem with Mathematica from a science point of view is that, being closed source, means you can't independently ensure the calculations are happening correctly. To be replicable, science involving data needs to use open source tools. Excel and MATLAB can't be used for real science?
- inigoalonso 9y ago
  At least you need to be very careful with them: https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1044-7 https://genomebiology.biomedcentral.com/articles/10.1186/s13... Abstract: "The spreadsheet software Microsoft Excel, when used with default settings, is known to convert gene names to dates and floating-point numbers. A programmatic scan of leading genomics journals reveals that approximately one-fifth of papers with supplementary Excel gene lists contain erroneous gene name conversions."
  - starpilot 9y ago
    Yeah, but that doesn't change the fact that 99% of the science/engineering/business world is reliant on Excel. Airplanes are designed in Excel.
    - PopePompous 9y ago
      I think your estimate of 99% is a wee bit high. At least in the field I'm familiar with, astronomy, the idea of using Excel for any serious computation or design would be met with laughter.
- jjgreen 9y ago
  Matlab's fine, but Excel has (had?) some serious defects in the basic statistical functions, in particular, it coundn't reliably calculate a standard deviation.
- iguy 9y ago
  Well since the author is an economist, questions about "real science" can be interpreted a few ways. For a fun look into the high standards of the field, a few years ago Piketty made the mistake of sharing his Excel files, in which all sorts of crucial adjustments were hard-coded into tables of data... https://marginalrevolution.com/marginalrevolution/2014/05/piketty-update.html https://marginalrevolution.com/marginalrevolution/2014/05/pi...
gaius 9y ago
The problem with Mathematica from a science point of view is that, being closed source, means you can't independently ensure the calculations are happening correctly Have there been any high profile failures root-caused to Mathematica (or MATLAB or any similar product) getting its sums wrong? I can find any news stories etc. Plenty of serious calculations were and are done on “closed source” HP and TI calculators too. Every serious scientific instrument with its own data capture uses a binary blob too somewhere, so if that’s a problem for you then you can’t even trust the raw data! And even if you have all of the code - you still need to worry if the proprietary, closed FPU is working “correctly”. This sounds very much like a post-hoc justification for “its free as in beer”. Do you think Wikipedia is more trustworthy than real references too? What about blogs?
- BlackFingolfin 9y ago
  I am not sure that there are/were "high profile failures", but serious bugs for sure, not just in Mathmetica but also in the calculators you mention; ser eg here: https://mathoverflow.net/questions/11517/computer-algebra-errors https://mathoverflow.net/questions/11517/computer-algebra-er... Now being open source does not prevent such errors. What I think is far more important here is open issue tracking: companies like Wolfram may not be exactly eager to let you know that there are serious bugs in their products (and what they are) unless they must. Being able to fix the bug yourself can of course be a real perk for some user's, but realistically, it's out if question for most.
- fdej 9y ago
  Here is an article from a couple of years ago about how one group of mathematicians were misled by (and had to spend some time tracking down) a bug in Mathematica determinant evaluation: http://www.ams.org/notices/201410/rnoti-p1249.pdf http://www.ams.org/notices/201410/rnoti-p1249.pdf
  - FractalLP 9y ago
    Thanks for posting. I'm sure bugs like this show up in any and all software. I do wish they had a public bug tracker though.
FractalLP 9y ago
I hear what you're saying and you're right that WinPython & Anaconda certainly help, but the documentation is still a long way off from Mathematica in my opinion. One thing in Python's favor though might be depth in certain categories. The machine learning stuff in Mathematica is very nice and high level if you want neural networks, but if you need PSO or GA, you'll probably have to write your own or grab someone else's notebook. It was difficult for me to support closed-source software as I've always supported linux for this reason. As far as ensuring accuracy of calculations, having a very large and highly technical user base over several decades helps, but I'm not sure how much this is used in theory. If a statistician publishes a paper using R, is anyone really going to check the R module source code? I bet this is a rare occurrence.

chillydawg 9y ago

I'm not sure I agree with that. If you publish your methods and make your data available, then a replicator has all they need. They'd have to reimplement the data pipeline anyway, otherwise it's not a replicated result it's just someone running your notebook with potentially the same bugs in it again.

starpilot 9y ago

> The problem with Mathematica from a science point of view is that, being closed source, means you can't independently ensure the calculations are happening correctly. To be replicable, science involving data needs to use open source tools. Excel and MATLAB can't be used for real science?

inigoalonso 9y ago
At least you need to be very careful with them: https://genomebiology.biomedcentral.com/articles/10.1186/s13059-016-1044-7 https://genomebiology.biomedcentral.com/articles/10.1186/s13... Abstract: "The spreadsheet software Microsoft Excel, when used with default settings, is known to convert gene names to dates and floating-point numbers. A programmatic scan of leading genomics journals reveals that approximately one-fifth of papers with supplementary Excel gene lists contain erroneous gene name conversions."
- starpilot 9y ago
  Yeah, but that doesn't change the fact that 99% of the science/engineering/business world is reliant on Excel. Airplanes are designed in Excel.
  - PopePompous 9y ago
    I think your estimate of 99% is a wee bit high. At least in the field I'm familiar with, astronomy, the idea of using Excel for any serious computation or design would be met with laughter.
jjgreen 9y ago
Matlab's fine, but Excel has (had?) some serious defects in the basic statistical functions, in particular, it coundn't reliably calculate a standard deviation.
iguy 9y ago
Well since the author is an economist, questions about "real science" can be interpreted a few ways. For a fun look into the high standards of the field, a few years ago Piketty made the mistake of sharing his Excel files, in which all sorts of crucial adjustments were hard-coded into tables of data... https://marginalrevolution.com/marginalrevolution/2014/05/piketty-update.html https://marginalrevolution.com/marginalrevolution/2014/05/pi...

gaius 9y ago

The problem with Mathematica from a science point of view is that, being closed source, means you can't independently ensure the calculations are happening correctly Have there been any high profile failures root-caused to Mathematica (or MATLAB or any similar product) getting its sums wrong? I can find any news stories etc. Plenty of serious calculations were and are done on “closed source” HP and TI calculators too. Every serious scientific instrument with its own data capture uses a binary blob too somewhere, so if that’s a problem for you then you can’t even trust the raw data! And even if you have all of the code - you still need to worry if the proprietary, closed FPU is working “correctly”. This sounds very much like a post-hoc justification for “its free as in beer”. Do you think Wikipedia is more trustworthy than real references too? What about blogs?

BlackFingolfin 9y ago
I am not sure that there are/were "high profile failures", but serious bugs for sure, not just in Mathmetica but also in the calculators you mention; ser eg here: https://mathoverflow.net/questions/11517/computer-algebra-errors https://mathoverflow.net/questions/11517/computer-algebra-er... Now being open source does not prevent such errors. What I think is far more important here is open issue tracking: companies like Wolfram may not be exactly eager to let you know that there are serious bugs in their products (and what they are) unless they must. Being able to fix the bug yourself can of course be a real perk for some user's, but realistically, it's out if question for most.
fdej 9y ago
Here is an article from a couple of years ago about how one group of mathematicians were misled by (and had to spend some time tracking down) a bug in Mathematica determinant evaluation: http://www.ams.org/notices/201410/rnoti-p1249.pdf http://www.ams.org/notices/201410/rnoti-p1249.pdf
- FractalLP 9y ago
  Thanks for posting. I'm sure bugs like this show up in any and all software. I do wish they had a public bug tracker though.

FractalLP 9y ago

I hear what you're saying and you're right that WinPython & Anaconda certainly help, but the documentation is still a long way off from Mathematica in my opinion. One thing in Python's favor though might be depth in certain categories. The machine learning stuff in Mathematica is very nice and high level if you want neural networks, but if you need PSO or GA, you'll probably have to write your own or grab someone else's notebook. It was difficult for me to support closed-source software as I've always supported linux for this reason. As far as ensuring accuracy of calculations, having a very large and highly technical user base over several decades helps, but I'm not sure how much this is used in theory. If a statistician publishes a paper using R, is anyone really going to check the R module source code? I bet this is a rare occurrence.