3 ms·

It's a details, but I keep seeing it: > Yet, [the scientists] struggle to move away from Python, because of network effects, and because Python’s beginner-frie

by cauch 4y ago

It's a details, but I keep seeing it: > Yet, [the scientists] struggle to move away from Python, because of network effects, and because Python’s beginner-friendliness is appealing to scientists for whom programming is not a first language. I don't believe it's the whole story. In my case, during my 13 years in academia, I saw my field going away from C++ and towards python. Not because of network effects (it was the opposite: it was more difficult to not use what everybody was using), or because scientists were not able to program (the entry language of the whole field was C++, and python arrived only because scientists with a deep knowledge of C++ started to themselves switch the core library to be usable with python). I think something that computer scientists forgot when they consider the subject is that the way computer scientists do software is just not working when you do science. In science, you use coding as an exploratory tool. You are lucky if 10% of your code ends up being used in your final publication. Because the 90% was only there to understand and to progress towards the proper direction. For this reason, things like declaring variables, which is very important when one makes a professional software, are too costly to be useful when you need to write down a piece of code that you will ever run once to check a small hypothesis, especially when you have another language not requiring it. Another aspect is that you will present your scientific results to your colleagues, not your code (they are not interested in that), and they will come up with questions or good ideas, all very good for science, but rarely compatible with the way your algorithm was built in the first place, and you will need to shoe-horn it into your code (to test it) without taking 3 weeks. In this case, python flexibility and hackability is very useful. It's also visible in the popularity of things like Jupyter notebooks (I have to acknowledge it even if I personally don't like working with such tools), which reuse a working approach similar to what was done in mathematica and matlab, that were created with the scientific workflow in mind. I'm sure python simplicity has played a role. But I have the feeling that some people are totally oblivious on the fact that there may be other reasons.

whatever1 4y ago
Computer scientists also assume that you know what inputs your program needs and what is the range of the outputs. That is out of touch with scientific research. We may change overnight completely the inputs, the core logic and the outputs. Having to babysit function signatures, manage memory and types throughout these activities is just draining.
- duped 4y ago
  That is also how "computer scientists" and software engineers work. Our time is just valued a lot higher so we've come up with techniques to make our work more efficient and faster, like structuring our code well using types and function signatures. The added bonus is you get science that's you know, repeatable. Because the difference between industrial code and prototype code is that it gets run so often there can't be glaring mistakes; it must be repeatable by default. We have different techniques for dealing with these problems, but writing good code is orthogonal to that (I don't think scientists need to be running static analysis, doing deep reviews, and having extensive integration/unit/mock testing throughout their code).
  - mumblemumble 4y ago
    I used to be a software engineer, and now I'm a data scientist. Not exactly the same thing as a real scientist, but I suspect that we have common cause in this area. One of the hard lessons in the transition was realizing that things that allowed me to work more efficiently when I was a software engineer instead reduce my efficiency in my new career. You might get a decent analogy of the difference by comparing photographs of the first transistor with pictures of every subsequent transistor. The first transistor's clearly going to be terrible in any production application. But the same characteristics that make it so terrible for practical use were also, to varying degrees, essential to or characteristic of the exploratory process that led to its creation. It's similar for my R&D code. In order to do my R&D work more efficiently and effectively, I need to do things that would be unholy in production code. This is why there's a separate and essential productionizing step where my output is heavily revamped and possibly even completely rewritten in a different programming language. re: repeatability, I've discovered that it, too, means something slightly different in a science context than it does in an engineering context.
  - wenus 4y ago
    > Our time is just valued a lot higher What the hell is this?
analog31 4y ago
All good points. I think the network effects are more important than they were in the past, because the network has gotten bigger. Today, new graduates make sure to put Python on their resumes, even if they've barely touched it. A colleague who left for another job thanked me for encouraging him to learn Python. Contrast that with 13 years ago, roughly when I switched to Python. My colleagues thought it was some weird toy, and encouraged me to learn C# instead. The past 13 years have also seen most programmers get over their aversion to open-source tools. It's true, as mentioned in another comment, that repeatability is important. But at least in experimental science, the repeatability of an experiment is a bigger hurdle than making the same code run twice. I use Python code to control my experiments, and the ability to read my way through a complex workflow is quite valuable. One thing I like is being able to bodge together a huge blob of data and metadata into a dict that's easy to store and unpack. That encourages me to keep more experimental data and metadata, and use it later. Python and its libraries have sprawled to the point where it's anything but simple.
roflyear 4y ago
It's nice to be able to collect your data and then do analysis on it in one language. Data work is mostly about getting your data in a place where you can work with it. Rarely are you handed a dataset that is shiny and ready for analysis or regression ... Python also has the benefit of tooling, where you can use tools you're familiar with and still work on most codebases.
Helmut10001 4y ago
This is spot on, thank you for this summary! I just finished working on an exploratory data science paper that was in the making for the last 5 years. In the beginning, I set the goal to version the whole process in git. It was incredibly difficult to not accumulate an immense mountain of tech debt during this whole time. We had 4 internal review rounds (5 researchers from two universities), then 2 immediate rejections ("not fitting the scope of journal"), which I considered Major Revisions (because you need to significantly adjust the paper to each new journal) and at the final journal now 1 major revision, maybe another one (but I am not sure yet). All in all, there were about 7 Major Revisions, each taking about 3 to 6 months of 8 hour shifts. The paper now consists of 10 (chained) Jupyter Notebooks and about 16000 lines of code with 1200 git revisions (the notebooks are versioned in Markdown using Jupytext). I cannot account for the fundamental changes that were necessary due to all the external input - this is really very difficult to plan for in the beginning, when you set out for such a project. I hear my IT colleague arguing "you need tests for all of this!" and then I face another day where I need to write 25 new functions in 6 hours, while preparing lectures, workshops, assess master theses etc. - impossible to me, without the flexibility of python. It is at least pretty different to how things work in production at companies, where you have at least some clear goals/rules.
- periheli0n 4y ago
  > The paper now consists of 10 Jupyter Notebooks and about 16000 lines of code with 1200 git revisions I‘ve been there too! And about 1 year after the paper was finally published, fresh starters in my lab couldn’t run the code anymore because of an obscure Pandas error caused by a version change. Then Python 2.7 was shelved and created more problems. In conclusion the considerable effort I put in to create Jupyter notebooks recreating every single figure from the paper was not worth it.
  - Helmut10001 4y ago
    I created Docker Container Images for every versioned "release" of the paper that we maintain in our Gitlab Registry (a CI builds those images automatically from `.gitlab-ci.yml`, `docker-compose.yml` using `docker:dind`), so you can pull a specific Docker Image for every version of the paper that will definitely have the correct dependencies (because the Jupyter Notebooks were tested with each specific version), including Jupyter itself.
bombolo 4y ago
> Another aspect is that you will present your scientific results to your colleagues, not your code (they are not interested in that), Ah, yes you do not present your code and that's a flaw of science. Because nobody can really reproduce your results without, but it seems modern science gave up on the whole reproducibility thing. > they are not interested in that Ah no… they are very much interested in that, but you don't give them out because you want to keep adding little details in the form of new papers, rather than letting other groups compete on the same research. So keeping the code proprietary makes it difficult for other groups to do better than you. It's all very unscientific and uncooperative really. Let's not pretend it's "science"…
- cauch 4y ago
  > but it seems modern science gave up on the whole reproducibility thing. It looks like you don't understand the definition and the point of reproducibility. If you just click on a button to recompute the results from the same code, this is not "reproducibility". The point of reproducibility is to have other scientists able to confirm your results in a fully independent way. They should be able to do it from scratch, without your code, and reach the same conclusion. Using the code is therefore against reproducibility: people who use the code are not reproducing the experiment. It is not restricted to the code. At CERN, the Atlas experiment and the CMS experiment are designed to be able to observe the same phenomenons, in order to bring independent measurements. Because of the notion of independence and reproducibility, while working in the same building, there are strong instructions to not share any non-published details with the members of the other experiment. The reason is that if there is an error in the reasoning, or if we observe a statistical fluctuations, or if there is a bug in the code, when the other experiment comes up with their results, their results will be unbiased. Additionally, it is good practice to have several team doing their analysis in parallel on the same subject, each using their own code. This is called "cross-check analyses" and is a very important scientific procedure that is able to discover code bugs way more efficiently than unit test or code review. > but you don't give them out Why do you say we don't give them out? The code is available to reviewers. It's on the collaboration git repository and anyone in the collaboration can check it out. They are not interested in that the same way they are not interested in the schematic of the Intel processor that was used in the laptop of the person who run the algorithm. They are, however, interested in understanding deeply the reasoning and the logic that led to the conclusion, to the point that, if they want, they can rewrite their own code (in any language they want) and reach the same result. They don't see the code (not because they can't, but because they are smart enough to understand what reproducibility means), but they know what the code is doing. > It's all very unscientific and uncooperative really. Let's not pretend it's "science"… You clearly have no idea what you are talking about. Therefore, your conclusions are worthless.

by cauch 4y ago

whatever1 4y ago

Computer scientists also assume that you know what inputs your program needs and what is the range of the outputs. That is out of touch with scientific research. We may change overnight completely the inputs, the core logic and the outputs. Having to babysit function signatures, manage memory and types throughout these activities is just draining.

duped 4y ago
That is also how "computer scientists" and software engineers work. Our time is just valued a lot higher so we've come up with techniques to make our work more efficient and faster, like structuring our code well using types and function signatures. The added bonus is you get science that's you know, repeatable. Because the difference between industrial code and prototype code is that it gets run so often there can't be glaring mistakes; it must be repeatable by default. We have different techniques for dealing with these problems, but writing good code is orthogonal to that (I don't think scientists need to be running static analysis, doing deep reviews, and having extensive integration/unit/mock testing throughout their code).
- mumblemumble 4y ago
  I used to be a software engineer, and now I'm a data scientist. Not exactly the same thing as a real scientist, but I suspect that we have common cause in this area. One of the hard lessons in the transition was realizing that things that allowed me to work more efficiently when I was a software engineer instead reduce my efficiency in my new career. You might get a decent analogy of the difference by comparing photographs of the first transistor with pictures of every subsequent transistor. The first transistor's clearly going to be terrible in any production application. But the same characteristics that make it so terrible for practical use were also, to varying degrees, essential to or characteristic of the exploratory process that led to its creation. It's similar for my R&D code. In order to do my R&D work more efficiently and effectively, I need to do things that would be unholy in production code. This is why there's a separate and essential productionizing step where my output is heavily revamped and possibly even completely rewritten in a different programming language. re: repeatability, I've discovered that it, too, means something slightly different in a science context than it does in an engineering context.
- wenus 4y ago
  > Our time is just valued a lot higher What the hell is this?

analog31 4y ago

All good points. I think the network effects are more important than they were in the past, because the network has gotten bigger. Today, new graduates make sure to put Python on their resumes, even if they've barely touched it. A colleague who left for another job thanked me for encouraging him to learn Python. Contrast that with 13 years ago, roughly when I switched to Python. My colleagues thought it was some weird toy, and encouraged me to learn C# instead. The past 13 years have also seen most programmers get over their aversion to open-source tools. It's true, as mentioned in another comment, that repeatability is important. But at least in experimental science, the repeatability of an experiment is a bigger hurdle than making the same code run twice. I use Python code to control my experiments, and the ability to read my way through a complex workflow is quite valuable. One thing I like is being able to bodge together a huge blob of data and metadata into a dict that's easy to store and unpack. That encourages me to keep more experimental data and metadata, and use it later. Python and its libraries have sprawled to the point where it's anything but simple.

roflyear 4y ago

It's nice to be able to collect your data and then do analysis on it in one language. Data work is mostly about getting your data in a place where you can work with it. Rarely are you handed a dataset that is shiny and ready for analysis or regression ... Python also has the benefit of tooling, where you can use tools you're familiar with and still work on most codebases.

Helmut10001 4y ago

This is spot on, thank you for this summary! I just finished working on an exploratory data science paper that was in the making for the last 5 years. In the beginning, I set the goal to version the whole process in git. It was incredibly difficult to not accumulate an immense mountain of tech debt during this whole time. We had 4 internal review rounds (5 researchers from two universities), then 2 immediate rejections ("not fitting the scope of journal"), which I considered Major Revisions (because you need to significantly adjust the paper to each new journal) and at the final journal now 1 major revision, maybe another one (but I am not sure yet). All in all, there were about 7 Major Revisions, each taking about 3 to 6 months of 8 hour shifts. The paper now consists of 10 (chained) Jupyter Notebooks and about 16000 lines of code with 1200 git revisions (the notebooks are versioned in Markdown using Jupytext). I cannot account for the fundamental changes that were necessary due to all the external input - this is really very difficult to plan for in the beginning, when you set out for such a project. I hear my IT colleague arguing "you need tests for all of this!" and then I face another day where I need to write 25 new functions in 6 hours, while preparing lectures, workshops, assess master theses etc. - impossible to me, without the flexibility of python. It is at least pretty different to how things work in production at companies, where you have at least some clear goals/rules.

periheli0n 4y ago
> The paper now consists of 10 Jupyter Notebooks and about 16000 lines of code with 1200 git revisions I‘ve been there too! And about 1 year after the paper was finally published, fresh starters in my lab couldn’t run the code anymore because of an obscure Pandas error caused by a version change. Then Python 2.7 was shelved and created more problems. In conclusion the considerable effort I put in to create Jupyter notebooks recreating every single figure from the paper was not worth it.
- Helmut10001 4y ago
  I created Docker Container Images for every versioned "release" of the paper that we maintain in our Gitlab Registry (a CI builds those images automatically from `.gitlab-ci.yml`, `docker-compose.yml` using `docker:dind`), so you can pull a specific Docker Image for every version of the paper that will definitely have the correct dependencies (because the Jupyter Notebooks were tested with each specific version), including Jupyter itself.

bombolo 4y ago

> Another aspect is that you will present your scientific results to your colleagues, not your code (they are not interested in that), Ah, yes you do not present your code and that's a flaw of science. Because nobody can really reproduce your results without, but it seems modern science gave up on the whole reproducibility thing. > they are not interested in that Ah no… they are very much interested in that, but you don't give them out because you want to keep adding little details in the form of new papers, rather than letting other groups compete on the same research. So keeping the code proprietary makes it difficult for other groups to do better than you. It's all very unscientific and uncooperative really. Let's not pretend it's "science"…

cauch 4y ago
> but it seems modern science gave up on the whole reproducibility thing. It looks like you don't understand the definition and the point of reproducibility. If you just click on a button to recompute the results from the same code, this is not "reproducibility". The point of reproducibility is to have other scientists able to confirm your results in a fully independent way. They should be able to do it from scratch, without your code, and reach the same conclusion. Using the code is therefore against reproducibility: people who use the code are not reproducing the experiment. It is not restricted to the code. At CERN, the Atlas experiment and the CMS experiment are designed to be able to observe the same phenomenons, in order to bring independent measurements. Because of the notion of independence and reproducibility, while working in the same building, there are strong instructions to not share any non-published details with the members of the other experiment. The reason is that if there is an error in the reasoning, or if we observe a statistical fluctuations, or if there is a bug in the code, when the other experiment comes up with their results, their results will be unbiased. Additionally, it is good practice to have several team doing their analysis in parallel on the same subject, each using their own code. This is called "cross-check analyses" and is a very important scientific procedure that is able to discover code bugs way more efficiently than unit test or code review. > but you don't give them out Why do you say we don't give them out? The code is available to reviewers. It's on the collaboration git repository and anyone in the collaboration can check it out. They are not interested in that the same way they are not interested in the schematic of the Intel processor that was used in the laptop of the person who run the algorithm. They are, however, interested in understanding deeply the reasoning and the logic that led to the conclusion, to the point that, if they want, they can rewrite their own code (in any language they want) and reach the same result. They don't see the code (not because they can't, but because they are smart enough to understand what reproducibility means), but they know what the code is doing. > It's all very unscientific and uncooperative really. Let's not pretend it's "science"… You clearly have no idea what you are talking about. Therefore, your conclusions are worthless.