5 ms·

Don't underestimate the impact this has on getting funding or even just tenure/etc recognition for working on numpy. I'm in industry these days, but coming fro

by jofer 6y ago

Don't underestimate the impact this has on getting funding or even just tenure/etc recognition for working on numpy. I'm in industry these days, but coming from the academic side, it's _really_ hard to get recognized for building the underlying infrastructure that tons of people use. I've built and maintained libraries that are used in a ton of publications, but was always told my work was "utterly and completely useless". It was also always unpublishable, as methods are never publishable in my field. Numpy has (obviously) vastly more respect and impact than my work, but the general problem remains. Articles like this are a _huge_ deal for that reason. It's an immense delayed recognition for over a decade of work from a lot of folks.

teej 6y ago
> my work was "utterly and completely useless I can never understand the arrogance of folks who would say something like this
- austincheney 6y ago
  I used to maintain a code beautifier/diff tool and hear the same things about how the idea was useless only to see those same people shortly there after use my tool or a close competitor. Once you see that pattern a few times you learn to ignore it in it’s entirety. It’s hard to tell why people behave like that. I presume it’s because many people have a great fear of originality and require social validation.
  - mjburgess 6y ago
    Asymmetry between complexity of principle and power. Ie., we are easily persuaded that something very complex will be very powerful (eg., a smart phone) -- but we intuitively regard something simple (eg., a hammer) as under-powered. Hard to say how well this actually holds, but I'd guess in both cases we arent really enumerating use-cases in our head, we're just using explanatory complexity as a guide to practical power. This is probably more extreme in cases where people have a specific notion of complexity in mind, eg., in academic environments where "tool A" is as simple as "tool B" if they use the same theoretical basis. ie., Tool C is worthwhile if it includes a more complex theory, as therefore it is more powerful.
- beambot 6y ago
  Academic administrators judge performance based on publication counts, journal impact factors, citations/h-index, and fundraising. Working on tooling doesn't fit in those buckets, so it's broadly-speaking "useless" to a researcher vying for promotions (eg tenure). It's an imperfect method of measuring true impact.
  - jofer 6y ago
    And to be fair, that was the context of the comment. It was meant to be harsh but true advice. It was quite arguably, at the time, worthless in the context of advancing my career. It turned into my career, eventually, but that wasn't the goal then.
- jcelerier 6y ago
  as a former phd student, it's pretty systemic - labs sometimes get funding as a direct mapping with how many papers get published by that lab. If you do cool work but that does not land you a paper, you are literally wasting the budget that your employer spent on you. Once you solve that (and thanks to things like this numpy papers, things may be starting to evolve !)
Random_ernest 6y ago
I recall a story where a friend was unable to publish a paper in which he wrote an alternative to a very commonly used commercial tool (that virtually everybody used) with roughly 10 times better performance. He open sourced it and all, it was extremely useful, but there was no new methodology, it was simply very well implemented. At a talk of his it lead to a very heated discussion where an older professor accused him of wasting government money on such nonsense.
- dr_zoidberg 6y ago
  Been there. A few years back I got a government scholarship for my PhD (which is still in progress, due to my follow up work). I basically built the foundation upon which to establish a new field for my university, and the region where I live. There are some professor who think that scholarship (and the little money it gave me) was wasted on my because I chose to build all of that from the ground up, instead of rushing through my PhD. By the way, those of that opinion are all professors who wanted me on their labs, but I turned them down...
  - yig 6y ago
    For every story like this, I believe there are many more in which the student simply writes their own implementation due to not invented here syndrome or engineering as a form of procrastination.
    - coliveira 6y ago
      Yes, it is easy to be sidetracked on writing software. Not that software is a bad thing, but research is something different.
    - V1ndaar 6y ago
      If you talked to me about my PhD for a few minutes you would surely put me into your "had to reinvent the wheel for no reason" category. As indeed, I wrote an analysis framework for my data (of a gaseous detector used for axion search) [0] instead of using an existing framework used by my predecessor. However, things are always more complicated than they seem. Many of those not talked about students who rewrite stuff probably have reasons! In my case the existing framework [1] was a monster that was bent to allow it to work with the kind of data we have in the first place. In my case my detector had several additional features, which fit _even less_ into the existing framework. It would have been a hack and still a significant amount of work to make it work well. To be fair, when I started this I expected it to be less work than it ended up being. But that's the story of software development. The advantages now are significant of course. I know the whole codebase. It does exactly what I want. I can extend it easily as I see fit. That doesn't mean I didn't also partly procrastinate writing software. Far from it. Hell, there was no reason to write a freaking plotting library (a sort of port of ggplot2 for Nim) [3]. But again, this means my thesis will have plots created natively using a TikZ backend while at the same time provide links to Vega-Lite plots for each and every plot in my thesis (which of course will include the data for each plot!). Finally, the most important point: A university / professor who only pays me for 20h a week does not get to tell me how I do my PhD. [0]: https://github.com/Vindaar/TimepixAnalysis https://github.com/Vindaar/TimepixAnalysis [1]: https://ilcsoft.desy.de/portal/software_packages/marlintpc/ https://ilcsoft.desy.de/portal/software_packages/marlintpc/ [2]: https://github.com/Vindaar/ggplotnim https://github.com/Vindaar/ggplotnim
scottlocklin 6y ago
I know Travis' and Paul Dubois[0] work at Livermore was immediately recognized as of towering importance almost immediately. I was down the highway at LBNL porting shitty Mathematica, IDL[1] and Fortran Diffraction Grating code to Numeric or whatever they called Numpy back then almost as soon as it was released. People probably don't remember their history, but pretty much the only open source intepreters of the day were things like Perl (whose math capabilities at the time were pretty lousy). Scientists paid for a shitload of Maple, Mathematica, IDL, Matlab and Igor[2] licenses; and there still weren't enough licenses to share code with your friends, because nobody had licenses for them all. Python 1.5 was the first non-mentat tier open source interpreter available that didn't get in your way as a scientist, and Numeric/Numpy was the first and still the most elementary piece that made it usable to science and numerics people. Might not have been letters to Nature tier back then, but Nature ain't what it used to be anyhow. [0] Since a lot of folks have actually forgotten Paul: http://www.pfdubois.com/bio.html http://www.pfdubois.com/bio.html [1] https://en.wikipedia.org/wiki/IDL_(programming_language) https://en.wikipedia.org/wiki/IDL_(programming_language) [2] https://en.wikipedia.org/wiki/IGOR_Pro https://en.wikipedia.org/wiki/IGOR_Pro
sandgiant 6y ago
I wholeheartedly agree. I think journal editors have a responsibility here too in promoting references to software libraries used in the articles they publish. I almost never see these in my field (astrophysics), even though they are readily available and very easy to include.
marmaduke 6y ago
We usually manage a Somethinginformatics journal publication to detail infrastructure work. At least Zenodo etc. And annoy users of the software to cite (a doi isn’t much to add to a manpage or log output). > for over a decade Probably two even if NumPy came out in 2005
throwaway-wroc 6y ago
props to your work and similar to numpy, i assume it has been immensely useful for loads of people. but 'building the underlying infrastructure that tons of people use' is not science. in my department we had to fail a phd student because 90% of his work was just implementing bunch of existing methods as a python library. useful, yes; science, no. wasn't his fault, had a shitty supervisor, but making useful tools is not the same as undertaking scientific research.
- wjnc 6y ago
  That's quite the wrong way of approaching science. The scientific method is based on building on the shoulders of giants. Those giants aren't the professors in the direct vicinity nor are they only the papers you cite. The whole of the process is science and if we need further specialization for building better tools (hey, maths and statistics are scientific tools as well) I would classify that as science to a large extent. What could be useful is openness in used tools and software and a way of getting citation counts for software used. It's nothing more than a table. That way the hotness of publication could start to flow for the underlying tools.
  - throwaway-wroc 6y ago
    > The scientific method is based on building on the shoulders of giants. > The whole of the process is science no. scientific research is proposing a useful model of an observable phenomenon. this is what you train for during a phd, at least in natural/life sciences: you learn how to test a hypothesis, not an easy skill. refactoring code or transforming bunch of C++ into a python library is useful, but it's not science. > What could be useful is openness in used tools and software and a way of getting citation counts for software used. It's nothing more than a table. That way the hotness of publication could start to flow for the underlying tools. agreed 100%
    - mantap 6y ago
      Maybe we should take all this "not science" software away from the scientists and see how much science they can do without it. If you write code that allows science to be done that couldn't be done otherwise then that is science. As a high profile example, a large amount of specialist software was developed for the LHC to allow it to process all the events coming from the detectors. It sounds like the refactoring here was not really that useful in the first place.
bitxbit 6y ago
This so much. I cannot stress enough what an impact various python developments have had on the economy as a whole. Sometimes the right tools can inspire people and that’s exactly what happened.

by jofer 6y ago

teej 6y ago

> my work was "utterly and completely useless I can never understand the arrogance of folks who would say something like this

austincheney 6y ago
I used to maintain a code beautifier/diff tool and hear the same things about how the idea was useless only to see those same people shortly there after use my tool or a close competitor. Once you see that pattern a few times you learn to ignore it in it’s entirety. It’s hard to tell why people behave like that. I presume it’s because many people have a great fear of originality and require social validation.
- mjburgess 6y ago
  Asymmetry between complexity of principle and power. Ie., we are easily persuaded that something very complex will be very powerful (eg., a smart phone) -- but we intuitively regard something simple (eg., a hammer) as under-powered. Hard to say how well this actually holds, but I'd guess in both cases we arent really enumerating use-cases in our head, we're just using explanatory complexity as a guide to practical power. This is probably more extreme in cases where people have a specific notion of complexity in mind, eg., in academic environments where "tool A" is as simple as "tool B" if they use the same theoretical basis. ie., Tool C is worthwhile if it includes a more complex theory, as therefore it is more powerful.
beambot 6y ago
Academic administrators judge performance based on publication counts, journal impact factors, citations/h-index, and fundraising. Working on tooling doesn't fit in those buckets, so it's broadly-speaking "useless" to a researcher vying for promotions (eg tenure). It's an imperfect method of measuring true impact.
- jofer 6y ago
  And to be fair, that was the context of the comment. It was meant to be harsh but true advice. It was quite arguably, at the time, worthless in the context of advancing my career. It turned into my career, eventually, but that wasn't the goal then.
jcelerier 6y ago
as a former phd student, it's pretty systemic - labs sometimes get funding as a direct mapping with how many papers get published by that lab. If you do cool work but that does not land you a paper, you are literally wasting the budget that your employer spent on you. Once you solve that (and thanks to things like this numpy papers, things may be starting to evolve !)

Random_ernest 6y ago

I recall a story where a friend was unable to publish a paper in which he wrote an alternative to a very commonly used commercial tool (that virtually everybody used) with roughly 10 times better performance. He open sourced it and all, it was extremely useful, but there was no new methodology, it was simply very well implemented. At a talk of his it lead to a very heated discussion where an older professor accused him of wasting government money on such nonsense.

dr_zoidberg 6y ago
Been there. A few years back I got a government scholarship for my PhD (which is still in progress, due to my follow up work). I basically built the foundation upon which to establish a new field for my university, and the region where I live. There are some professor who think that scholarship (and the little money it gave me) was wasted on my because I chose to build all of that from the ground up, instead of rushing through my PhD. By the way, those of that opinion are all professors who wanted me on their labs, but I turned them down...
- yig 6y ago
  For every story like this, I believe there are many more in which the student simply writes their own implementation due to not invented here syndrome or engineering as a form of procrastination.
  - coliveira 6y ago
    Yes, it is easy to be sidetracked on writing software. Not that software is a bad thing, but research is something different.
  - V1ndaar 6y ago
    If you talked to me about my PhD for a few minutes you would surely put me into your "had to reinvent the wheel for no reason" category. As indeed, I wrote an analysis framework for my data (of a gaseous detector used for axion search) [0] instead of using an existing framework used by my predecessor. However, things are always more complicated than they seem. Many of those not talked about students who rewrite stuff probably have reasons! In my case the existing framework [1] was a monster that was bent to allow it to work with the kind of data we have in the first place. In my case my detector had several additional features, which fit _even less_ into the existing framework. It would have been a hack and still a significant amount of work to make it work well. To be fair, when I started this I expected it to be less work than it ended up being. But that's the story of software development. The advantages now are significant of course. I know the whole codebase. It does exactly what I want. I can extend it easily as I see fit. That doesn't mean I didn't also partly procrastinate writing software. Far from it. Hell, there was no reason to write a freaking plotting library (a sort of port of ggplot2 for Nim) [3]. But again, this means my thesis will have plots created natively using a TikZ backend while at the same time provide links to Vega-Lite plots for each and every plot in my thesis (which of course will include the data for each plot!). Finally, the most important point: A university / professor who only pays me for 20h a week does not get to tell me how I do my PhD. [0]: https://github.com/Vindaar/TimepixAnalysis https://github.com/Vindaar/TimepixAnalysis [1]: https://ilcsoft.desy.de/portal/software_packages/marlintpc/ https://ilcsoft.desy.de/portal/software_packages/marlintpc/ [2]: https://github.com/Vindaar/ggplotnim https://github.com/Vindaar/ggplotnim

scottlocklin 6y ago

I know Travis' and Paul Dubois[0] work at Livermore was immediately recognized as of towering importance almost immediately. I was down the highway at LBNL porting shitty Mathematica, IDL[1] and Fortran Diffraction Grating code to Numeric or whatever they called Numpy back then almost as soon as it was released. People probably don't remember their history, but pretty much the only open source intepreters of the day were things like Perl (whose math capabilities at the time were pretty lousy). Scientists paid for a shitload of Maple, Mathematica, IDL, Matlab and Igor[2] licenses; and there still weren't enough licenses to share code with your friends, because nobody had licenses for them all. Python 1.5 was the first non-mentat tier open source interpreter available that didn't get in your way as a scientist, and Numeric/Numpy was the first and still the most elementary piece that made it usable to science and numerics people. Might not have been letters to Nature tier back then, but Nature ain't what it used to be anyhow. [0] Since a lot of folks have actually forgotten Paul: http://www.pfdubois.com/bio.html http://www.pfdubois.com/bio.html [1] https://en.wikipedia.org/wiki/IDL_(programming_language) https://en.wikipedia.org/wiki/IDL_(programming_language) [2] https://en.wikipedia.org/wiki/IGOR_Pro https://en.wikipedia.org/wiki/IGOR_Pro

sandgiant 6y ago

I wholeheartedly agree. I think journal editors have a responsibility here too in promoting references to software libraries used in the articles they publish. I almost never see these in my field (astrophysics), even though they are readily available and very easy to include.

marmaduke 6y ago

We usually manage a Somethinginformatics journal publication to detail infrastructure work. At least Zenodo etc. And annoy users of the software to cite (a doi isn’t much to add to a manpage or log output). > for over a decade Probably two even if NumPy came out in 2005

throwaway-wroc 6y ago

props to your work and similar to numpy, i assume it has been immensely useful for loads of people. but 'building the underlying infrastructure that tons of people use' is not science. in my department we had to fail a phd student because 90% of his work was just implementing bunch of existing methods as a python library. useful, yes; science, no. wasn't his fault, had a shitty supervisor, but making useful tools is not the same as undertaking scientific research.

wjnc 6y ago
That's quite the wrong way of approaching science. The scientific method is based on building on the shoulders of giants. Those giants aren't the professors in the direct vicinity nor are they only the papers you cite. The whole of the process is science and if we need further specialization for building better tools (hey, maths and statistics are scientific tools as well) I would classify that as science to a large extent. What could be useful is openness in used tools and software and a way of getting citation counts for software used. It's nothing more than a table. That way the hotness of publication could start to flow for the underlying tools.
- throwaway-wroc 6y ago
  > The scientific method is based on building on the shoulders of giants. > The whole of the process is science no. scientific research is proposing a useful model of an observable phenomenon. this is what you train for during a phd, at least in natural/life sciences: you learn how to test a hypothesis, not an easy skill. refactoring code or transforming bunch of C++ into a python library is useful, but it's not science. > What could be useful is openness in used tools and software and a way of getting citation counts for software used. It's nothing more than a table. That way the hotness of publication could start to flow for the underlying tools. agreed 100%
  - mantap 6y ago
    Maybe we should take all this "not science" software away from the scientists and see how much science they can do without it. If you write code that allows science to be done that couldn't be done otherwise then that is science. As a high profile example, a large amount of specialist software was developed for the LHC to allow it to process all the events coming from the detectors. It sounds like the refactoring here was not really that useful in the first place.

bitxbit 6y ago

This so much. I cannot stress enough what an impact various python developments have had on the economy as a whole. Sometimes the right tools can inspire people and that’s exactly what happened.