3 ms·
I'm one of those scientists, working in industry R&D. I use Python for everything from Jupyter notebooks to a roughly 8000 line program that runs a prototype of
by analog31 6y ago
I'm one of those scientists, working in industry R&D. I use Python for everything from Jupyter notebooks to a roughly 8000 line program that runs a prototype of a product. Python pulled me away from Visual Basic several years ago, and then I have a longer history going back through Turbo Pascal and ultimately originating with primordial BASIC. On the hardware side, I use C within the Arduino ecosystem. I've got roughly a half dozen other scientists at my work site using Python at various levels.
Confession: Code quality is a problem. I care a lot about code quality. My team recently brought on a couple of people with real programming background, and I've asked them to help me bring myself up to date on good techniques, and it has helped. I was lucky that some of my former training, e.g., in Pascal, was not just about how to program, but actually how to write good code (for its time).
But here's my view. Don't just tell us that our code sucks. We already know it, but we're under the same pressures as anybody else, and programming is a force multiplier that we won't give up. We need instructions that we can understand at our level, that will help us do this stuff better, perhaps incrementally. Starting with a strict discipline is hard, but Python supports the ability to go back through working code and improve it.
- The_Amp_Walrus 6y agoI'm working with some researches on about ~40k lines of Python disease modelling code and these are the things that helped us becomes more productive: - tests (unit, smoke, acceptance) (eg. https://mattsegal.dev/alternate-test-styles.html https://mattsegal.dev/alternate-test-styles.html, https://understandlegacycode.com/approval-tests/ https://understandlegacycode.com/approval-tests/) - continuous integration (via GitHub actions eg: https://mattsegal.dev/pytest-on-github-actions.html https://mattsegal.dev/pytest-on-github-actions.html) - 10x performance improvements via cProfile (https://julien.danjou.info/guide-to-python-profiling-cprofile-concrete-case-carbonara/ https://julien.danjou.info/guide-to-python-profiling-cprofil...) - getting fast feedback on code changes by visualizing outputs with Streamlit (https://www.streamlit.io/ https://www.streamlit.io/) - input parameter validation (eg. https://pydantic-docs.helpmanual.io/ https://pydantic-docs.helpmanual.io/, https://mattsegal.dev/cerberus-config-validation.html https://mattsegal.dev/cerberus-config-validation.html) - total automation of data processing job (eg. https://www.prefect.io/ https://www.prefect.io/ - we don't use this but that's the kind of thing we have)
- Jugurtha 6y agoYour post reminds me of a segment[0] of a Pycon keynote where Jacob Kaplan Moss talks about "real programmers". One of the anecdotes was that he was hiring programmers and he stumbled on a student who had worked on a project who told him she can't apply because she "isn't really a programmer". >I've asked them to help me bring myself up to date on good techniques Here's an old draft of our onboarding document[1], and a few comments here on HN about this[2]. PS: in case anyone is wondering how I found the segment in question, I use a Chrome extension called "YouTube Captions Search"[3] went to the video, activated closed captions, clicked on the extension's icon, and searched for "geographic" because I remembered that the system the student had built was a geographic information system (GIS). - [0]: https://youtu.be/iOp0mSDCh6c?t=1084 https://youtu.be/iOp0mSDCh6c?t=1084 - [1]: https://jhadjar.gitlab.io/kbase/hiring/#resources-to-learn-our-stack https://jhadjar.gitlab.io/kbase/hiring/#resources-to-learn-o... - [2]: https://news.ycombinator.com/item?id=15576496 https://news.ycombinator.com/item?id=15576496 - [3]: https://chrome.google.com/webstore/detail/youtube-captions-search/kimbeggjgnmckoikpckibeoaocafcpbg?hl=en https://chrome.google.com/webstore/detail/youtube-captions-s...
- iainctduncan 6y agoThis is an area I want to get more into actually. I've done a bunch of freelancing for scientists, and it seems to me that there is now a wave of scientists hitting the code size and complexity level where understanding the principles of maintainable software architecture is become pretty important. One of the gigs was reverse-documenting some RCPP because they had been left with a ball-of-mud someone had to write in a hurry and no one knew how to break it up. It was interesting, and cool to help people doing real science. If you have any thoughts about the viability of teaching architecture to scientists, I'm all ears. You can find me on linked on or my user name plus the email of a behemoth starting with G... ;-)