7 ms·

There is a non-trivial amount of work being done at university research labs that has yet to feel the effects of the machine learning revolution because PIs are

There is a non-trivial amount of work being done at university research labs that has yet to feel the effects of the machine learning revolution because PIs aren’t investing (or cannot afford) programmers, and scientific equipment is manufactured by a small handful of companies who have little incentive to innovate. Please note that n=2, but both myself and a friend of met have met people working in life science labs that were assigning research assistants to manually count cells in images, and manually sweeping through images to find stills that were in focus.

technics256 7y ago
I'm a consultant and work in AI and cloud stuff - do these lab instruments have cameras or ways to feed the images into an ai model? I can see about creating something basic for you.
- dwrodri 7y ago
  The images are z-stacks[1] and the images are in a proprietary format called LIF (Leica Image Format). The images are taken from this camera[2]. I was actually doing some of my own research into doing this, but as I mentioned it appears that Leica has put very little effort into making the output of their hardware easy to work with. The LIF is a binary blob wrapped in an XML structure. There has been some good work done by the people Open Microscopy Environment building some Java libraries for parsing these files, but to be quite frank I'm a bit rough around the edges when it comes to working with Java and all the computer vision stuff I know is in Python and C++. If you're serious about this offer, then I can look into getting you some samples (if you can't easily find them yourself). But full disclosure, I don't think any PI or lab that I know of would be able to afford GPU compute hours, but most of them have access to a cluster, so if you can deliver a JAR (or a Docker container) I could demonstrate how they could quickly deploy that themselves on their own hardware. 1:https://cam.facilities.northwestern.edu/588-2/z-stack/ https://cam.facilities.northwestern.edu/588-2/z-stack/
  - technics256 7y ago
    Thanks! I have a lot of credits to play with..your email is not on your profile. But mine is, can you message me there so we can chat offline? I can take a Stab, but the file format and Java stuff could be a problem. Let's see
quaquaqua1 7y ago
Interesting problem. If that sort of thing is able to be solved with a few raspis, open mpi, and some ML libraries then cool. But if it's more complex than that, basically it's an MA thesis in itself :)
- IggleSniggle 7y ago
  Even with a few raspis, open mpi, and some ML libraries, can probably be an MA thesis.
chudi 7y ago
Yeah, these are the problems with cs phd's, they are in huge demand to actually do interdisciplinary work outside of their field and usually don't get enough recognition for the value they put in the actual research.
titanomachy 7y ago
I learned programming in order to do my job more effectively while I was employed as a research assistant in a university neuroscience lab. At some point I realized that programmers in industry get paid 10x more (not an exaggeration, literally 10x) so I left after a couple years. I enjoyed my time there a lot, I was building tools to help very smart and motivated people solve cool science problems. My users sat across the lab bench from me (the original open office plan?) and iteration/feedback cycles were super fast. People were appreciative and I could see how much time I was saving them. It was very motivating. Some of the more useful stuff I built got picked up by other labs in the group. Maybe I'll do that kind of thing again someday, if I decide I've had my fill of megacorp programming.
- mynegation 7y ago
  Out of curiosity - what are the typical tools that you built? I am in industry but sometimes have a crisis of meaning. I would not be able to sit next to researches but was wondering if there is a gap in open source tools that I (or someone else) could help to cover.
  - chudi 7y ago
    I did similar work at a neurosciences lab, my work was usually building experiments at large (think html5, android games that were actual experiments), building infrastructure, connecting some kind of sensor to a computer and saving the data, guiding bachelors students with their final project that required some kind of programming, not very different from my usual work but with a lot of emphasis in reproducibility and log about every single metric that you can track that may be of the interest of the scientist.
  - HeyLaughingBoy 7y ago
    There is a ton of this kind of work available. I run into various people online who need a simple automation task done, but either don't know where to start, or start and don't know how to make progress. e.g., a month ago I made a small machine for a dental research lab that basically added a feature to a piece of test equipment. The feature was so basic and obvious, I have no idea why the equipment didn't already have it. But it was an opportunity for me to step in and make a small piece of hardware to do what they need at little cost. A few months before that I made a GUI for a customized bit of veterinary equipment, and so on... These jobs are out there. They're quick and easy for an experienced engineer but the problem is finding them consistently. My main focus these days is trying to do just that.
karmelapple 7y ago
Agreed. But also agreed there’s little room for paying for software in this world, either. From my anecdotal observations, grad students or postdocs learn just the bit of code they need to accomplish a few things, then either never sharing the code online or never attempting to maintain it (because they never would have learned that anyway, nor do they have the time or motivation). And there’s no significant funding to change this.
peterwoerner 7y ago
True, there is a nontrivial amount of very useful software which could be written for universities across the science and engineering fields, but very little money to pay for it. Generally, grant money cannot be spent on software so investment in tools is negligible.
- thfuran 7y ago
  >Generally, grant money cannot be spent on software Really? That seems a few decades behind the times.
  - peterwoerner 7y ago
    That was the case for me and I graduated about 2 years ago. Professors were paying from their salary to get e.g. Matlab. More specifically, software is usually not a direct expense so it needs to be paid from the grant overhead, but the university frequently wants to build new buildings or something else instead. It's the same for private government contractors.
    - deleted 7y ago
      [deleted]
ereinertsen 7y ago
The Broad Institute has assembled a really talented and growing team of engineers and data scientists in an effort to accelerate science: https://www.broadinstitute.org/data-sciences-platform https://www.broadinstitute.org/data-sciences-platform (DSP). They develop open-source platforms that are inspired by and seek to solve problems lived firsthand by collaborators (such as myself) and the greater scientific community. The Broad has created an environment and provided resources to enable engineers to do great work. The DSP feels like a blend between tech company and academic research institution. The people I've interacted with are technically superb (and also nice). If you are interested in contributing to the intersection of life science and compute, would strongly recommend you check them out. This may be off topic if you consider above to not be sufficiently "blue collar", but biomedicine in general (from the entire spectrum of basic science all the way to care delivery) is in dire need of smart computational folks.

technics256 7y ago

I'm a consultant and work in AI and cloud stuff - do these lab instruments have cameras or ways to feed the images into an ai model? I can see about creating something basic for you.

dwrodri 7y ago
The images are z-stacks[1] and the images are in a proprietary format called LIF (Leica Image Format). The images are taken from this camera[2]. I was actually doing some of my own research into doing this, but as I mentioned it appears that Leica has put very little effort into making the output of their hardware easy to work with. The LIF is a binary blob wrapped in an XML structure. There has been some good work done by the people Open Microscopy Environment building some Java libraries for parsing these files, but to be quite frank I'm a bit rough around the edges when it comes to working with Java and all the computer vision stuff I know is in Python and C++. If you're serious about this offer, then I can look into getting you some samples (if you can't easily find them yourself). But full disclosure, I don't think any PI or lab that I know of would be able to afford GPU compute hours, but most of them have access to a cluster, so if you can deliver a JAR (or a Docker container) I could demonstrate how they could quickly deploy that themselves on their own hardware. 1:https://cam.facilities.northwestern.edu/588-2/z-stack/ https://cam.facilities.northwestern.edu/588-2/z-stack/
- technics256 7y ago
  Thanks! I have a lot of credits to play with..your email is not on your profile. But mine is, can you message me there so we can chat offline? I can take a Stab, but the file format and Java stuff could be a problem. Let's see

quaquaqua1 7y ago

Interesting problem. If that sort of thing is able to be solved with a few raspis, open mpi, and some ML libraries then cool. But if it's more complex than that, basically it's an MA thesis in itself :)

IggleSniggle 7y ago
Even with a few raspis, open mpi, and some ML libraries, can probably be an MA thesis.

chudi 7y ago

Yeah, these are the problems with cs phd's, they are in huge demand to actually do interdisciplinary work outside of their field and usually don't get enough recognition for the value they put in the actual research.

titanomachy 7y ago

I learned programming in order to do my job more effectively while I was employed as a research assistant in a university neuroscience lab. At some point I realized that programmers in industry get paid 10x more (not an exaggeration, literally 10x) so I left after a couple years. I enjoyed my time there a lot, I was building tools to help very smart and motivated people solve cool science problems. My users sat across the lab bench from me (the original open office plan?) and iteration/feedback cycles were super fast. People were appreciative and I could see how much time I was saving them. It was very motivating. Some of the more useful stuff I built got picked up by other labs in the group. Maybe I'll do that kind of thing again someday, if I decide I've had my fill of megacorp programming.

mynegation 7y ago
Out of curiosity - what are the typical tools that you built? I am in industry but sometimes have a crisis of meaning. I would not be able to sit next to researches but was wondering if there is a gap in open source tools that I (or someone else) could help to cover.
- chudi 7y ago
  I did similar work at a neurosciences lab, my work was usually building experiments at large (think html5, android games that were actual experiments), building infrastructure, connecting some kind of sensor to a computer and saving the data, guiding bachelors students with their final project that required some kind of programming, not very different from my usual work but with a lot of emphasis in reproducibility and log about every single metric that you can track that may be of the interest of the scientist.
- HeyLaughingBoy 7y ago
  There is a ton of this kind of work available. I run into various people online who need a simple automation task done, but either don't know where to start, or start and don't know how to make progress. e.g., a month ago I made a small machine for a dental research lab that basically added a feature to a piece of test equipment. The feature was so basic and obvious, I have no idea why the equipment didn't already have it. But it was an opportunity for me to step in and make a small piece of hardware to do what they need at little cost. A few months before that I made a GUI for a customized bit of veterinary equipment, and so on... These jobs are out there. They're quick and easy for an experienced engineer but the problem is finding them consistently. My main focus these days is trying to do just that.

karmelapple 7y ago

Agreed. But also agreed there’s little room for paying for software in this world, either. From my anecdotal observations, grad students or postdocs learn just the bit of code they need to accomplish a few things, then either never sharing the code online or never attempting to maintain it (because they never would have learned that anyway, nor do they have the time or motivation). And there’s no significant funding to change this.

peterwoerner 7y ago

True, there is a nontrivial amount of very useful software which could be written for universities across the science and engineering fields, but very little money to pay for it. Generally, grant money cannot be spent on software so investment in tools is negligible.

thfuran 7y ago
>Generally, grant money cannot be spent on software Really? That seems a few decades behind the times.
- peterwoerner 7y ago
  That was the case for me and I graduated about 2 years ago. Professors were paying from their salary to get e.g. Matlab. More specifically, software is usually not a direct expense so it needs to be paid from the grant overhead, but the university frequently wants to build new buildings or something else instead. It's the same for private government contractors.
  - deleted 7y ago
    [deleted]

ereinertsen 7y ago

The Broad Institute has assembled a really talented and growing team of engineers and data scientists in an effort to accelerate science: https://www.broadinstitute.org/data-sciences-platform https://www.broadinstitute.org/data-sciences-platform (DSP). They develop open-source platforms that are inspired by and seek to solve problems lived firsthand by collaborators (such as myself) and the greater scientific community. The Broad has created an environment and provided resources to enable engineers to do great work. The DSP feels like a blend between tech company and academic research institution. The people I've interacted with are technically superb (and also nice). If you are interested in contributing to the intersection of life science and compute, would strongly recommend you check them out. This may be off topic if you consider above to not be sufficiently "blue collar", but biomedicine in general (from the entire spectrum of basic science all the way to care delivery) is in dire need of smart computational folks.