4 ms·

The microscope makers

40 points by lainon 9y ago

PoachedSausage 9y ago
Fascinating and very similar to what we have to go through in the particle physics community to get experiments to output data. I find it strange though that a good microscope team doesn't include an electronic engineer, yet: >requires a knowledge of optics, mechanics, electronics, computer programming and biology.
- frozenport 9y ago
  There are no "good microscope teams", the guy in the article worked for a noble prize laureate who routinely brags about having unlimited funding (for example, at Photonics West). There is nothing novel about the design and implementation, perhaps they got some new science with the experiments... I knew a guy who got a paper out in nature photonics, it took him 2 years to build a straight forward design. When his PI founded a company, the company re-did the work from scratch. It took them less than two months to make a C++ version. Features included the system not crashing. Later they got FDA approval. In summary, most people working in the field are graduate students with no practice experience, and competent engineers only appear when you need to commercialize a medical instrument. There are no microscope teams.
- posterboy 9y ago
  Particle Physics is high power, as far as I know, so the requirements for an electrical engineer might be quite different to low power drive circuitry. On the other hand, signal processing is inherently intertwined with electronics, so I'm sure they have technicians on board to deal with that, too.
- dekhn 9y ago
  Most of the electrical components in a modern high end microscope are basically off-the-shelf. You don't need a fulltime electrical engineer.
andbberger 9y ago
Oh fun, I can speak to this. I was at Janelia for the last year and a half, and actually just submitted a paper with Keller. I did ML stuff for reconstruction and otherwise post-magics with the images. There are a lot more people in that place working on scopes than just Keller and Betzig! Whole institution is basically vertically integrated fluorescence microscopy. You got your folks that work on making specific parts of mice or fly brains glow, your folks that build the special scopes, VR rigs for the mice to run around in while they're getting imaged... IMO we are close the limits of what can be done with 'classical' microscopy, classical meaning basically as practiced now. The real strides are going to be made in sophisticated reconstruction techniques leveraging the latest and greatest in ML and high fidelity simulations of the scope itself, which allow us to get close to reconstructing at the physical limit of information captured by the scope. But even with current 'classical' microscopy such as Keller et. al's Simview, the datasets we are collecting far exceed in complexity the neuro communities ability to analyze them, in particular at Janelia. Amongst other frustrations I have with the institution is the complete lack of interest in serious theory. Instead its all grasping at straws with very expensive and sophisticated straw grabbing apparati. Weird place, amazing science nonetheless. But my advice for neuro people considering a visit is to stay far away.
- peppery 9y ago
  Your work with ML and images sounds incredible! Do you have a link (e.g. on arxiv or elsewhere) that describes your approach using ML for image reconstruction in greater detail? How would you recommend building up one's combined intuition in optical theory, the relevant ML techniques, and the biological substances themselves, to the level where you can innovate in this task as you have done? Also, for those interested in the concept of building better images using higher fidelity simulations of the microscope itself, presumably Andrew meant studies along these lines: https://arxiv.org/abs/1702.07336 https://arxiv.org/abs/1702.07336
  - andbberger 9y ago
    That paper has the right general idea, you need a generative model of the scope and if you're clever you can use that to improve your reconstructions. But it's missing a couple key things that it workable in practice. One missing ingredient is a probabilistic model of the thing you're imaging. The others are secret ;) Secret because I have a dream for a crazy startup based on this idea which I don't have the means to do now. Although I generally hate being secretive about, like, knowledge and for sure the value creation happens during execution - but just humor me this time, ok? > How would you recommend building up one's combined intuition in optical theory, the relevant ML techniques, and the biological substances themselves, to the level where you can innovate in this task as you have done? Well, I'm flattered but I haven't done any substantial innovation here... My recommendation would be to become a theorist, learn math and physics and work your way up the hierarchy. You need to understand the whole picture, how it works on each level, and how the levels fit together - then you can run thought experiments. A good generative model of the world.

40 points by lainon 9y ago

Fascinating and very similar to what we have to go through in the particle physics community to get experiments to output data. I find it strange though that a good microscope team doesn't include an electronic engineer, yet: >requires a knowledge of optics, mechanics, electronics, computer programming and biology.

frozenport 9y ago
There are no "good microscope teams", the guy in the article worked for a noble prize laureate who routinely brags about having unlimited funding (for example, at Photonics West). There is nothing novel about the design and implementation, perhaps they got some new science with the experiments... I knew a guy who got a paper out in nature photonics, it took him 2 years to build a straight forward design. When his PI founded a company, the company re-did the work from scratch. It took them less than two months to make a C++ version. Features included the system not crashing. Later they got FDA approval. In summary, most people working in the field are graduate students with no practice experience, and competent engineers only appear when you need to commercialize a medical instrument. There are no microscope teams.
posterboy 9y ago
Particle Physics is high power, as far as I know, so the requirements for an electrical engineer might be quite different to low power drive circuitry. On the other hand, signal processing is inherently intertwined with electronics, so I'm sure they have technicians on board to deal with that, too.
dekhn 9y ago
Most of the electrical components in a modern high end microscope are basically off-the-shelf. You don't need a fulltime electrical engineer.

andbberger 9y ago

Oh fun, I can speak to this. I was at Janelia for the last year and a half, and actually just submitted a paper with Keller. I did ML stuff for reconstruction and otherwise post-magics with the images. There are a lot more people in that place working on scopes than just Keller and Betzig! Whole institution is basically vertically integrated fluorescence microscopy. You got your folks that work on making specific parts of mice or fly brains glow, your folks that build the special scopes, VR rigs for the mice to run around in while they're getting imaged... IMO we are close the limits of what can be done with 'classical' microscopy, classical meaning basically as practiced now. The real strides are going to be made in sophisticated reconstruction techniques leveraging the latest and greatest in ML and high fidelity simulations of the scope itself, which allow us to get close to reconstructing at the physical limit of information captured by the scope. But even with current 'classical' microscopy such as Keller et. al's Simview, the datasets we are collecting far exceed in complexity the neuro communities ability to analyze them, in particular at Janelia. Amongst other frustrations I have with the institution is the complete lack of interest in serious theory. Instead its all grasping at straws with very expensive and sophisticated straw grabbing apparati. Weird place, amazing science nonetheless. But my advice for neuro people considering a visit is to stay far away.

peppery 9y ago
Your work with ML and images sounds incredible! Do you have a link (e.g. on arxiv or elsewhere) that describes your approach using ML for image reconstruction in greater detail? How would you recommend building up one's combined intuition in optical theory, the relevant ML techniques, and the biological substances themselves, to the level where you can innovate in this task as you have done? Also, for those interested in the concept of building better images using higher fidelity simulations of the microscope itself, presumably Andrew meant studies along these lines: https://arxiv.org/abs/1702.07336 https://arxiv.org/abs/1702.07336
- andbberger 9y ago
  That paper has the right general idea, you need a generative model of the scope and if you're clever you can use that to improve your reconstructions. But it's missing a couple key things that it workable in practice. One missing ingredient is a probabilistic model of the thing you're imaging. The others are secret ;) Secret because I have a dream for a crazy startup based on this idea which I don't have the means to do now. Although I generally hate being secretive about, like, knowledge and for sure the value creation happens during execution - but just humor me this time, ok? > How would you recommend building up one's combined intuition in optical theory, the relevant ML techniques, and the biological substances themselves, to the level where you can innovate in this task as you have done? Well, I'm flattered but I haven't done any substantial innovation here... My recommendation would be to become a theorist, learn math and physics and work your way up the hierarchy. You need to understand the whole picture, how it works on each level, and how the levels fit together - then you can run thought experiments. A good generative model of the world.