5 ms·

Oh fun, I can speak to this. I was at Janelia for the last year and a half, and actually just submitted a paper with Keller. I did ML stuff for reconstruction a

Oh fun, I can speak to this. I was at Janelia for the last year and a half, and actually just submitted a paper with Keller. I did ML stuff for reconstruction and otherwise post-magics with the images. There are a lot more people in that place working on scopes than just Keller and Betzig! Whole institution is basically vertically integrated fluorescence microscopy. You got your folks that work on making specific parts of mice or fly brains glow, your folks that build the special scopes, VR rigs for the mice to run around in while they're getting imaged... IMO we are close the limits of what can be done with 'classical' microscopy, classical meaning basically as practiced now. The real strides are going to be made in sophisticated reconstruction techniques leveraging the latest and greatest in ML and high fidelity simulations of the scope itself, which allow us to get close to reconstructing at the physical limit of information captured by the scope. But even with current 'classical' microscopy such as Keller et. al's Simview, the datasets we are collecting far exceed in complexity the neuro communities ability to analyze them, in particular at Janelia. Amongst other frustrations I have with the institution is the complete lack of interest in serious theory. Instead its all grasping at straws with very expensive and sophisticated straw grabbing apparati. Weird place, amazing science nonetheless. But my advice for neuro people considering a visit is to stay far away.

peppery 9y ago
Your work with ML and images sounds incredible! Do you have a link (e.g. on arxiv or elsewhere) that describes your approach using ML for image reconstruction in greater detail? How would you recommend building up one's combined intuition in optical theory, the relevant ML techniques, and the biological substances themselves, to the level where you can innovate in this task as you have done? Also, for those interested in the concept of building better images using higher fidelity simulations of the microscope itself, presumably Andrew meant studies along these lines: https://arxiv.org/abs/1702.07336 https://arxiv.org/abs/1702.07336
- andbberger 9y ago
  That paper has the right general idea, you need a generative model of the scope and if you're clever you can use that to improve your reconstructions. But it's missing a couple key things that it workable in practice. One missing ingredient is a probabilistic model of the thing you're imaging. The others are secret ;) Secret because I have a dream for a crazy startup based on this idea which I don't have the means to do now. Although I generally hate being secretive about, like, knowledge and for sure the value creation happens during execution - but just humor me this time, ok? > How would you recommend building up one's combined intuition in optical theory, the relevant ML techniques, and the biological substances themselves, to the level where you can innovate in this task as you have done? Well, I'm flattered but I haven't done any substantial innovation here... My recommendation would be to become a theorist, learn math and physics and work your way up the hierarchy. You need to understand the whole picture, how it works on each level, and how the levels fit together - then you can run thought experiments. A good generative model of the world.
  - peppery 9y ago
    Thanks for your sharing your thoughts! > One missing ingredient is a probabilistic model of the thing you're imaging Ooh, this (making the prior for the true image signal more informative by incorporating knowledge of the structure of the signal) is clever. Here, when you say model, you mean a description that is based on the (bio)physics of your sample? (E.g., knowing that objects being imaged obey diffusion equations informs your maximum likelihood estimation of the true signal?) > Secret because I have a dream for a crazy startup based on this idea which I don't have the means to do now. Although I generally hate being secretive about, like, knowledge and for sure the value creation happens during execution - but just humor me this time, ok? Of course; such is the right (and joy) of an innovator to define how one's own idea is disseminated/actualized! (~: It is intriguing that you feel your idea has the character of best being pursued via a startup (addressing some crucial unmet commercial need), rather than via the academic model (e.g. transformative Nature publication) more commonly used for improved microscopy techniques. Best of luck in this pursuit; I look forward to seeing your startup's innovations someday soon! > You need to understand the whole picture, how it works on each level, and how the levels fit together - then you can run thought experiments. A good generative model of the world. This idea of developing a "good generative model of the world" is a beautiful aspiration for all of us to have. Thanks for your insights!

peppery 9y ago

Your work with ML and images sounds incredible! Do you have a link (e.g. on arxiv or elsewhere) that describes your approach using ML for image reconstruction in greater detail? How would you recommend building up one's combined intuition in optical theory, the relevant ML techniques, and the biological substances themselves, to the level where you can innovate in this task as you have done? Also, for those interested in the concept of building better images using higher fidelity simulations of the microscope itself, presumably Andrew meant studies along these lines: https://arxiv.org/abs/1702.07336 https://arxiv.org/abs/1702.07336

andbberger 9y ago
That paper has the right general idea, you need a generative model of the scope and if you're clever you can use that to improve your reconstructions. But it's missing a couple key things that it workable in practice. One missing ingredient is a probabilistic model of the thing you're imaging. The others are secret ;) Secret because I have a dream for a crazy startup based on this idea which I don't have the means to do now. Although I generally hate being secretive about, like, knowledge and for sure the value creation happens during execution - but just humor me this time, ok? > How would you recommend building up one's combined intuition in optical theory, the relevant ML techniques, and the biological substances themselves, to the level where you can innovate in this task as you have done? Well, I'm flattered but I haven't done any substantial innovation here... My recommendation would be to become a theorist, learn math and physics and work your way up the hierarchy. You need to understand the whole picture, how it works on each level, and how the levels fit together - then you can run thought experiments. A good generative model of the world.
- peppery 9y ago
  Thanks for your sharing your thoughts! > One missing ingredient is a probabilistic model of the thing you're imaging Ooh, this (making the prior for the true image signal more informative by incorporating knowledge of the structure of the signal) is clever. Here, when you say model, you mean a description that is based on the (bio)physics of your sample? (E.g., knowing that objects being imaged obey diffusion equations informs your maximum likelihood estimation of the true signal?) > Secret because I have a dream for a crazy startup based on this idea which I don't have the means to do now. Although I generally hate being secretive about, like, knowledge and for sure the value creation happens during execution - but just humor me this time, ok? Of course; such is the right (and joy) of an innovator to define how one's own idea is disseminated/actualized! (~: It is intriguing that you feel your idea has the character of best being pursued via a startup (addressing some crucial unmet commercial need), rather than via the academic model (e.g. transformative Nature publication) more commonly used for improved microscopy techniques. Best of luck in this pursuit; I look forward to seeing your startup's innovations someday soon! > You need to understand the whole picture, how it works on each level, and how the levels fit together - then you can run thought experiments. A good generative model of the world. This idea of developing a "good generative model of the world" is a beautiful aspiration for all of us to have. Thanks for your insights!