6 ms·
Rendering protein structures inside cells at the atomic level with Unreal Engine
- canadiantim 3y agoWow, Unreal!
- eurekin 3y agoOne of videos: https://twitter.com/manorlaboratory/status/1763094135639294366/video/4 https://twitter.com/manorlaboratory/status/17630941356392943... Anybody came across the code?
- gilleain 3y agoThere's a link in the paper to a turorial: https://blake.bcm.edu/emanwiki/EMAN2/unreal_render https://blake.bcm.edu/emanwiki/EMAN2/unreal_render which uses ChimeraX: https://www.cgl.ucsf.edu/chimerax/ https://www.cgl.ucsf.edu/chimerax/
- dekhn 3y agoso it's using ChimeraX to turn a PDB file (protein or DNA structure) into an isosurface and then triangulating the surface into a mesh which is then rendered in unreal.
- gpnt 3y agovideos: https://www.biorxiv.org/content/10.1101/2023.12.08.570879v1.supplementary-material https://www.biorxiv.org/content/10.1101/2023.12.08.570879v1....
- COGlory 3y agohttps://github.com/cryoem/eman2/blob/master/programs/e2spt_mapptclstotomo_ue5.py https://github.com/cryoem/eman2/blob/master/programs/e2spt_m...
- gilleain 3y agoReminds me of the drawings of David Goodsell: https://ccsb.scripps.edu/goodsell/ https://ccsb.scripps.edu/goodsell/ which are similarly about the 'packed' nature of cells.
- timtimmy 3y agoI just released a biology education app very much like the preprint for the Vision Pro launch (and soon for iPad/iPhone). I worked with David Goodsell's group to integrate their whole-cell bacteria model and David wrote the content. It looks like this: https://twitter.com/timd_ca/status/1753250624677007492 https://twitter.com/timd_ca/status/1753250624677007492 Our first bit of content is a tour through a 300 million atom bacteria cell for Apple Vision Pro (>60 fps, stereoscopic, atomic resolution). We developed the tech for iPhone, iPad and AVP mobile GPUs (UE5 doesn't support this on the devices we're targeting). iPad: https://twitter.com/timd_ca/status/1592948101144547328 https://twitter.com/timd_ca/status/1592948101144547328 The linked preprint is beautiful, and I love the pipeline. I wonder if it's possible to export to other tools like Blender? The linked preprint is part of a pretty cool field of research into mesoscale modeling and visualization. For me these are a few of the standout papers, projects and works in the area (and there are many more): - le Muzic et al. "Multi-Scale Rendering of Large Biomolecular Datasets" 2015 [1] - - Ivan Viola's group helped pioneer large scale molecular visualization. This reference should be in the preprint, IMO. - Maritan et al. "3D Whole Cell Model of a Mycoplasma Bacterium" [2] - - This is out of David Goodsell's lab and the model I'm using. - Stevens et al. "Molecular dynamics simulation of an entire cell" [3] - Brady Johnston's Molecular Nodes addon for Blender [4] - YASARA PetWorld [5] [1] : https://www.cg.tuwien.ac.at/research/publications/2015/cellVIEW_2015/cellVIEW_2015-paper.pdf https://www.cg.tuwien.ac.at/research/publications/2015/cellV... [2] : https://ccsb.scripps.edu/gallery/mycoplasma_model/ https://ccsb.scripps.edu/gallery/mycoplasma_model/ [3] : https://twitter.com/JanAdStevens/status/1615693906137473030 https://twitter.com/JanAdStevens/status/1615693906137473030 and https://www.frontiersin.org/articles/10.3389/fchem.2023.1106495/full https://www.frontiersin.org/articles/10.3389/fchem.2023.1106... [4] : https://bradyajohnston.github.io/MolecularNodes/ https://bradyajohnston.github.io/MolecularNodes/ [5] : http://download.yasara.org/petworld/index.html http://download.yasara.org/petworld/index.html
- aendruk 3y ago
- COGlory 3y agoMuyuan Chen is one of (maybe the?) primary developer of the sub-tomogram averaging portion of the EMAN2 software package (linked below in another comment). Typically what you do is take a 3D tomogram (think like a scan) using a microscope, but it's extremely noisy. Then you go through and extract all the particles that are identical, but in different orientations, in the tomogram. So if the same protein is there multiple times, you can align them to each other, and average them together to increase the signal. Then you clone back in the higher signal averaged volume at the position and orientation that you found them in originally. The one-line command to go from EMAN2 coordinates to Unreal Engine 5 is kind of crazy. As usual on these (rare) threads, I'm happy to answer any questions about structural biology or cryo-EM.
- frodo8sam 3y agoSounds pretty cool, how does EMAN2 deal with dynamic structures? I assume you'll get garbage if sufficiently different conformations get averaged together. Is there some kind of clustering to find similar conformations as is sometimes done in cryo-EM?
- COGlory 3y agoYes, at multiple levels. You can do a heterogeneous refinement that takes the structures and solves for n number of averages, trying to use something like PCA to maximize the distances between the averages. Particles get sorted into the average they contribute most constructively to. That works well for compositional heterogeneity or large conformational differences. For minor differences, there's something called the guassian mixture model (lots of software packages have similar, but GMM is EMAN2's version). https://blake.bcm.edu/emanwiki/EMAN2/e2gmm https://blake.bcm.edu/emanwiki/EMAN2/e2gmm What you can get out of the other end is a volume series, that shows reconstructed 3D volumes along various axes of conformational variability. This quickly turns into a multi-dimensional problem, but it has been very successful in, for instance, seeing all the different states of an active ribosome.
- dalke 3y agoDo you know what the author means by "Current visualization software, such as UCSF ChimeraX6 , can only render one or a few protein structures at the atomic level." I haven't used VMD for about 30 years, but even in the 1990s I was using it to visualize the full poliovirus structure (4 proteins in 2PLV * 60 copies, as I recall). It took about 6-10 seconds per update on our SGI Onyx, but again, that was 25 years ago.
- abraxas 3y agothis reminds me of the video from years ago that blew my mind: https://vimeo.com/76306502 https://vimeo.com/76306502
- GeoAtreides 3y agorequires login : ( but it might be this video: https://www.youtube.com/watch?v=wJyUtbn0O5Y https://www.youtube.com/watch?v=wJyUtbn0O5Y
- abraxas 3y agoThis is an abberviated version of the same yes, I'm surprised though that the vimeo link isn't working without login. I don't have an account with them and was able to play it without issues.
- littlestymaar 3y agoSame for me, it says something about the video lacking an age classification.
- yardshop 3y agoIt didn't require one for me. There was a Google login box but I closed it and the video played fine. Maybe a regional thing?
- throwaway8877 3y agoI have seen this video. The complexity of life is mind blowing. Even more so the fact that we know anything about it.
- mchinen 3y agoThis is fascinating to watch. I understand that they get the proteins from PDB/ChimeraX, but how much manual process is involved to map and place the individual proteins? The paper says it gets the protein locations from CryoET tomograms, but I'd be surprised if these let you automatically identify which proteins are where, and exactly how to place them, and even less so how the ligomers bind together to form larger structures - for example, in the video the membrane surfaces are very smooth, and look almost textbook picture perfect, which suggests they come from a hardcoded model or are smoothed in some way. One part of the paper mentions subatomic averaging from the tomogram, but another mentions: > From a tomogram (Figure3a), we select the particles and determine the orientation of the crown of the spike, as well as the stalk that connects the spike to the membrane Is this a manual process, where the researcher is using his mental model of how the proteins are fit and placing/rotating individual proteins? Or do the tools they developed let this be automated. Both are impressive! If the former, I'm blown away by the effort it must take to make these kinds of videos.
- COGlory 3y agoIt's a guided, but automated process. EMAN2 (the software used/partially written by the author), for instance, has a convolutional neural network particle picker. So you can pick a few particles by hand, pick some noise, pick some bad particles, train a neural network, and then inference that network to pick the remaining particles throughout the tomogram. There are a variety of other methods too. There is simple 6 dimensional real-space cross correlation. You can place points by hand, or according to a model. For instance, if you are trying to identify viral spike proteins, and the virus is spherical, and the spike proteins are always on the surface of the sphere, you have a great starting point. So you can say "place points at n interval along the surface of this sphere" and then oversample the spherical surface. You then take a reference volume (can be generated a number of ways), and check each point you placed to see how well it matches the reference volume. You can allow for rotations and translations of the reference volume at each point, and if you find points that are too close together, you can merge them automatically. If you have a high contrast, relatively static protein (such as a ribosome), you can do 2D template matching in the tomogram, where you use a central slice (or maybe a collapsed projection) to do cross correlation in 3 dimensions instead of six (X translation, Y, translation, Z rotation). Or you can beef that up with more neural network/YOLO type stuff. EDIT: To expand on this, continuous density like membranes can be roughly modeled just with techniques like thresholding, watershedding, etc. There are some neural nets such as Membrain [0] and tomoseg [1] (also by the author of this paper), but membranes certainly are trickier. I typically segment membranes by hand (and do so rarely). [0] https://www.sciencedirect.com/science/article/pii/S0169260722003728 https://www.sciencedirect.com/science/article/pii/S016926072... [1] https://blake.bcm.edu/emanwiki/EMAN2/Programs/tomoseg https://blake.bcm.edu/emanwiki/EMAN2/Programs/tomoseg
- benjismith 3y agoThis reminds me of the amazing molecular animations of Drew Berry, which he showed in this TED talk: https://youtu.be/WFCvkkDSfIU?si=JNe06VS8TjIrHpqh https://youtu.be/WFCvkkDSfIU?si=JNe06VS8TjIrHpqh Which was 12 years ago! After watching that video, I had a much greater appreciation for how our bodies are made up of trillions of tiny protein machines. Fascinating stuff!!
- shagie 3y agohttps://xvivo.com/examples/the-inner-life-of-the-cell/ https://xvivo.com/examples/the-inner-life-of-the-cell/ I didn't realize there was also a Powering the Cell: Mitochondria video. The classic one narrated - https://youtu.be/QplXd76lAYQ https://youtu.be/QplXd76lAYQ One of my pandemic YouTube binges was watching Ron Vale videos about kinesin and dynein. https://youtu.be/9RUHJhskW00 https://youtu.be/9RUHJhskW00 https://youtu.be/lVwKiWSu8XE https://youtu.be/lVwKiWSu8XE https://youtu.be/FRtqfpO8THU https://youtu.be/FRtqfpO8THU And searching for Ron Vale will bring a number of other videos about molecular machines.
- benjismith 3y agoNice! Thank you for the links!
- dekhn 3y agoMy friend in grad school was in Ron's group. He built a microscope that visualized individual kinesin molecules and measured their speed using fluorescent labelling. The whole thing was held together with a bunch of scripts written in LabView. Ron had oodles of money and was able to support long-term software development of open source software like MicroManager, which gives a common interface to a wide range of microscopy software. The systems he studies are literally little motors that can attach to biological surfaces and drive around in specific directions, pick up payloads, and then drive to other places. They work in very different way from how humans engineer tiny motors and understanding/engineering their behavior was a major focus in the early 2000s.
- 3y ago
- vdm 3y agoVideos: https://www.biorxiv.org/content/10.1101/2023.12.08.570879v1.supplementary-material https://www.biorxiv.org/content/10.1101/2023.12.08.570879v1....
- leptons 3y agoNice videos but I'm always reminded when watching this type of molecular biology video that it's missing all the water molecules. These proteins and things aren't floating around in empty space.
- amelius 3y agoWhat is the meaning of color in these visualizations? Does every functional unit have its own color?
- leoncvlt 3y agoIf you get a kick out of 3D renderings of cells and molecules, you're gonna have a field day with the work done at https://random42.com/ https://random42.com/. PSA: I started working there as a 3D artist but now lead the interactive department. You'd be surprised at how much a good art direction really makes a difference in scientific visualization. Real-time graphics advanced considerably in the last couple years but it's always a challenge to transport that nice, smooth pre-rendered look over to mobile devices and the web at 60 frames per second (90 on virtual reality headsets, to boot...)
- dr_kiszonka 3y agoWow - these are stunning! I am curious if you have any "realistic time" animations, e.g., where blood circulates with the speed close to the one in the human body.
- zmmmmm 3y agoWould love a way to see these in 3D in VR / MR.
- protoman3000 3y agoI would like to ask a question and add before that that I have no intent to judge, discredit or diminish the value of this. It’s merely that I really don’t understand and would like to gain insight. The question is: How is this a scientific contribution? Or, to ask it differently: What makes this a scientific contribution?
- bglazer 3y agoIt’s more of an engineering accomplishment. I could see this being useful for exploratory data analysis of large protein (mesoscale) complexes. A surprising amount of science starts with a grad student staring at a really complex plot for a really long time, then suddenly going “oh shit thats weird”. That kind of realization is terribly difficult if your visualization tools are fighting you the whole time.
- BigParm 3y agoPeople overlook game engines for plotting and data visualization. You can have your character fly around whatever math/geometry you plotted. It’s great.