10 ms·

Recreating Our Galaxy in a Supercomputer

• jessriedel 10y ago
  For context, this question of whether vanilla dark matter models are in conflict with galactic observations has been raging for many years. (It includes both the dwarf galaxy problem mentioned in the article, and things like the "core-cusp" problem concerning the radial dark matter density distribution.) These numerical simulations are fiendishly complicated, and there's always a risk that you just keep adding new tweaks or effects until you get the results you're looking for. I'd be interested in reading a skeptical assessment from the folks on the other side of this question.
• starmole 10y ago
  Can somebody who knows about this tell me how much of the novelty in this is CS and how much physics? For example in graphics all the physics of light is well understood - it's just infeasible to compute - so all advances are in CS and how to sample better. Is this the case here? Or is it adding new models?
  • sillysaurus3 10y ago
    For example in graphics all the physics of light is well understood - it's just infeasible to compute - so all advances are in CS and how to sample better. Nitpick: The physics of light is understood. Human vision is also understood. But critically, human vision is completely overlooked in most graphical applications. It's not accurate to say that "the path toward making a computer-generated video indistinguishable from a camcorder recording is just a matter of sampling better." There will need to be a fundamental shift in the industry's thinking before realism will be achieved. It's a fascinating topic that occupied many years of my life, though this is a bit off-topic.
    • Le_SDT 10y ago
      I'm curious. What do you mean by "Human vision is completely overlooked"? Are you talking about color perception? 3d rendering? Thanks :)
      • dharma1 10y ago
        Maybe foveation? dynamic range perception?
      • sillysaurus3 10y ago
        There's pretty much no way to avoid writing a large comment in response to this. So, with apologies... It was my focus for a long time to achieve perfectly photorealistic rendering. I come from a gamedev background, and I've been fascinated since around age ten about how to get a computer to paint pictures. By 17 I was writing game engines mostly equivalent to quake, and used this portfolio to get into the industry. The next three years were spent getting as close as possible to the bleeding edge of real-time graphics development. I remember having long debates with a colleague about what high-dynamic range lighting "meant." "If we spin around in our office chair, our brains do not suddenly change the overall brightness of this office. Why should we be programming games to do this? Why is everyone doing things that way?" My concerns were of a more fundamental nature as well. What is a diffuse texture? A diffuse texture is a well-understood concept in realtime graphics. Anyone with a basic knowledge of shaders should immediately recognize: color = lighting * diffuse + ambient Trouble is, it doesn't correspond to reality even slightly. It's not even roughly close. It happens to look good to humans, and that's why we use it. But the trouble was, the further I tried to probe the mystery of realism in computer graphics, the more I ran against this phenomenon of "We use X technique because it looks good." So I turned to research papers. Books. The medical field. Everywhere that was remotely related to possible breakthroughs in photorealistic rendering. Research papers are excellent for assembling techniques, but not results. The books in human vision and color science were more promising, yet most of the industry seemed (and still seems) to pay little attention to them. Compare a book about color perception to, say, http://www.pbrt.org/ http://www.pbrt.org/ and you'll see a stark difference. Flip through the table of contents and you get transformations, shapes, primitives, color and radiometery, sampling, reflection, materials, texture, scattering, light sources, monte carlo integration... And for what? We know that these techniques simply do not produce computer-generated videos that a human will identify as a real-life image. It's not for lack of processing power. There is a disconnect between the old rules and those that will ultimately result in real-time realism, and you won't find it in that table of contents. Now, the trouble with writing all of this is that if I knew how to do it, I'd have done it already. It's a life-long search, and it's not so easy to refute an entire industry without being (rightly) dismissed. But if you wish to know what I suspect are the ways forward, it's this: Get a camera. Take photos. Compare these photos to the results of the algorithms you write. Iterate on your algorithms until they are producing results that match something that already captures nature, not our beliefs about how we ought to be able to capture nature. "Just throw in physical models and presto!" has not thus far been true. You'll notice, for example, that computer graphics in videogames have plateaued. They get more impressive with each generation, but that impressiveness does not get them progressively closer to looking real. Nor should it. A computer game tells a story. The closer it looks to real life, the more restricted the artists are, along with the rest of the design of the game. So we turn to the movie industry for hope. But it's restricted in exactly the same way. The research papers are all along the lines of new techniques to try, or studies of existing pipelines and how to deal with their complexities. It's not fundamental research. As someone who has spent his life in pursuit of realism in computer-generated video, my recommendation is this: Read DaVinci's journal. Pay attention to what each page is saying. He had to discover from first principles what makes a painting look real, and why. You'll notice that he spends most of his time talking about human vision and our perceptions of color. If someone is going to make this development happen, it's not going to come from the game industry, and it won't come from the movie industry. That leaves you. Hopefully this will encourage some of you to pursue this. Once you accept that most of the computer graphics industry isn't actually focused on achieving realism, you'll start to develop your own techniques. My hope is that this will eventually lead to a breakthrough.
    • dekhn 10y ago
      There are scenes in Pixar's movies, rendered entirely by Renderman, that are effectively indistinguishable from a high resolution recording of a real instance of the subject. That really was all about algorithms and sampling.
  • privong 10y ago
    > Can somebody who knows about this tell me how much of the novelty in this is CS and how much physics? It's a little bit of both. Over the past 5–8 years, some new numerical techniques ("moving mesh") have been developed (or at least first applied in astronomy) for hydrodynamic simulations[0,1]. These new techniques do a much better job of handling fluid instabilities, which can be important for properly simulating the gas in galaxies (which can have a huge range in densities and temperatures). Also, people have developed what they call "cosmological zoom" simulations. In these simulations, you run a low-resolution (sometimes dark-matter only) simulation of a fairly large volume of the universe. Then you identify a particular dark matter halo at the end, that you think matches the type of galaxy you're interested in (say, a Milky Way type galaxy). Then you go back to the initial conditions, re-initialize it at very high resolution and include "normal matter" (baryons) and re-simulate the whole thing to study the formation of that particular dark matter halo and its associated galaxy. Finally, the general increase in computing speed has allowed people to add in more physics to the simulations and run higher-resolution simulations. In this case, it is the addition of a physically-motivated model for energy injection into the gas in galaxies, from the ongoing star formation. This means energy from supernovae explosions and associated shocks, stellar winds, and momentum from scattering/absorption of photons by dust grains[2,3]. This more realistic set of models manages to reproduce the overall efficiency of star formation (what fraction of the gas gets turned into stars over some period of time) _and_ create a realistic temperature-density structure for the gas. Now that people seem confident that these models can reproduce the overall star formation properties in galaxies, they are starting to see what this type of star formation / feedback model implies for the properties of galaxies. Of course, there's still lots of work to be done in making detailed tests of these models. While they reproduce the overall efficiency of star formation, we aren't yet sure if they can reproduce the shapes and sizes of galaxies. But people are working on that and we should know before too long... [0] https://arxiv.org/abs/0901.4107 https://arxiv.org/abs/0901.4107 [1] https://arxiv.org/abs/1409.7395 https://arxiv.org/abs/1409.7395 [2] https://arxiv.org/abs/1311.2073 https://arxiv.org/abs/1311.2073 [3] https://arxiv.org/abs/1307.5639 https://arxiv.org/abs/1307.5639
  • Zardoz84 10y ago
    And if these image comes from the simulation... Where is the stars bar around the core ??
  • adekok 10y ago
    > Can somebody who knows about this tell me how much of the novelty in this is CS and how much physics? It's not so much a novelty as it is pushing the envelope. The main problem with these kind of simulations is that they're O(N^2). You can't get away with modelling each star, you have to model the interaction of each star with every other star. And then add in interstellar dust, which has non-neglible mass. And then run the simulation for 8 billion years. There are naturally some optimizations and short-cuts to be made. :)
    • privong 10y ago
      > The main problem with these kind of simulations is that they're O(N^2). You can't get away with modelling each star, you have to model the interaction of each star with every other star. Summary of what's below: The type of simulations described in the article do not scale like O(N^2) but more like O(N log N). A small correction here.. The gravitational force calculation for N-body simulations are only O(N^2) if they're done using direct summation and force calculation. That's only necessary if you care about the motions of individual stars. The largest simulations I'm aware of that do that use a few 10s or 100s of millions of stars. They're typically used for studying individual star clusters, the regions around supermassive black holes, and perhaps dwarf galaxies. But the simulation in this article/press release has a few 10s of billions of particles, I believe. And those particles are all much more massive than individual stars. So while there are "star particles" in the simulations, it's more accurate to think of them as a set of particles which constitute a monte carlo sampling of the stellar mass distribution. One aspect of that is you now no longer care about the detailed trajectories of individual particles[0], instead caring about the overall representation of the mass distribution. So you don't need as high of accuracy in the force calculation. Going back to the 1980's, people have been using numerical tricks to reduce the scaling of the simulations with the particle number. This includes algorithms such as treecodes, particle-mesh codes, etc., which reduce the scaling to O(N log N) or even O(N). These are sufficient to run simulations of describing the overall gravitational field. The scaling for hydro codes is probably more complicated, particularly for simulations which use a moving mesh. There's a nice summary of numerical techniques for this type of work here: http://arxiv.org/abs/1412.5187 http://arxiv.org/abs/1412.5187 [0] One can determine if interactions between individual stars is important to a system's evolution by comparing the 2-body relaxation time to the age of the system. This is discussed on pages 8–9 of the preprint linked above.
  • yobaby1234 10y ago
    Well, if you look at it, its physics all the way .. After all, CS is nothing but simple mathematics for dummies who can't but sit in front of a computer and blow themselves for being the masters of the world .!!
• fiatjaf 10y ago
  Nice, you can't model a worm, but you can model a galaxy.
  • misja111 10y ago
    http://www.openworm.org/ http://www.openworm.org/
    • FeepingCreature 10y ago
      Still don't have a working integrated model.
  • JabavuAdams 10y ago
    Molecules are tricky. They're obviously machines, but you're dealing with Quantum Mechanics if you go to first principles.
    • dekhn 10y ago
      Which is why, for nearly all simulations, you use Born-Oppenheimer approximation and molecular dynamics using force fields derived from QM, but without the full explicit QM treatment. This is fine because with only rare examples (as far as modern science knows), QM treatment is not necessary for most molecular machines.
      • nonbel 10y ago
        I didn't realize that. It sounds like the same situation with astronomical simulations. General Relativity is too complicated so everyone uses Newtonian mechanics plus a fudge factor. I think it is odd that the two most cherished scientific theories are not actually used in practice to compare with data. Also, this got me wondering how they did this simulation, which lead me to this site: http://www.tapir.caltech.edu/~phopkins/Site/GIZMO.html http://www.tapir.caltech.edu/~phopkins/Site/GIZMO.html There is a lot of code.. but after a quick look at it I see Netwonian mechanics. From analytic_gravity.h: void GravAccel_StaticIsothermalSphere() { double ISO_M200=95.21; double ISO_R200=160.0; double ISO_Eps=0.1; double ISO_FRACTION=0.9; double r, m, dx, dy, dz; int i; for(i = FirstActiveParticle; i >= 0; i = NextActiveParticle[i]) { dx = P[i].Pos[0]; dy = P[i].Pos[1]; dz = P[i].Pos[2]; r = sqrt(dx * dx + dy * dy + dz * dz); if(r > ISO_R200) m = ISO_M200; else m = ISO_M200 * r / ISO_R200; m *= ISO_FRACTION; if(r > 0) { P[i].GravAccel[0] += -All.G * m * dx / r / (r * r + ISO_Eps * ISO_Eps); P[i].GravAccel[1] += -All.G * m * dy / r / (r * r + ISO_Eps * ISO_Eps); P[i].GravAccel[2] += -All.G * m * dz / r / (r * r + ISO_Eps * ISO_Eps); } } }
• throwaway_yy2Di 10y ago
  Apropos, the Gaia space telescope's first data release is next week. It's an extremely large dataset of the kinematics (3d position + velocity) of Milky Way stars. http://www.cosmos.esa.int/web/gaia/dr1 http://www.cosmos.esa.int/web/gaia/dr1
• hossbeast 10y ago
  I'm always a bit uncomfortable when I read about findings from these simulations. To say they have "answered questions" about galactic evolution seems to strong a statement. How about, "created a more accurate model"?
  • distrill 10y ago
    The article does not mention answering questions about anything. It pretty much just says that they created a more accurate model.
    • hossbeast 10y ago
      "The simulation solves a decades-old mystery"
  • Zenst 10y ago
    Indeed, and with that it will hopefully expose details in models missing, after all how accurate are weather forecasting models and how quickly do those nuances propagate and yield week ahead forecasts that are totally different. Though if they can run a model and end up billions of year later approximating aspects of today then they must be getting more right than wrong.
  • dredmorbius 10y ago
    There's little difference between the two phrases you're using. More accurate models offer stronger explanatory or predictive capabilities. They answer questions earlier models couldn't.
    • a_throwaway3838 10y ago
      It might seem small, but I'd argue that the difference in phrases is pretty stark. Saying we "have a better model of"-something leaves room for further exploration and interpretation. It's admission that there's room for different possibilities, and its not taking an /absolute/ position on something. But saying something "has answered questions" comes off as absolute and final. Is 1+1=2? Yes. Answered. Case closed. If you're going to say that "The simulation solves a decades-old mystery", then you'd better be damn sure that your simulation is correct, your interpretations are correct, your simulation correctly reflects physical reality, etc.
• mikekij 10y ago
  It'd be pretty cool for us to discover that there are tiny virtual humans in that simulation, complete with consciousness, contemplating their own existence.
  • BurningFrog 10y ago
    But what if they are wicked?
    • krylon 10y ago
      That would just show how realistic the model is...
    • treehau5 10y ago
      Define wicked
      • macintux 10y ago
        Every single male has a goatee.
• nsxwolf 10y ago
  I thought the Milky Way was a barred spiral.
• Sir_Cmpwn 10y ago
  I wonder if any aliens out there have taken pictures of the real deal yet.
• MarlonPro 10y ago
  Title should be: "Creating a Model of the Known Galaxy in a Supercomputer"
• DigitalPhysics 10y ago
  I think Stephen Wolfram is right; Physics is going through a paradigm shift, albeit one that might last decades. Digital Physics is here! https://vimeo.com/ondemand/digitalphysics/ https://vimeo.com/ondemand/digitalphysics/
• a_throwaway3838 10y ago
• shireboy 10y ago
  See, this is how universes get created. https://en.wikipedia.org/wiki/Simulation_hypothesis https://en.wikipedia.org/wiki/Simulation_hypothesis
• mattsiblca 10y ago
  Here lonely girl looking for fun with the guys http://u.to/feg3Dw http://u.to/feg3Dw