12 ms·

The cosmological constant is physics’ most embarrassing problem

79 points by samizdis 6y ago

politelemon 6y ago
I've read many accounts (in popular-science articles and books, so never at source) of Einstein's "blunder" the cosmological constant. Was it really a case of him second-guessing himself as it's often portrayed, or was there more history and thinking behind him dropping it?
- mhh__ 6y ago
  Although the "history" in physics pedagogy is often made up, IIRC it is just a case of him trying to enforce a static universe.
  - pa7x1 6y ago
    He added it to enforce a static universe, which was the predominant viewpoint at the time (more rooted on theology than experimental evidence). Then he dropped it and called it "his most embarrassing mistake" when Hubble showed the expansion of the universe. Finally, our current models put it back in to explain the accelerated expansion of the universe. Einstein definitely made a mistake but his mistake was not adding the cosmological constant, it was adding it for the wrong reasons. He should have added it as a free parameter to be fitted against empirical data.
    - mhh__ 6y ago
      Think of the accolades if he'd correctly predicted expansion
      - kreeben 6y ago
        To this day we would still be talking about him.
      - jfengel 6y ago
        The problem is that, given relativity, he could have equally predicted collapse. There was really no reason to think it would go either way. If he'd said "the universe therefore must be expanding" he'd have been unjustified.
    - raattgift 6y ago
      It wasn't theology at all. It's just that he was equipped with a less rigorous analogue of the Raychaudhuri equation and an intuition about the focusing theorems one could extract from it. In short: a universe with reasonable distributions of matter in it (galaxies) would collapse to a point eventually. Since the Earth was billions of years old (with improved estimates growing in age), the universe must be older still, yet it had not collapsed. Either there was a lot less matter in the universe than there appeared (and again, better observations kept finding more "nebulae" and then galaxies) or something acted against the tendency to collapse. A simple scalar multiplier on the metric tensor was a straightforward and simple way to proxy for the latter. "Static" in this context simply meant "non-collapsing over billions of years", with the value depending on the averaged matter-density. Later he and de Sitter discovered that the multiplier could be set to zero (and consequently omitted from the field equations) while allowing an arbitrarily long -- eternal, even -- expansion of a hot big bang universe, even for arbitrarily large average matter-densities. It wasn't until decades after his death that better observations demanded the return of the small positive cosmological constant. This time it wasn't to prevent total collapse, but rather to account for the gravitational drag of early galaxy clusters on each other not slowing the Einstein-de Sitter expansion (and even later speeding the expansion up). The cosmological constant term was always a free parameter. It's just that there was no point in writing it down if it was measured as zero.
- simonh 6y ago
  He dropped it when Edwin Hubble showed that the galaxies are moving away from us and that the universe is expanding. At that point the notion of a constant value that imposes a static universe became untenable. Note that it wasn't so much that the constant as such became untenable, but if the purpose of introducing it is to create a static situation and the situation is showed not to be static, then the reason for selecting the given value goes away. If you no longer have a reason to set any given value, you have to consider whether you need it at all.
- raattgift 6y ago
  Several people -- and the article at the top -- talk about a "static" universe. What this means is that the universe doesn't collapse into a point, even though it is filled with galaxies. Something had to offset the long-term tendency of masses to gravitate towards one another, and he introduced the most simple mathematical object to represent a plausible "something". When it became apparent that distant quasars were moving away from our galaxy, he recognized that simple inertia from forces exerted on quasars in the distant past could push an eventual total collapse arbitrarily far into the future. This meant that the cosmological constant could be set to zero without observational conflict, and so he simply removed it as superfluous. He and a collaborator formulated the Einstein-de Sitter universe[1], which is a model of a universe that expands forever with zero cosmological constant. The model remains useful in teaching cosmology, as it is a good representation for the history of the universe back to the formation of the first galaxies to today. Observations of the Cosmic Microwave Background after his death raised difficulties for the Einstein-de Sitter model, but it wasn't until the accelerating expansion of the universe was detected in the late 1990s that it became necessary to add a non-zero cosmological constant back in. In essence, if an Einstein-de Sitter like universe had galaxy clusters flying apart at some threshold speed they could still fly apart for arbitrary zillions of years, however they would still exert a drag on each other, particularly early on when they are relatively close together. No such drag can be found, so observations do not require explanations more complicated than a small positive Cosmological Constant on an otherwise Einstein-de Sitter universe (and most such more complicated explanations predict things we should see but don't, or mispredict things we do see). I don't know why he called it a blunder. That seems overly dramatic. He thought evidence compelled an adjustment to his original field equations to avoid the universe collapsing; later observations made him think that adjustment was unnecessary, so he removed it. He died before even later observations made everyone else working in physical cosmology think that first adjustment was necessary after all, to avoid the universe's expansion slowing down at early times more than it did. That happens a lot to all sorts of physicists if they work in a particular area long enough, and Einstein himself proposed, modified, and abandoned numerous other models and theories over the course of his long career. - -- [1] encyclopaedic and reasonable, rather than overwhelmingly technical : https://en.wikipedia.org/wiki/Einstein%E2%80%93de_Sitter_universe https://en.wikipedia.org/wiki/Einstein%E2%80%93de_Sitter_uni...
ridaj 6y ago
"The universe is under no obligation to make sense to you" — Neil DeGrasse Tyson IMO the most embarrassing problem in physics is people pushing non-falsifiable, unscientific theories (multiverse). The fact that some futzing constant needs or doesn't need to be added in order to bring a model in line with empirical observation, in comparison, is quite benign.
- JimBlackwood 6y ago
  What you said is very true, it doesn’t apply to the cosmological constant though. It is not so much that we need it for a model to be in line with empirical observation, it’s that empirical observation gives one value and our models predict another value. The discrepancy is big and if we can’t make our models work empirical observations, that is really embarrassing. Regarding non-falsifiable theories; there’s not a physicist that thinks theories should not be tested. A lot of work is done to make these theories testable. Unfortunately, they are often only testsble on open problems since the limit of these theories just reduces to GR for instance.
- jMyles 6y ago
  > IMO the most embarrassing problem in physics is people pushing non-falsifiable, unscientific theories (multiverse). Is this a not-so-subtle dig at Beacham and other experimentalists who make room for multiverse? Or do I jump to the wrong conclusion? Two questions for you: 1) Does 'falsifiable' in this context per se mean that there needs to be some particle physics experiment that can be designed to test the theory? Is there no room for theoretical physics? Is Rovelli, for example, per se barking up the wrong tree by this measure? 2) Aren't these theories reasonably falsifiable insofar as they'll eventually result in experiments designed to map out the possible domains of the space? eg, what if we build a much higher energy collider and finally find real evidence of the super-symmetry particles?
  - lmm 6y ago
    > 2) Aren't these theories reasonably falsifiable insofar as they'll eventually result in experiments designed to map out the possible domains of the space? eg, what if we build a much higher energy collider and finally find real evidence of the super-symmetry particles? Would you treat not finding super-symmetry particles as falsifying those theories? AIUI those theories admit the possibility of such particles but also the possibility of such particles not existing, so they're not actually falsifiable that way.
sanxiyn 6y ago
I am not sure why the anthropic principle is not accepted as a solution to this problem. The principle is obviously true, and the problem is solved by the application of the principle.
- akvadrako 6y ago
  Because it's too powerful and can be used to explain almost anything. A good theory should make specific and rigid predictions with few free parameters.
- physicsguy 6y ago
  Because in science we tend to use "Occam's razor" approach - we don't make assumptions that are more than necessary to explain the behaviour of something. Most adherents of the anthropic principle assumes "Oh hey, we probably live in a multiverse" to justify it, but we have no strong evidence that that is true. So making that assumption is difficult to justify. It may be true! But we don't know it is, and until it can be proved, the theory rests on shaky ground. It might be that another, simpler theory can explain the same behaviour. Should we assume that a simpler theory is true? No, not necessarily. But it lies on the adherents of a more complex theory to come up with reasons why that theory is right, by making predictions that can be tested.
- whatisthiseven 6y ago
  As an analogy, I think everyone here would be quite upset if someone made the claim, "why shouldn't we trust Musk, Gates, Jobs, etc at their word when they say what caused them success? Obviously it is true because they are successful"! If you reject the premise I proposed, but not the anthropic principle, then you have just moved the goal posts on when to accept blind tautologies without further examination.
  - GoblinSlayer 6y ago
    What further examination can help reject the anthropic principle?
jhoechtl 6y ago
I recently raised a question in the Physics Stackexchange about cosmological constants, especially the fine structure constant (which is already said to be not-so constant) and the relation to gravitaional force: https://physics.stackexchange.com/questions/603669/are-gravitation-and-the-fine-structure-constant-related/603688#603688 https://physics.stackexchange.com/questions/603669/are-gravi...
dr-detroit 6y ago
red shift/blue misunderstandings and a broken model of an expanding universe are what our time period will be mocked for
ike77 6y ago
If the vacuum is producing a force, why would it push outward from us and not in every directions, including toward us, canceling itself out in the process?
- cambalache 6y ago
  Same way as the pressure only pushes "inward". The sign is reversed.
  - ike77 6y ago
    I don't get it? If there is vacuum everywhere: vacuum --- us --- vacuum --- galaxy --- vacuum Why would galaxy go away from us? I expect that the vacuum on the left of the galaxy applies the same force on it as the vacuum on its right and thus should have a null net effect on its movement?
    - cambalache 6y ago
      Imagine that space (well space-time) it is a single elastic band and you pull it by both ends, you will have a more or less uniform tension in the middle part of the band and yet everything is receding from you. Same principle here. Using your same diagram vacuum <---- us -----> <----vacuum----> <-----galaxy---->
- raattgift 6y ago
  The expanding vacuum doesn't produce a force; galaxy clusters embedded in the expanding vacuum are completely inertial. The expansion is called a metric expansion because if we substitute galaxy clusters for isolated human-like observers, those observers will not find a measure of distance between them that is constant on arbitrary systems of coordinates. (Cosmologists like to use particular systems of coordinates which are comoving with the expansion to capture the fact that no forces push these clusters off their inertial motion. However, there are arbitrarily many systems of coordinates which are not comoving with the expansion, and switching to any of those means that your coordinate distance to faraway clusters of galaxies isn't constant. In some of those systems of coordinates it is easy to be misled into seeing https://en.wikipedia.org/wiki/Fictitious_force https://en.wikipedia.org/wiki/Fictitious_force . Again, a change of systems of coordinates, especially to comoving ones, can make those frame-dependent forces vanish.) One can also uses systems of coordinates to treat the metric expansion of space as the increase over time of a gravitational potential measured outside a galaxy cluster. The metric expansion means that over time an object has further to fall from outside a galaxy cluster to one of the supermassive black holes inside the cluster, with related observables like an increase in the gravitational redshift of light climbing out of the cluster. The same extra-galactic observer sees an increase in its gravitational potential relative to every galaxy cluster in the universe, because the radial distance (in spherical coordinates) to all those galaxy clusters increases over time. We can substitute an entire galaxy cluster for our isolated human-like observer without substantially altering the picture I'm trying to paint above.
posix_me_less 6y ago
“I think most people in the cosmology and astrophysics community believe it's a problem because they've been told that for a long time.” This. Quantum theory does not necessarily predict big cosmological constant. All big terms can be cancelled / regularized by some modification of the theory that keeps the results intact. The only problem here is that there is no agreement in how to properly quantize fields and some people think zero point fluctuations are real and necessary to explain some phenomena. However others point out there isn't single rock-solid example of such a need. Not even Casimir effect needs vacuum fluctuations to be explained. But these facts didn't penetrate into mainstream cosmology yet.

79 points by samizdis 6y ago

I've read many accounts (in popular-science articles and books, so never at source) of Einstein's "blunder" the cosmological constant. Was it really a case of him second-guessing himself as it's often portrayed, or was there more history and thinking behind him dropping it?

mhh__ 6y ago
Although the "history" in physics pedagogy is often made up, IIRC it is just a case of him trying to enforce a static universe.
- pa7x1 6y ago
  He added it to enforce a static universe, which was the predominant viewpoint at the time (more rooted on theology than experimental evidence). Then he dropped it and called it "his most embarrassing mistake" when Hubble showed the expansion of the universe. Finally, our current models put it back in to explain the accelerated expansion of the universe. Einstein definitely made a mistake but his mistake was not adding the cosmological constant, it was adding it for the wrong reasons. He should have added it as a free parameter to be fitted against empirical data.
  - mhh__ 6y ago
    Think of the accolades if he'd correctly predicted expansion
    - kreeben 6y ago
      To this day we would still be talking about him.
    - jfengel 6y ago
      The problem is that, given relativity, he could have equally predicted collapse. There was really no reason to think it would go either way. If he'd said "the universe therefore must be expanding" he'd have been unjustified.
  - raattgift 6y ago
    It wasn't theology at all. It's just that he was equipped with a less rigorous analogue of the Raychaudhuri equation and an intuition about the focusing theorems one could extract from it. In short: a universe with reasonable distributions of matter in it (galaxies) would collapse to a point eventually. Since the Earth was billions of years old (with improved estimates growing in age), the universe must be older still, yet it had not collapsed. Either there was a lot less matter in the universe than there appeared (and again, better observations kept finding more "nebulae" and then galaxies) or something acted against the tendency to collapse. A simple scalar multiplier on the metric tensor was a straightforward and simple way to proxy for the latter. "Static" in this context simply meant "non-collapsing over billions of years", with the value depending on the averaged matter-density. Later he and de Sitter discovered that the multiplier could be set to zero (and consequently omitted from the field equations) while allowing an arbitrarily long -- eternal, even -- expansion of a hot big bang universe, even for arbitrarily large average matter-densities. It wasn't until decades after his death that better observations demanded the return of the small positive cosmological constant. This time it wasn't to prevent total collapse, but rather to account for the gravitational drag of early galaxy clusters on each other not slowing the Einstein-de Sitter expansion (and even later speeding the expansion up). The cosmological constant term was always a free parameter. It's just that there was no point in writing it down if it was measured as zero.
simonh 6y ago
He dropped it when Edwin Hubble showed that the galaxies are moving away from us and that the universe is expanding. At that point the notion of a constant value that imposes a static universe became untenable. Note that it wasn't so much that the constant as such became untenable, but if the purpose of introducing it is to create a static situation and the situation is showed not to be static, then the reason for selecting the given value goes away. If you no longer have a reason to set any given value, you have to consider whether you need it at all.
raattgift 6y ago
Several people -- and the article at the top -- talk about a "static" universe. What this means is that the universe doesn't collapse into a point, even though it is filled with galaxies. Something had to offset the long-term tendency of masses to gravitate towards one another, and he introduced the most simple mathematical object to represent a plausible "something". When it became apparent that distant quasars were moving away from our galaxy, he recognized that simple inertia from forces exerted on quasars in the distant past could push an eventual total collapse arbitrarily far into the future. This meant that the cosmological constant could be set to zero without observational conflict, and so he simply removed it as superfluous. He and a collaborator formulated the Einstein-de Sitter universe[1], which is a model of a universe that expands forever with zero cosmological constant. The model remains useful in teaching cosmology, as it is a good representation for the history of the universe back to the formation of the first galaxies to today. Observations of the Cosmic Microwave Background after his death raised difficulties for the Einstein-de Sitter model, but it wasn't until the accelerating expansion of the universe was detected in the late 1990s that it became necessary to add a non-zero cosmological constant back in. In essence, if an Einstein-de Sitter like universe had galaxy clusters flying apart at some threshold speed they could still fly apart for arbitrary zillions of years, however they would still exert a drag on each other, particularly early on when they are relatively close together. No such drag can be found, so observations do not require explanations more complicated than a small positive Cosmological Constant on an otherwise Einstein-de Sitter universe (and most such more complicated explanations predict things we should see but don't, or mispredict things we do see). I don't know why he called it a blunder. That seems overly dramatic. He thought evidence compelled an adjustment to his original field equations to avoid the universe collapsing; later observations made him think that adjustment was unnecessary, so he removed it. He died before even later observations made everyone else working in physical cosmology think that first adjustment was necessary after all, to avoid the universe's expansion slowing down at early times more than it did. That happens a lot to all sorts of physicists if they work in a particular area long enough, and Einstein himself proposed, modified, and abandoned numerous other models and theories over the course of his long career. - -- [1] encyclopaedic and reasonable, rather than overwhelmingly technical : https://en.wikipedia.org/wiki/Einstein%E2%80%93de_Sitter_universe https://en.wikipedia.org/wiki/Einstein%E2%80%93de_Sitter_uni...

ridaj 6y ago

"The universe is under no obligation to make sense to you" — Neil DeGrasse Tyson IMO the most embarrassing problem in physics is people pushing non-falsifiable, unscientific theories (multiverse). The fact that some futzing constant needs or doesn't need to be added in order to bring a model in line with empirical observation, in comparison, is quite benign.

JimBlackwood 6y ago
What you said is very true, it doesn’t apply to the cosmological constant though. It is not so much that we need it for a model to be in line with empirical observation, it’s that empirical observation gives one value and our models predict another value. The discrepancy is big and if we can’t make our models work empirical observations, that is really embarrassing. Regarding non-falsifiable theories; there’s not a physicist that thinks theories should not be tested. A lot of work is done to make these theories testable. Unfortunately, they are often only testsble on open problems since the limit of these theories just reduces to GR for instance.
jMyles 6y ago
> IMO the most embarrassing problem in physics is people pushing non-falsifiable, unscientific theories (multiverse). Is this a not-so-subtle dig at Beacham and other experimentalists who make room for multiverse? Or do I jump to the wrong conclusion? Two questions for you: 1) Does 'falsifiable' in this context per se mean that there needs to be some particle physics experiment that can be designed to test the theory? Is there no room for theoretical physics? Is Rovelli, for example, per se barking up the wrong tree by this measure? 2) Aren't these theories reasonably falsifiable insofar as they'll eventually result in experiments designed to map out the possible domains of the space? eg, what if we build a much higher energy collider and finally find real evidence of the super-symmetry particles?
- lmm 6y ago
  > 2) Aren't these theories reasonably falsifiable insofar as they'll eventually result in experiments designed to map out the possible domains of the space? eg, what if we build a much higher energy collider and finally find real evidence of the super-symmetry particles? Would you treat not finding super-symmetry particles as falsifying those theories? AIUI those theories admit the possibility of such particles but also the possibility of such particles not existing, so they're not actually falsifiable that way.

sanxiyn 6y ago

I am not sure why the anthropic principle is not accepted as a solution to this problem. The principle is obviously true, and the problem is solved by the application of the principle.

akvadrako 6y ago
Because it's too powerful and can be used to explain almost anything. A good theory should make specific and rigid predictions with few free parameters.
physicsguy 6y ago
Because in science we tend to use "Occam's razor" approach - we don't make assumptions that are more than necessary to explain the behaviour of something. Most adherents of the anthropic principle assumes "Oh hey, we probably live in a multiverse" to justify it, but we have no strong evidence that that is true. So making that assumption is difficult to justify. It may be true! But we don't know it is, and until it can be proved, the theory rests on shaky ground. It might be that another, simpler theory can explain the same behaviour. Should we assume that a simpler theory is true? No, not necessarily. But it lies on the adherents of a more complex theory to come up with reasons why that theory is right, by making predictions that can be tested.
whatisthiseven 6y ago
As an analogy, I think everyone here would be quite upset if someone made the claim, "why shouldn't we trust Musk, Gates, Jobs, etc at their word when they say what caused them success? Obviously it is true because they are successful"! If you reject the premise I proposed, but not the anthropic principle, then you have just moved the goal posts on when to accept blind tautologies without further examination.
- GoblinSlayer 6y ago
  What further examination can help reject the anthropic principle?

jhoechtl 6y ago

I recently raised a question in the Physics Stackexchange about cosmological constants, especially the fine structure constant (which is already said to be not-so constant) and the relation to gravitaional force: https://physics.stackexchange.com/questions/603669/are-gravitation-and-the-fine-structure-constant-related/603688#603688 https://physics.stackexchange.com/questions/603669/are-gravi...

dr-detroit 6y ago

red shift/blue misunderstandings and a broken model of an expanding universe are what our time period will be mocked for

ike77 6y ago

If the vacuum is producing a force, why would it push outward from us and not in every directions, including toward us, canceling itself out in the process?

cambalache 6y ago
Same way as the pressure only pushes "inward". The sign is reversed.
- ike77 6y ago
  I don't get it? If there is vacuum everywhere: vacuum --- us --- vacuum --- galaxy --- vacuum Why would galaxy go away from us? I expect that the vacuum on the left of the galaxy applies the same force on it as the vacuum on its right and thus should have a null net effect on its movement?
  - cambalache 6y ago
    Imagine that space (well space-time) it is a single elastic band and you pull it by both ends, you will have a more or less uniform tension in the middle part of the band and yet everything is receding from you. Same principle here. Using your same diagram vacuum <---- us -----> <----vacuum----> <-----galaxy---->
raattgift 6y ago
The expanding vacuum doesn't produce a force; galaxy clusters embedded in the expanding vacuum are completely inertial. The expansion is called a metric expansion because if we substitute galaxy clusters for isolated human-like observers, those observers will not find a measure of distance between them that is constant on arbitrary systems of coordinates. (Cosmologists like to use particular systems of coordinates which are comoving with the expansion to capture the fact that no forces push these clusters off their inertial motion. However, there are arbitrarily many systems of coordinates which are not comoving with the expansion, and switching to any of those means that your coordinate distance to faraway clusters of galaxies isn't constant. In some of those systems of coordinates it is easy to be misled into seeing https://en.wikipedia.org/wiki/Fictitious_force https://en.wikipedia.org/wiki/Fictitious_force . Again, a change of systems of coordinates, especially to comoving ones, can make those frame-dependent forces vanish.) One can also uses systems of coordinates to treat the metric expansion of space as the increase over time of a gravitational potential measured outside a galaxy cluster. The metric expansion means that over time an object has further to fall from outside a galaxy cluster to one of the supermassive black holes inside the cluster, with related observables like an increase in the gravitational redshift of light climbing out of the cluster. The same extra-galactic observer sees an increase in its gravitational potential relative to every galaxy cluster in the universe, because the radial distance (in spherical coordinates) to all those galaxy clusters increases over time. We can substitute an entire galaxy cluster for our isolated human-like observer without substantially altering the picture I'm trying to paint above.

posix_me_less 6y ago

“I think most people in the cosmology and astrophysics community believe it's a problem because they've been told that for a long time.” This. Quantum theory does not necessarily predict big cosmological constant. All big terms can be cancelled / regularized by some modification of the theory that keeps the results intact. The only problem here is that there is no agreement in how to properly quantize fields and some people think zero point fluctuations are real and necessary to explain some phenomena. However others point out there isn't single rock-solid example of such a need. Not even Casimir effect needs vacuum fluctuations to be explained. But these facts didn't penetrate into mainstream cosmology yet.