5 ms·

In physics, the conserved quantity isn't always time. Invariance over time translation is specifically conservation of energy. Invariance over spatial transla

In physics, the conserved quantity isn't always time. Invariance over time translation is specifically conservation of energy. Invariance over spatial translation is conservation of momentum, invariance over spatial rotation is conservation of conservation of angular momentum, invariance of electromagnetic field is conservation of current, and invariance of wave function phase is conservation of charge. I think the analogue in machine learning is conservation over changes in the training data. After all, the point of machine learning is to find general models that describe the training data given, and minimize the loss function. Assuming that a useful model can be trained, the whole point is that it generalizes to new, unseen instances with minimal losses, i.e. the model remains invariant under shifts in the instances seen. The more interesting part to me is what this says about philosophy of physics. Noether's Theorem can be restated as "The laws of physics are invariant under X transformation", where X is the gauge symmetry associated with the conservation law. But maybe this is simply a consequence of how we do physics. After all, the point of science is to produce generalized laws from empirical observations. It's trivially easy to find a real-world situation where conservation of energy does not hold (any system with friction, which is basically all of them), but the math gets very messy if you try to actually model the real data, so we rely on approximations that are close enough most of the time. And if many people take empirical measurements at many different points in space, and time, and orientations, you get generalized laws that hold regardless of where/when/who takes the measurement. Machine learning could be viewed as doing science on empirically measurable social quantities. It won't always be accurate, as individual machine-learning fails show. But it's accurate enough that it can provide useful models for civilization-scale quantities.

sdenton4 3y ago
A nice way to formulate (most) data augmentations is: a family of functions A = {a} such that our optimized neural network f obeys f(x) ~= f(a(x)). So in this case, we're explicitly defining the set of desired invariances.
Aardwolf 3y ago
Is there any way to deduce which invariance gives which conservation? I mean for example: how can you tell that time invariance is the one paired with conservation of energy? Why is e.g. time invariance not paired with momentum, current, or anything else, but specifically energy? I know that I can remember momentum is paired with translation simply because there's both the angular momentum and the non-angular momentum one and in space you have translation and rotation, so for time energy is the only one that's left over, but I'm not looking for a trick to remember it, I'm looking for the fundamental reason, as well as how to tell what will be paired with some invariance when looking at some other new invariance
- chunky1994 3y ago
  The conserved quantity is derived from Noether's theorem itself. One thing that is a bit hairy is that Noether's theorem only applies to a continuous, smooth (physical -> there is some wiggle room here) space. When deriving the conservation of energy from Noether's theorem you basically say that your Lagrangian (which is just a set of equations that describes a physical system) is invariant over time. When you do that you automatically get that energy is conserved. Each invariant produces a conserved quantity as explained in parent comment when you apple a specific transformation that is supposed to not change the system (i.e remain invariant). Now in doing this you're also invoking the principle of least action (by using Lagrangians to describe the state of a physical system) but that is a separate topic.
- Cleonis 3y ago
  In retrospect: the earliest recognition of a conserved quantity was Kepler's law of areas. Isaac Newton later showed that Kepler's law of areas is a specific instance of a property that obtains for any central force, not just the (inverse square) law of gravity. About symmetry under change of orientation: for a given (spherically symmetric) source of gravitational interaction the amount of gravitational force is the same in any orientation. For orbital motion the motion is in a plane, so for the case of orbital motion the relevant symmetry is cilindrical symmetry with respect to the plane of the orbit. The very first derivation that is presented in Newton's Principia is a derivation that shows that for any central force we have: in equal intervals of time equal amounts of area are swept out. (The swept out area is proportional to the angular momentum of the orbiting object. That is, the area law anticipated the principle of conservation of angular momentum) A discussion of Newton's derivation, illustrated with diagrams, is available on my website: http://cleonis.nl/physics/phys256/angular_momentum.php http://cleonis.nl/physics/phys256/angular_momentum.php The thrust of the derivation is that if the force that the motion is subject to is a central force (cilindrical symmetry) then angular momentum is conserved. So: In retrospect we see that Newton's demonstration of the area law is an instance of symmetry-and-conserved-quantity-relation being used. Symmetry of a force under change of orientation has as corresponding conserved quantity of the resulting (orbiting) motion: conservation of angular momentum. About conservation laws: The law of conservation of angular momentum and the law of conservation of momentum are about quantities that are associated with specific spatial characteristics, and the conserved quantity is conserved over time. I'm actually not sure about the reason(s) for classification of conservation of energy. My own view: we have that kinetic energy is not associated with any form of keeping track of orientation; the velocity vector is squared, and that squaring operation discards directional information. More generally, Energy is not associated with any spatial characteristic. Arguably Energy conservation is categorized as associated with symmetry under time translation because of absence of association with any spatial characteristic.
jedbrown 3y ago
> It's trivially easy to find a real-world situation where conservation of energy does not hold (any system with friction, which is basically all of them) Conservation of energy absolutely still holds, but entropy is not conserved so the process is irreversible. If your model doesn't include heat, then discrete energy won't be conserved in a process that produces heat, but that's your modeling choice, not a statement about physics. It is common to model such processes using a dissipation potential.
- nostrademons 3y ago
  Right, but I'm saying that it's all modeling choices, all the way down. Extend the model to include thermal energy and most of the time it holds again - but then it falls down if you also have static electricity that generates a visible spark (say, a wool sweater on a slide) or magnetic drag (say, regenerative braking on a car). Then you can include models for those too, but you're introducing new concepts with each, and the math gets much hairier. We call the unified model where we abstract away all the different forms of energy "conservation of energy", but there are a good many practical systems where making tangible predictions using conservation of energy gives wrong answers. Basically this is a restatement of Box's Aphorism ("All models are wrong, but some are useful") or the ideas in Thomas Kuhn's "The Structure of Scientific Revolutions". The goal of science is to from concrete observations to abstract principles which ideally will accurately predict the value of future concrete observations. In many cases, you can do this. But not all. There is always messy data that doesn't fit into neat, simple, general laws. Usually the messy data is just ignored, because it can't be predicted and is assumed to average out or generally be irrelevant in the end. But sometimes the messy outliers bite you, or someone comes up with a new way to handle them elegantly, and then you get a paradigm shift. And this has implications for understanding what machine learning is or why it's important. Few people would think that a model linking background color to likeliness to click on ads is a fundamental physical quality, but Google had one 15+ years ago, and it was pretty accurate, and made them a bunch of money. Or similarly, most people wouldn't think of a model of the English language as being a fundamental physical quality, but that's exactly what an LLM is, and they're pretty useful too.
  - 3y ago
chunky1994 3y ago
I'm a bit skeptical to give up conservation of energy in a system with friction. Isn't it more accurate to say that if we were to calculate every specific interaction we'd still end up having conservation of energy. Now whether or not we're dealing with a closed system etc becomes important but if we were to able to truly model the entire physical system with friction, we'd still adhere to our conservation laws. So they are not approximations, but are just terribly difficult calculations, no? Maybe I'm misunderstanding your point, but this should be true regardless of our philosophy of physics correct?
- nyrikki 3y ago
  It is an analogy stating that dissipative systems do not have a Lagrangian, Noether's work applies to Lagrangian systems Conservation laws in particular are measurable properties of an isolated physical system do not change as the system evolves over time. It is important to remember that Physics is about finding useful models that make useful predictions about a system. So it is important to not confuse the map for the territory. Gibbs free energy and Helmholtz free energy are not conserved. As thermodynamics, entropy, and entropy are difficult topics due to didactic half-truths, here is a paper that shows that the nbody problem becomes invariant and may be undecidable due to what is a similar issue (in a contrived fashion) http://philsci-archive.pitt.edu/13175/ http://philsci-archive.pitt.edu/13175/ While Noether's principle often allows you to see things that can often be simplified in an equation, often it allows you to not just simplify 'terribly difficult calculations' but to actually find computationally possible calculations.
empath-nirvana 3y ago
> In physics, the conserved quantity isn't always time. Invariance over time translation is specifically conservation of energy. That's not what i meant. When you talk about "conservation of angular momentum", the symmetry is invariance over rotation, but the angular momentum is conserved _over time_.

sdenton4 3y ago

A nice way to formulate (most) data augmentations is: a family of functions A = {a} such that our optimized neural network f obeys f(x) ~= f(a(x)). So in this case, we're explicitly defining the set of desired invariances.

Aardwolf 3y ago

Is there any way to deduce which invariance gives which conservation? I mean for example: how can you tell that time invariance is the one paired with conservation of energy? Why is e.g. time invariance not paired with momentum, current, or anything else, but specifically energy? I know that I can remember momentum is paired with translation simply because there's both the angular momentum and the non-angular momentum one and in space you have translation and rotation, so for time energy is the only one that's left over, but I'm not looking for a trick to remember it, I'm looking for the fundamental reason, as well as how to tell what will be paired with some invariance when looking at some other new invariance

chunky1994 3y ago
The conserved quantity is derived from Noether's theorem itself. One thing that is a bit hairy is that Noether's theorem only applies to a continuous, smooth (physical -> there is some wiggle room here) space. When deriving the conservation of energy from Noether's theorem you basically say that your Lagrangian (which is just a set of equations that describes a physical system) is invariant over time. When you do that you automatically get that energy is conserved. Each invariant produces a conserved quantity as explained in parent comment when you apple a specific transformation that is supposed to not change the system (i.e remain invariant). Now in doing this you're also invoking the principle of least action (by using Lagrangians to describe the state of a physical system) but that is a separate topic.
Cleonis 3y ago
In retrospect: the earliest recognition of a conserved quantity was Kepler's law of areas. Isaac Newton later showed that Kepler's law of areas is a specific instance of a property that obtains for any central force, not just the (inverse square) law of gravity. About symmetry under change of orientation: for a given (spherically symmetric) source of gravitational interaction the amount of gravitational force is the same in any orientation. For orbital motion the motion is in a plane, so for the case of orbital motion the relevant symmetry is cilindrical symmetry with respect to the plane of the orbit. The very first derivation that is presented in Newton's Principia is a derivation that shows that for any central force we have: in equal intervals of time equal amounts of area are swept out. (The swept out area is proportional to the angular momentum of the orbiting object. That is, the area law anticipated the principle of conservation of angular momentum) A discussion of Newton's derivation, illustrated with diagrams, is available on my website: http://cleonis.nl/physics/phys256/angular_momentum.php http://cleonis.nl/physics/phys256/angular_momentum.php The thrust of the derivation is that if the force that the motion is subject to is a central force (cilindrical symmetry) then angular momentum is conserved. So: In retrospect we see that Newton's demonstration of the area law is an instance of symmetry-and-conserved-quantity-relation being used. Symmetry of a force under change of orientation has as corresponding conserved quantity of the resulting (orbiting) motion: conservation of angular momentum. About conservation laws: The law of conservation of angular momentum and the law of conservation of momentum are about quantities that are associated with specific spatial characteristics, and the conserved quantity is conserved over time. I'm actually not sure about the reason(s) for classification of conservation of energy. My own view: we have that kinetic energy is not associated with any form of keeping track of orientation; the velocity vector is squared, and that squaring operation discards directional information. More generally, Energy is not associated with any spatial characteristic. Arguably Energy conservation is categorized as associated with symmetry under time translation because of absence of association with any spatial characteristic.

jedbrown 3y ago

> It's trivially easy to find a real-world situation where conservation of energy does not hold (any system with friction, which is basically all of them) Conservation of energy absolutely still holds, but entropy is not conserved so the process is irreversible. If your model doesn't include heat, then discrete energy won't be conserved in a process that produces heat, but that's your modeling choice, not a statement about physics. It is common to model such processes using a dissipation potential.

nostrademons 3y ago
Right, but I'm saying that it's all modeling choices, all the way down. Extend the model to include thermal energy and most of the time it holds again - but then it falls down if you also have static electricity that generates a visible spark (say, a wool sweater on a slide) or magnetic drag (say, regenerative braking on a car). Then you can include models for those too, but you're introducing new concepts with each, and the math gets much hairier. We call the unified model where we abstract away all the different forms of energy "conservation of energy", but there are a good many practical systems where making tangible predictions using conservation of energy gives wrong answers. Basically this is a restatement of Box's Aphorism ("All models are wrong, but some are useful") or the ideas in Thomas Kuhn's "The Structure of Scientific Revolutions". The goal of science is to from concrete observations to abstract principles which ideally will accurately predict the value of future concrete observations. In many cases, you can do this. But not all. There is always messy data that doesn't fit into neat, simple, general laws. Usually the messy data is just ignored, because it can't be predicted and is assumed to average out or generally be irrelevant in the end. But sometimes the messy outliers bite you, or someone comes up with a new way to handle them elegantly, and then you get a paradigm shift. And this has implications for understanding what machine learning is or why it's important. Few people would think that a model linking background color to likeliness to click on ads is a fundamental physical quality, but Google had one 15+ years ago, and it was pretty accurate, and made them a bunch of money. Or similarly, most people wouldn't think of a model of the English language as being a fundamental physical quality, but that's exactly what an LLM is, and they're pretty useful too.
- 3y ago

chunky1994 3y ago

I'm a bit skeptical to give up conservation of energy in a system with friction. Isn't it more accurate to say that if we were to calculate every specific interaction we'd still end up having conservation of energy. Now whether or not we're dealing with a closed system etc becomes important but if we were to able to truly model the entire physical system with friction, we'd still adhere to our conservation laws. So they are not approximations, but are just terribly difficult calculations, no? Maybe I'm misunderstanding your point, but this should be true regardless of our philosophy of physics correct?

nyrikki 3y ago
It is an analogy stating that dissipative systems do not have a Lagrangian, Noether's work applies to Lagrangian systems Conservation laws in particular are measurable properties of an isolated physical system do not change as the system evolves over time. It is important to remember that Physics is about finding useful models that make useful predictions about a system. So it is important to not confuse the map for the territory. Gibbs free energy and Helmholtz free energy are not conserved. As thermodynamics, entropy, and entropy are difficult topics due to didactic half-truths, here is a paper that shows that the nbody problem becomes invariant and may be undecidable due to what is a similar issue (in a contrived fashion) http://philsci-archive.pitt.edu/13175/ http://philsci-archive.pitt.edu/13175/ While Noether's principle often allows you to see things that can often be simplified in an equation, often it allows you to not just simplify 'terribly difficult calculations' but to actually find computationally possible calculations.

empath-nirvana 3y ago

> In physics, the conserved quantity isn't always time. Invariance over time translation is specifically conservation of energy. That's not what i meant. When you talk about "conservation of angular momentum", the symmetry is invariance over rotation, but the angular momentum is conserved _over time_.