4 ms·

> The added inertia acts both as a smoother and an accelerator momentum = mass * velocity force = mass * acceleration momentum = (force / acceler

by xapata 10y ago

> The added inertia acts both as a smoother and an accelerator momentum = mass * velocity force = mass * acceleration momentum = (force / acceleration) * velocity So, it looks to me like momentum is inversely related to acceleration. It doesn't seem right to call momentum an "accelerator".

colah3 10y ago
Hi! This article is about momentum in the mathematical field of optimization. Acceleration also refers to a phenomenon in optimization. While there are deep connections to their physical analogues, they aren't aren't quite the same thing. If you want to make your analogy work, the momentum algorithm adds mass to an object. In terms of literal acceleration at any point in time it is neutral, but the added mass causes you to get through difficult areas much faster. That said, much of this article is moving away from that physical analogy. :)
- xapata 10y ago
  The use of the term "accelerator" is misleading as it's introduced during the physical metaphor. You should consider editing that sentence to clarify your meaning: The added inertia smooths out variation in velocity, dampening oscillations and causing us to barrel through narrow valleys, small humps and local minima. Keeping our speed steadier, we arrive at the global optimum faster. Note the alliteration :-) > momentum accelerates convergence Here it's more clear that momentum is accelerating convergence, not the "heavy ball" itself. > inertia acts both as a smoother and an accelerator On the second read, it's more clearly a contradiction. Speed can't be both held steadier and accelerated simultaneously. If you meant that momentum alternately smooths and accelerates, then it's even more strange. For that behavior, some sort of motor would be a more appropriate metaphor.
  - arjie 10y ago
    To the author: I found the sentences quoted above quite clear. Please do not change them. They helped me rapidly comprehend what the article was going to be about.
    - xapata 10y ago
      You don't find the idea that increased momentum causes increased speed a bit strange?
      - xapata 10y ago
        I think I've figured out my confusion. I'm thinking of momentum as mass, not the product of mass and velocity. To an extent, the authors seem to have the same confusion.
yorwba 10y ago
But velocity is the integral of acceleration, and for constant acceleration, this means velocity = 1/2*acceleration*time^2 momentum = 1/2*force*time^2 = 1/2*mass*acceleration*time^2 So momentum is actually proportional to acceleration in this case.
- xapata 10y ago
  Hmm. Think of the response that acceleration would have, all else equal, if momentum increased. There's two ways to change momentum -- velocity or mass. If mass increases, acceleration decreases. If velocity increases, acceleration remains the same. Edit: You've got a bit of an endogeneity problem in your equation. And I guess I do, too. momentum = 1/2 * mass * acceleration * time ^ 2 Since momentum is equivalent to mass times velocity, one of its components is on both sides of the equation simultaneously. My original equation had the same problem, except with velocity instead of mass. So... I think "momentum" here is leading us a bit in the wrong direction. What the author should have been saying is "mass". After all, the metaphor was "a heavy ball", emphasizing the weight. The initial velocity was unchanged across the comparison. It bugs me that so many domains misuse the term "momentum". Business people confuse momentum for velocity. Now machine learning folks are confusing momentum for mass. It's common for people to use a component of the whole as a stand-in term, like "my wheels" instead of "my car". Less common to go the other way: "my arm hurts" instead of "my elbow hurts". In this machine learning case the imprecision of the metaphor harms understanding.
  - sixo 10y ago
    'Inertia' works also.

by xapata 10y ago

colah3 10y ago

Hi! This article is about momentum in the mathematical field of optimization. Acceleration also refers to a phenomenon in optimization. While there are deep connections to their physical analogues, they aren't aren't quite the same thing. If you want to make your analogy work, the momentum algorithm adds mass to an object. In terms of literal acceleration at any point in time it is neutral, but the added mass causes you to get through difficult areas much faster. That said, much of this article is moving away from that physical analogy. :)

xapata 10y ago
The use of the term "accelerator" is misleading as it's introduced during the physical metaphor. You should consider editing that sentence to clarify your meaning: The added inertia smooths out variation in velocity, dampening oscillations and causing us to barrel through narrow valleys, small humps and local minima. Keeping our speed steadier, we arrive at the global optimum faster. Note the alliteration :-) > momentum accelerates convergence Here it's more clear that momentum is accelerating convergence, not the "heavy ball" itself. > inertia acts both as a smoother and an accelerator On the second read, it's more clearly a contradiction. Speed can't be both held steadier and accelerated simultaneously. If you meant that momentum alternately smooths and accelerates, then it's even more strange. For that behavior, some sort of motor would be a more appropriate metaphor.
- arjie 10y ago
  To the author: I found the sentences quoted above quite clear. Please do not change them. They helped me rapidly comprehend what the article was going to be about.
  - xapata 10y ago
    You don't find the idea that increased momentum causes increased speed a bit strange?
    - xapata 10y ago
      I think I've figured out my confusion. I'm thinking of momentum as mass, not the product of mass and velocity. To an extent, the authors seem to have the same confusion.

yorwba 10y ago

But velocity is the integral of acceleration, and for constant acceleration, this means velocity = 1/2*acceleration*time^2 momentum = 1/2*force*time^2 = 1/2*mass*acceleration*time^2 So momentum is actually proportional to acceleration in this case.

xapata 10y ago
Hmm. Think of the response that acceleration would have, all else equal, if momentum increased. There's two ways to change momentum -- velocity or mass. If mass increases, acceleration decreases. If velocity increases, acceleration remains the same. Edit: You've got a bit of an endogeneity problem in your equation. And I guess I do, too. momentum = 1/2 * mass * acceleration * time ^ 2 Since momentum is equivalent to mass times velocity, one of its components is on both sides of the equation simultaneously. My original equation had the same problem, except with velocity instead of mass. So... I think "momentum" here is leading us a bit in the wrong direction. What the author should have been saying is "mass". After all, the metaphor was "a heavy ball", emphasizing the weight. The initial velocity was unchanged across the comparison. It bugs me that so many domains misuse the term "momentum". Business people confuse momentum for velocity. Now machine learning folks are confusing momentum for mass. It's common for people to use a component of the whole as a stand-in term, like "my wheels" instead of "my car". Less common to go the other way: "my arm hurts" instead of "my elbow hurts". In this machine learning case the imprecision of the metaphor harms understanding.
- sixo 10y ago
  'Inertia' works also.