10 ms·

You are right, however the same way with automatic differentiation, you never get the expression, with automatic integration it would be the same.

laplacesdemon48 6y ago
Can you please elaborate on "you never get the expression"? When you input the variable values into a symbolic derivative you just get a value at the end. d/dx x^2 = 2x. If x = 0.5 then d/dx = 1. The same is true for symbolic integrals. For most practical applications, we don't really care about the full symbolic expression. We just want the answer, or at least a good approximation. This post uses a specific example of the difference between two Beta distributions. We want to get that 0.71. It is very hard to "automatically" make that happen.
- adamnemecek 6y ago
  Are you familiar with automatic differentiation? https://blog.demofox.org/2014/12/30/dual-numbers-automatic-differentiation/ https://blog.demofox.org/2014/12/30/dual-numbers-automatic-d... note that you get f(x) and f'(x) for some x without getting the derived function.
  - justinpombrio 6y ago
    Ooh, that's a neat technique. I think you can get an expression for the derivative from it. Say we want the derivative of f(x) = x^3 - 2x^2 + 5. That becomes: (x+e)^3 - 2(x+e)^2 + 5 = x^3 + 3x^2e - 2x^2 -4xe + 5 = (x^3 - 2x^2 + 5) + (3x^2 - 4x)e The term in front of 'e' is "3x^2 - 4x", which is f'(x).
    - judofyr 6y ago
      Just to clarify: This result follows directly from the definition of the derivative: f'(x) = lim_{e->0} (f(x+e) - f(x))/e If you're able to express f(x+e) on the form (f(x) + y e) then it follows that y is the derivative. It also should be noted that auto-diff doesn't let you skip the rules for derivation and you're using the same calculation as you would to show that e.g. f'(x^n)=n*x^{n-1}.
- foxes 6y ago
  For algebraic expressions you get an expression if you expand in dual numbers. But what about something like cos(x). Either you can lazily evaluate the power series or you know its sin(x).
srean 6y ago
Closed form integration is fundamentally different. What's the integral of exp( sqrt ( 1 + (tan^(3/2) X)2 ) ) ) ? We only know a handful of forms that can be integrated in closed form and its down to our creativity to discover new forms that can be integrated (same deal with solving differential equations and the reasons are the same). The forms that we know how to integrate can be done by a computer. CAS tools will do that for you. For example Mathematica.
- adamnemecek 6y ago
  Are you familiar with automatic differentiation?
  - srean 6y ago
    I am. Try the integration analogue of it on the fairly simple example i gave. The auto-diff of that is a straightforward.
    - adamnemecek 6y ago
      Right. I'm still working out the autoint thing so no. I can't try it right now.
  - judofyr 6y ago
    The reason why auto-diff is exact and amazing is because (1) the derivative operation is "decomposable" through the chain rule and (2) we've found the symbolic derivative of the basic functions. There's not a mathematical trick in auto-diff which we recently discovered. It's mainly a reformulation of the properties of the derivative which turns out to be useful for computers. Finding an "auto-integrate method" would probably involve finding a way of calculating the integral in a decomposable way, and that indeed would be amazing, but I don't really see that happening any time soon.

laplacesdemon48 6y ago

Can you please elaborate on "you never get the expression"? When you input the variable values into a symbolic derivative you just get a value at the end. d/dx x^2 = 2x. If x = 0.5 then d/dx = 1. The same is true for symbolic integrals. For most practical applications, we don't really care about the full symbolic expression. We just want the answer, or at least a good approximation. This post uses a specific example of the difference between two Beta distributions. We want to get that 0.71. It is very hard to "automatically" make that happen.

adamnemecek 6y ago
Are you familiar with automatic differentiation? https://blog.demofox.org/2014/12/30/dual-numbers-automatic-differentiation/ https://blog.demofox.org/2014/12/30/dual-numbers-automatic-d... note that you get f(x) and f'(x) for some x without getting the derived function.
- justinpombrio 6y ago
  Ooh, that's a neat technique. I think you can get an expression for the derivative from it. Say we want the derivative of f(x) = x^3 - 2x^2 + 5. That becomes: (x+e)^3 - 2(x+e)^2 + 5 = x^3 + 3x^2e - 2x^2 -4xe + 5 = (x^3 - 2x^2 + 5) + (3x^2 - 4x)e The term in front of 'e' is "3x^2 - 4x", which is f'(x).
  - judofyr 6y ago
    Just to clarify: This result follows directly from the definition of the derivative: f'(x) = lim_{e->0} (f(x+e) - f(x))/e If you're able to express f(x+e) on the form (f(x) + y e) then it follows that y is the derivative. It also should be noted that auto-diff doesn't let you skip the rules for derivation and you're using the same calculation as you would to show that e.g. f'(x^n)=n*x^{n-1}.
foxes 6y ago
For algebraic expressions you get an expression if you expand in dual numbers. But what about something like cos(x). Either you can lazily evaluate the power series or you know its sin(x).

srean 6y ago

Closed form integration is fundamentally different. What's the integral of exp( sqrt ( 1 + (tan^(3/2) X)2 ) ) ) ? We only know a handful of forms that can be integrated in closed form and its down to our creativity to discover new forms that can be integrated (same deal with solving differential equations and the reasons are the same). The forms that we know how to integrate can be done by a computer. CAS tools will do that for you. For example Mathematica.

adamnemecek 6y ago
Are you familiar with automatic differentiation?
- srean 6y ago
  I am. Try the integration analogue of it on the fairly simple example i gave. The auto-diff of that is a straightforward.
  - adamnemecek 6y ago
    Right. I'm still working out the autoint thing so no. I can't try it right now.
- judofyr 6y ago
  The reason why auto-diff is exact and amazing is because (1) the derivative operation is "decomposable" through the chain rule and (2) we've found the symbolic derivative of the basic functions. There's not a mathematical trick in auto-diff which we recently discovered. It's mainly a reformulation of the properties of the derivative which turns out to be useful for computers. Finding an "auto-integrate method" would probably involve finding a way of calculating the integral in a decomposable way, and that indeed would be amazing, but I don't really see that happening any time soon.