3 ms·

Do one for deltas or gradients. I always have a hard time determining exactly what they apply to.

stevefan1999 5y ago
then you need to know the first principle, sometime aka the definition of slope/rate/change, and limits/infinitesimals, which first principle is lim h->0{ (f(x+h)-f(x))/((x+h)-x) }, but since computers cannot express them exactly -- due to it being a discrete system, computers can only approximate infinitesimals to some extent, if symbolic algebra systems (e.g. SageMath/SymPy) are not involved. You can still try to implement it as a real computer function such as `var dydx = (f: (x) => number, h: number = 0.00000001) => x => (f(x+h)-f(x))/h` by putting an extremely small value of h to get a pretty good result.
dougabug 5y ago
Think of a gradient as a kind of vector slope. In one dimension, the derivative of a function is its slope at any given point; since in this case there is only one direction against which to measure the slope, we would typically consider the derivative of a single variable function to be a scalar rather than a vector. For a multivariable function, the gradient points in the direction of maximum ascent, and its magnitude is the steepness with which the (graph of the) function (which is a surface) changes in that direction.
ColinWright 5y ago
The idea of the gradient of a function is to ask this question: When I jiggle the input, how much does the output change? That can depend on where you're doing it. For f(x)=x^2, jiggling x around 0 results in almost no change at all: f(-0.1) = 0.01 f( 0.0) = 0.00 f( 0.1) = 0.01 But jiggling around a value of 10 makes a much larger difference: f( 9.9) = 98.01 f(10.0) = 100.00 f(10.1) = 102.01 Here, around the value of x=10, jiggling x up and down by 0.1 changes the output by about 2. Experimenting a lot, you can discover that for f(x)=x^2 you will find that when the jiggling (call it j) is small, around a value x, the amount of change is roughly (2x)(j). So roughly: f(x+j) ~ f(x)+2xj Gradient is usually expressed as "Rise over Run". The "Rise" here is the change in f, so it's f(x+j)-f(x), and the "Run" is j, the amount of jiggle, and what this has shown is that for f(x)=x^2, "Rise of Run", the gradient, is (roughly) 2x. You can write all that in code, but those are the ideas, and I hope it answers your question.
- coolgeek 5y ago
  > you can discover that for f(x)=x^2 you will find that when the jiggling (call it j) is small, around a value x, the amount of change is roughly (2x)(j) Are you implying that the change corresponds to the derivative of the function, or is that just a coincidence in this case? (I'm guessing from your characterization of rise over run that the answer is yes. But you know what they say about assumptions.)
  - ColinWright 5y ago
    >> you can discover that for f(x)=x^2 you will find that when the jiggling (call it j) is small, around a value x, the amount of change is roughly (2x)(j) > Are you implying that the change corresponds to the derivative of the function, or is that just a coincidence in this case? When you have a function f(x), the amount that f(x) changes as you jiggle the value x is roughly given by the amount of jiggle times the derivative f'(x) [also known by many other names]. That's more-or-less what "derivative" means in this context. So under appropriate and rather generous conditions: f(x+j) ~ f(x) + j.f'(x) Does that answer your question? I've answered the question I think you asked, but that's not always the the question you intended.
    - coolgeek 5y ago
      I guess the question I was really asking was is the gradient of a function equivalent to the derivative of the function? But a quick look at the wiki page for gradient suggests that my calc I/II intuition of derivatives is insufficient here
      - ColinWright 5y ago
        For reasonable functions f:R->R, yes, the gradient of the graph of the function is the value of the derivative at that point. But "Gradient" is a term that's used in wider contexts, so the real answer is "it depends". But I've had a look at this page: https://en.wikipedia.org/wiki/Gradient https://en.wikipedia.org/wiki/Gradient And that really does say that the "Gradient" is basically the derivative, suitably generalised for larger dimensions.

stevefan1999 5y ago

then you need to know the first principle, sometime aka the definition of slope/rate/change, and limits/infinitesimals, which first principle is lim h->0{ (f(x+h)-f(x))/((x+h)-x) }, but since computers cannot express them exactly -- due to it being a discrete system, computers can only approximate infinitesimals to some extent, if symbolic algebra systems (e.g. SageMath/SymPy) are not involved. You can still try to implement it as a real computer function such as `var dydx = (f: (x) => number, h: number = 0.00000001) => x => (f(x+h)-f(x))/h` by putting an extremely small value of h to get a pretty good result.

dougabug 5y ago

Think of a gradient as a kind of vector slope. In one dimension, the derivative of a function is its slope at any given point; since in this case there is only one direction against which to measure the slope, we would typically consider the derivative of a single variable function to be a scalar rather than a vector. For a multivariable function, the gradient points in the direction of maximum ascent, and its magnitude is the steepness with which the (graph of the) function (which is a surface) changes in that direction.

ColinWright 5y ago

The idea of the gradient of a function is to ask this question: When I jiggle the input, how much does the output change? That can depend on where you're doing it. For f(x)=x^2, jiggling x around 0 results in almost no change at all: f(-0.1) = 0.01 f( 0.0) = 0.00 f( 0.1) = 0.01 But jiggling around a value of 10 makes a much larger difference: f( 9.9) = 98.01 f(10.0) = 100.00 f(10.1) = 102.01 Here, around the value of x=10, jiggling x up and down by 0.1 changes the output by about 2. Experimenting a lot, you can discover that for f(x)=x^2 you will find that when the jiggling (call it j) is small, around a value x, the amount of change is roughly (2x)(j). So roughly: f(x+j) ~ f(x)+2xj Gradient is usually expressed as "Rise over Run". The "Rise" here is the change in f, so it's f(x+j)-f(x), and the "Run" is j, the amount of jiggle, and what this has shown is that for f(x)=x^2, "Rise of Run", the gradient, is (roughly) 2x. You can write all that in code, but those are the ideas, and I hope it answers your question.

coolgeek 5y ago
> you can discover that for f(x)=x^2 you will find that when the jiggling (call it j) is small, around a value x, the amount of change is roughly (2x)(j) Are you implying that the change corresponds to the derivative of the function, or is that just a coincidence in this case? (I'm guessing from your characterization of rise over run that the answer is yes. But you know what they say about assumptions.)
- ColinWright 5y ago
  >> you can discover that for f(x)=x^2 you will find that when the jiggling (call it j) is small, around a value x, the amount of change is roughly (2x)(j) > Are you implying that the change corresponds to the derivative of the function, or is that just a coincidence in this case? When you have a function f(x), the amount that f(x) changes as you jiggle the value x is roughly given by the amount of jiggle times the derivative f'(x) [also known by many other names]. That's more-or-less what "derivative" means in this context. So under appropriate and rather generous conditions: f(x+j) ~ f(x) + j.f'(x) Does that answer your question? I've answered the question I think you asked, but that's not always the the question you intended.
  - coolgeek 5y ago
    I guess the question I was really asking was is the gradient of a function equivalent to the derivative of the function? But a quick look at the wiki page for gradient suggests that my calc I/II intuition of derivatives is insufficient here
    - ColinWright 5y ago
      For reasonable functions f:R->R, yes, the gradient of the graph of the function is the value of the derivative at that point. But "Gradient" is a term that's used in wider contexts, so the real answer is "it depends". But I've had a look at this page: https://en.wikipedia.org/wiki/Gradient https://en.wikipedia.org/wiki/Gradient And that really does say that the "Gradient" is basically the derivative, suitably generalised for larger dimensions.