5 ms·
Higher Order Derivatives of Transforms
- shiandow 3y agoDon't think I'll ever get used to stars in LaTeX when someone means simple multiplication. The preceding blog post[1] seems to contain the more interesting parts though. This is just (d/dt)^n e^At x = A^n e^At x, which is kind of obvious from the definition of e^At. [1]: https://nosferalatu.com/DerivativesLogarithmsTransforms.html https://nosferalatu.com/DerivativesLogarithmsTransforms.html
- cycomanic 3y agoThe article (even more the previous article) really misses a reference to matrix exponentials [1]. Everything about X(t) = X^t becomes a bit clearer. Also I think using the term transform is a bit loose here. IIRC Matrix exponentials only work with linear transforms that are represented by a square matrix. [1] https://en.m.wikipedia.org/wiki/Matrix_exponential https://en.m.wikipedia.org/wiki/Matrix_exponential
- magicalhippo 3y agoThe previous post in the series[1], helpfully linked to in the introduction, has a lot more details of the underlying concepts. It also set my mind wandering to the not-technically-related functional derivatives[2], where you vary the function slightly rather than the argument value. I'm not great at math, but I do love this what-if exploration you can do in math. Due to the various proofs underlying it all it seems sometimes more fruitful than similar exploration in programming, where one might quickly stumble upon obscure compiler errors or similar obstacles. [1]: https://nosferalatu.com/DerivativesLogarithmsTransforms.html https://nosferalatu.com/DerivativesLogarithmsTransforms.html [2]: https://en.wikipedia.org/wiki/Functional_derivative https://en.wikipedia.org/wiki/Functional_derivative
- abhgh 3y agoIf you're looking for interesting derivative-adjacent ideas, I would also recommend Clarke derivatives [1]. They occasionally show up in ML papers, e.g., [2], [3]. Unrelated bu tangential, another place where you need derivatives but don't have access to them (standard or otherwise) is in the area of black-box optimization. Within this area, Bayesian Optimization (BayesOpt) has picked up quite a bit, which I've successfully used quite a bit in my work - I've an introduction here [4]. There is also a good book available online for free on the topic [5]. [1] https://en.wikipedia.org/wiki/Clarke_generalized_derivative https://en.wikipedia.org/wiki/Clarke_generalized_derivative [2] https://proceedings.neurips.cc/paper/2021/file/70afbf2259b4449d8ae1429e054df1b1-Paper.pdf https://proceedings.neurips.cc/paper/2021/file/70afbf2259b44... [3] http://proceedings.mlr.press/v202/lee23p/lee23p.pdf http://proceedings.mlr.press/v202/lee23p/lee23p.pdf [4] https://blog.quipu-strands.com/bayesopt_1_key_ideas_GPs https://blog.quipu-strands.com/bayesopt_1_key_ideas_GPs [5] https://bayesoptbook.com/ https://bayesoptbook.com/
- ogogmad 3y agoFunctional derivative : Frechet derivative :: gradient : directional derivative
- c32c33429009ed6 3y agoWhat is meant by a Transform in this context? The author doesn't seem to give a definition.
- magicalhippo 3y agoThey give a brief statement in their previous post (see link at start of post), essentially it's a linear transformation also known as linear map[1]. [1]: https://en.wikipedia.org/wiki/Linear_map https://en.wikipedia.org/wiki/Linear_map
- c32c33429009ed6 3y agoI read their previous post, and nowhere do they explicitly say "a transform is a...". One might assume that it is indeed a linear transformation, as you suggest, but it shouldn't be up to the reader to do detective work just to understand the objects the author is talking about.
- magicalhippo 3y agoWhat I meant was that based on what they said in their previous post ("[g]iven a transform T and a point x, we can find the transformed point with T∗x") and the interactive graphics, I felt certain they meant a linear map. I agree it's sloppy, at least a reference or something should be given if one doesn't want to spend time on the full definition.
- makerdiety 3y agoMaybe the author could have used a more general notion, then, if omission and brevity were going to be present? Like, instead of a linear map or transform, he could have said an operator or something. I don't know what is the general form of a transform or linear map. I think it's something like operator, though.
- magicalhippo 3y ago
- dimatura 3y agoI like this summary of Lie theory in the context of robotics: https://arxiv.org/pdf/1812.01537.pdf https://arxiv.org/pdf/1812.01537.pdf
- tnecniv 3y agoI like this tutorial because it doesn’t get too bogged down in abstractions and has numerous examples. When I’ve tried to learn differential geometry in the past, standard texts get very abstract very quickly and it’s hard for me to envision what the generalization is doing for me.
- gradschoolfail 3y agoTerence Tao has a book “Higher order Fourier analysis”, can anyone chip in on whether the two are related? Here is a video, it seems to focus on aaplications in number theory, but I cant tell as a layman https://www.youtube.com/watch?v=nr2Xv9v9CZc https://www.youtube.com/watch?v=nr2Xv9v9CZc For example, this is the tradFourier version of what hes trying to do https://en.wikipedia.org/wiki/Hardy–Ramanujan–Littlewood_circle_method https://en.wikipedia.org/wiki/Hardy–Ramanujan–Littlewood_cir...
- shiandow 3y agoThe techniques in this article would be considered common knowledge for any of the prerequisite classes you'd need to take to be able to follow a course on Higher order Fourier analysis. I wouldn't say they're unrelated, but if you want to know whether both uses of the phrase "higher order" have any relation then no.
- gradschoolfail 3y agoThank you! I remember seeing in the Tao book a Fourier operator that looks something like the “integral” of exp(ik^2 x).. (or exp(ik x^2), haha) As a followup, is it possible to ELI5 roughly what Tao meant by “higher-order”?
- coherentpony 3y agoThe first paragraph of the preface on page 7 in the book gives an explicit example of “higher order”: https://terrytao.files.wordpress.com/2011/03/higher-book.pdf https://terrytao.files.wordpress.com/2011/03/higher-book.pdf
- gradschoolfail 3y agoLollol thats what i half remembered.. i guess then what Terence Tao meant by higher order is lower order in the featured artiicle!