5 ms·

Laugh. I was hoping no one would ask since I don't have a great answer! We used "An introduction to Hilbert space" by Young. It's fine? There's stuff in the

by kxyvr 5y ago

Laugh. I was hoping no one would ask since I don't have a great answer! We used "An introduction to Hilbert space" by Young. It's fine? There's stuff in there like Sturm-Liouville systems that I don't particularly use or care for. I also never use it as a reference. Here's a hodgepodge of other books related to functional analysis that I like more, but don't directly answer your question. I got a lot of benefit out of "Convex Functional Analysis" by Kurdila and Zabarankin. They sort of have a high level overview of different functional analysis topics, including Hilbert Spaces, but with the ultimate goal of proving what they call the Generalized Weierstrass Theorem. Essentially, when does a function has an inf and when is it attained. Even if you don't care about optimization theory, I very much appreciated their survey of topics to get there. I occasionally also use "Introductory Functional Analysis with Applications" by Kreyszig as a reference. I think this was the first time I saw cleanly the difference between an adjoint and the Hilbert-adjoint of an operator, which was constantly confusing to me prior to that point. The last one I like is "Nonlinear Funtional Analysis and its Applications I: Fixed-Point Theorems" by Zeidler. He wrote, I think, five volumes, but this is the only one that I use. Anyway, he presents differentiation, Taylor theorem, and the implicit function theorem very well in function spaces. The first four chapters are great as a reference. Since I'm listing off obscure books, for integration, I like "A Concise Introduction to the Theory of Integration" by Stroock. I actually don't like his newer book, "Essentials of Integration Theory for Analysis" as much as the older book. Anyway, I find it very dense, but well written. Essentially, I like the first five chapters, which culminates with the divergence theorem, which ultimately gives a precise description of integration by parts in more than one dimension. He also answers precisely the question about the difference between Reimann and Lebesgue integrals.

bigdict 5y ago
Thank you! What is that extra power that comes from considering Hilbert spaces as opposed to staying in good ol' R^n?
- kxyvr 5y ago
  Primarily, it gives us the power to work with functions that return other functions. In many ways, it's kind of like going from array processing in Fortran to functional programming. Outside of the esoteric, there's a choice that has to be made while modeling as to when to discretize the system into a linear system. The ability to use a function space means that we can manipulate the formulation into another form before discretizing, which impacts the convergence theory as well as the practical performance. The reason that I mention Hilbert spaces is that they have more structure than a general function space, which makes working with them easier, but still general enough to be useful. Essentially, we get an inner product as well as the ability to enumerate an orthonormal basis, which makes it feel more like working with linear algebra. Even in R^n, I believe strongly that it's important to code with these abstractions. First, it makes parallelizing the algorithms easier. If we treat a vector as simply an array of numbers and use the dot product, then the code requires a more significant rewrite when moving to multi-computer parallelism. If we treat them as a generic vector object and have an interface that works with inner products, addition, scaling, etc, then the same code can work either in serial or in parallel given two different implementations of the vectors and their operations. In addition to the abstractions, the choice of inner product is important. If we have a linear operator in a Hilbert space and discretize it, we generally have to discretize three things: the operator, it's Hilbert adjoint, and the inner product. If we implement this blindly, they're not consistent after discretization. Meaning, the property that we should get is that <Ax,y> = <x,adj(A)y>, but this probably isn't true if not done carefully. Generally, we can freely discretize two out of three of those operations and then the third one needs to be adapted for consistency. Maybe you want to choose the discretization for the operator and its adjoint, but this probably requires a non dot product for an inner product. Alternatively, maybe you're optimizing some problem and realize that some of your variables are out of scale, so it's converging slowly. You can certainly just rescale your variables with a diagonal scaling. However, you can also change your inner product, which changes the gradient, which also rescales the problem in a different way. Mostly, that's to say that an inner product should be a choice that one freely and intentionally makes. Learning to work in Hilbert spaces forces us to become comfortable with this approach.

by kxyvr 5y ago

bigdict 5y ago

Thank you! What is that extra power that comes from considering Hilbert spaces as opposed to staying in good ol' R^n?

kxyvr 5y ago
Primarily, it gives us the power to work with functions that return other functions. In many ways, it's kind of like going from array processing in Fortran to functional programming. Outside of the esoteric, there's a choice that has to be made while modeling as to when to discretize the system into a linear system. The ability to use a function space means that we can manipulate the formulation into another form before discretizing, which impacts the convergence theory as well as the practical performance. The reason that I mention Hilbert spaces is that they have more structure than a general function space, which makes working with them easier, but still general enough to be useful. Essentially, we get an inner product as well as the ability to enumerate an orthonormal basis, which makes it feel more like working with linear algebra. Even in R^n, I believe strongly that it's important to code with these abstractions. First, it makes parallelizing the algorithms easier. If we treat a vector as simply an array of numbers and use the dot product, then the code requires a more significant rewrite when moving to multi-computer parallelism. If we treat them as a generic vector object and have an interface that works with inner products, addition, scaling, etc, then the same code can work either in serial or in parallel given two different implementations of the vectors and their operations. In addition to the abstractions, the choice of inner product is important. If we have a linear operator in a Hilbert space and discretize it, we generally have to discretize three things: the operator, it's Hilbert adjoint, and the inner product. If we implement this blindly, they're not consistent after discretization. Meaning, the property that we should get is that <Ax,y> = <x,adj(A)y>, but this probably isn't true if not done carefully. Generally, we can freely discretize two out of three of those operations and then the third one needs to be adapted for consistency. Maybe you want to choose the discretization for the operator and its adjoint, but this probably requires a non dot product for an inner product. Alternatively, maybe you're optimizing some problem and realize that some of your variables are out of scale, so it's converging slowly. You can certainly just rescale your variables with a diagonal scaling. However, you can also change your inner product, which changes the gradient, which also rescales the problem in a different way. Mostly, that's to say that an inner product should be a choice that one freely and intentionally makes. Learning to work in Hilbert spaces forces us to become comfortable with this approach.