10 ms·
Functions Are Vectors (2023)
- skybrian 1y agoIt seems like mentioning some of the applications at the beginning would motivate learning all these definitions.
- deleted 1y ago[deleted]
- almostgotcaught 1y ago> "The material is not motivated." Not motivated? Judas just stick a dagger in my heart. This material needs no motivation. Just do it. Faith will come. He's teaching you analysis. Not selling you a used car. By the time you are ready to read this book you should not need motivation from the author as to why you need to know analysis. You should just feel a burning in you chest that can only be quenched by arguments involving an arbitrary sequence {x_n} that converges to x in X. https://www.amazon.com/review/R23MC2PCAJYHCB https://www.amazon.com/review/R23MC2PCAJYHCB
- skybrian 1y agoNot sure what I'm supposed to get from that. I guess some people care a little too much about math and have trouble relating to others?
- almostgotcaught 1y agoyou're supposed to get that the cynical lens you're applying here doesn't fit - if you aren't intrinsically motivated to read this stuff then it's not for you. which is fine btw because (functional) analysis isn't a required class.
- TheRealPomax 1y agoIf you need "practical applications" for some part of math to have value to you, then large parts of math will not be for you. That's fine, but that's also something you should accept and internalize: math is already its own application, we dig through it in order to better understand it, and that understanding will (with rather advanced higher education) be applicable to other fields, which in turn may have practical uses. Those practical uses are someone else's problem to solve (even if they rely on math to solve them), and they can write their own web pages on how functions as vectors help solve specific problems in a way that's more insightful than using "traditional" calculus, and get those upvoted on HN. But this link has a "you must be this math to ride" gate, it's not for everyone, and that's fine. It's a world wide web, there's room for all levels of information. You need to already appreciate the problems that you encountered in non-trivial calculus to appreciate this interpretation of what a function even is and how to exploit the new power that gives you.
- skybrian 1y agoI don't see any such "math gate" on this link. Also, this math does have practical applications, but they're not mentioned until very late in the article. My suggestion is that briefly mentioning them up front might be nice. I didn't mean to start a big argument about it.
- almostgotcaught 1y agoi'll never fathom why people on hn treat a post as an auto-invite for unsolicited feedback.
- LegionMammal978 1y agoYet some parts of math are 'preferred' over others, in that most 'serious' mathematicians would rather read 100 pages about functional analysis than 100 pages of meandering definitions from some rando trying to solve the Collatz conjecture. Some people would like to have a filter for what to spend their time on, better than "your elders before you have deemed these ideas deeply important". One such filter is "Can these ideas tell us nontrivial things about other areas of math?" That is, "Do they have applications?" Short of the strawman of immediate economic value, I don't think it's wrong to view a subject with light skepticism if it seemingly ventures off into its own ivory tower without relating back to anything else. A few well-designed examples can defuse this skepticism.
- ethan_smith 1y agoThis perspective is crucial for understanding signal processing, machine learning, and quantum mechanics. Viewing functions as vectors enables practical techniques like Fourier transforms and kernel methods that underlie many modern technologies.
- sixo 1y agoThe genre of this article is not pedagogical, really. One usually learns these techniques in the course of a particular field like physics, electrical engineering, or theoretical chemistry. This article is best thought of as "a story you've seen before, but told from the beginning / ground up, with a lot of the connections to other topics and examples laid out for you". For that purpose, it's excellent, perhaps the best I've ever seen. It might also whet the appetite of a novice, but it's not really for that.
- gizmo686 1y agoThe first paragraph and table of context both mention applications.
- skybrian 1y agoYes, so it does. Perhaps I read too quickly.
- deleted 1y ago[deleted]
- pvg 1y agoDiscussion at the time https://news.ycombinator.com/item?id=36921446 https://news.ycombinator.com/item?id=36921446
- nyrikki 1y agoThe one place that I think the previous discussion lost something important, at least to me with functions. The popular lens is the porcupine concept when infinite dimensions for functions is often more effective when thought of as around 8:00 in this video. https://youtu.be/q8gng_2gn70 https://youtu.be/q8gng_2gn70 While that video obviously is not fancy, it will help with building an intuition about fixed points. Explaining how the dimensions are points needed to describe a functions in a plane and not as much about orthogonal dimensions. Specifically with fixed points and non-expansive mappings. Hopefully this helps someone build intuitions.
- olddustytrail 1y ago> infinite dimensions for functions is often more effective when thought of as around 8:00 I guess it works if you look at it sideways.
- chongli 1y agoI see this a lot with math concepts as they begin to get more abstract: strange visualizations to try to build intuition. I think this is ultimately a dead-end approach which misleads rather than enlightens. To me, the proper way of continuing to develop intuition is to abandon visualization entirely and start thinking about the math in a linguistic mode. Thus, continuous functions (perhaps on the closed interval [0,1] for example) are vectors precisely because this space of functions meet the criteria for a vector space: * (+) vector addition where adding two continuous functions on a domain yields another continuous function on that domain * (.) scalar multiplication where multiplying a continuous function by a real number yields another continuous function with the same domain * (0) the existence of the zero vector which is simply the function that maps its entire domain of [0,1] to 0 (and we can easily verify that this function is continuous) We can further verify the other properties of this vector space which are: * associativity of vector addition * commutativity of vector addition * identity element for vector addition (just the zero vector) * additive inverse elements (just multiply f by -1 to get -f) * compatibility of scalar multiplication with field multiplication (i.e a(bf) = (ab)f, where a and b are real numbers and f is a function) * identity element for scalar multiplication (just the number 1) * distributivity of scalar multiplication over vector addition (so a(f + g) = af + ag) * distributivity of scalar multiplication over scalar addition (so (a + b)f = af + bf) So in other words, instead of trying to visualize an infinite-dimensional space, we’re just doing high school algebra with which we should already be familiar. We’re just manipulating symbols on paper and seeing how far the rules take us. This approach can take us much further when we continue on to the ideas of normed vector spaces (abstracting the idea of length), sequences of vectors (a sequence of functions), and Banach spaces (giving us convergence and the existence of limits of sequences of functions).
- Scene_Cast2 1y agoSame thing in video form explained by a different person - https://youtu.be/mhEFJr5qvLo https://youtu.be/mhEFJr5qvLo
- deleted 1y ago[deleted]
- malwrar 1y agoSo cool! This is the first time I’ve ever read about a math idea and felt a deep pull to know more.
- tempodox 1y agoOh, my. Alice, meet rabbit hole.
- MalbertKerman 1y agoThe jump from spherical harmonics to eigenfunctions on a general mesh, and the specific example mesh chosen, might be the finest mathematical joke I've seen this decade.
- sixo 1y agoWould you explain the joke for the rest of us?
- xeonmc 1y agoSpherical Haromics approximating Spherical Cows?
- dark__paladin 1y agoassume spherical cow
- MalbertKerman 1y agoIt's quietly reversing the traditional "We approximate the cow to be a sphere" and showing how the spherical math can, in fact, be generalized to solutions on the cow.
- gsf_emergency_2 1y agoRelated to this footnote in TFA? >If you’re alarmed by the fact that the set of all real functions does not form a HILBERT SPACE, you’re probably not in the target audience of this post." Video: https://youtu.be/q8gng_2gn70?t=8m3s https://youtu.be/q8gng_2gn70?t=8m3s Thanks to https://news.ycombinator.com/item?id=44481933 https://news.ycombinator.com/item?id=44481933
- a3w 1y agoNice: the variable l and m values can allow you to get orbitals from chemistry. (This is where I learned at least half of the math on this page: theoretical chemistry.)
- deleted 1y ago[deleted]
- xeonmc 1y agoalso known as Applied Quantum Mechanics.
- ttoinou 1y agoIsn't this the opposite way ? Vectors are functions whose input space are discrete dimensions. Let's not pretend going from natural numbers to real is "simple", reals numbers are a fascinating non-obvious math discovery. And also the passage from a few numbers to all natural numbers (aleph0) is non obvious. So basically we have two alephs passages to transforms N-D vectors as functions over reals.
- xeonmc 1y agoVectors are not necessarily discrete-domained. Anything that satisfies the vector space properties is a vector.
- ttoinou 1y agoI agree but I'm operating under the assumption of the article Conceptualizing functions as infinite-dimensional vectors lets us apply the tools of linear algebra to a vast landscape of new problems
- layer8 1y agoLinear algebra isn’t limited to discrete-dimensional vector spaces. Or what do you mean?
- ttoinou 1y agoSee my other comment sibling. And he's starting from the assumption vectors are finite (cf. the article)
- Sharlin 1y agoHe does not assume anything! Any assumption is in your head only. Of course he starts from the specific type of vector spaces that's the most familiar to readers. But then he shows that there's nothing that requires a vector space to have a finite, or even countably infinite, dimension. What matters are the axioms.
- sixo 1y agoA few questions occur to me while reading this, which I am far from qualified to answer: - How much of this structure survives if you work on "fuzzy" real numbers? Can you make it work? Where I don't necessarily mean "fuzzy" in the specific technical sense, but in any sense in which a number is defined only up to a margin of error/length scale, which in my mind is similar to "finitism", or "automatic differentiation" in ML, or a "UV cutoff" in physics. I imagine the exact definition will determine how much vectorial structure survives. The obvious answer is that it works like a regular Fourier transform but with a low-pass filter applied, but I imagine this might not be the only answer. - Then if this is possible, can you carry it across the analogy in the other direction? What would be the equivalent of "fuzzy vectors"? - If it isn't possible, what similar construction on the fuzzy numbers would get you to the obvious endpoint of a "fourier analysis with a low pass filter pre-applied?" - The argument arrives at fourier analysis by considering an orthonormal diagonalization of the Laplacian. In linear algebra, SVD applies more generally than diagonalizations—is there an "SVD" for functions?
- xeonmc 1y agoI’d guess that it would be factored as “nonlinearity”, which might be characterized as some form of harmonic distortion, analogous to clipping nonlinearity of finite-ranged systems? Perhaps some conjugate relation could be established between finite-range in one domain and finite-resolution in another, in terms of the effect such nonlinearities have on the spectral response.
- sitkack 1y agoA fuzzy vector is a Gaussian? Thinking of what it would be in 1, 2, 3 and n dimensions.
- sfpotter 1y ago1. Numerical methods for solving differential and integral equations are algorithms for solving algebraic equations (vector solutions) that arise from discretizing infinite-dimensional operator equations (function solutions). When we talk about whether these methods work, we usually do so in terms of their consistency and stability. There is a multistage things that happens here: we start by talking about the well-posedness of the original equation (e.g. the PDE), then the convergence of the mathematical discretization, and then examine what happens when we try to program this thing on a computer. Usually what happens is these algorithms will get implemented "on top" of numerical linear algebra, where algorithms like Gaussian elimination, and different iterative solvers, have been studied very carefully from the perspective of floating point rounding errors etc. This kind of subsumes your concern about "fuzzy" real numbers. Remember that in double precision, if the number "1.0" represents "1 meter", then machin epsilon is atomic scale. So, frequently, you can kind of assume the whole process "just works"... 2/3. I'm not really sure what you mean by these questions... But if you want to do "fourier analysis with a filter preapplied", you'd probably just work with within some space of bandlimited functions. If you only care around N Fourier modes, any time you do an operation which exceeds that number of modes, you need to chop the result back to down to size. 4. In this context, it's really the SVD of an operator you're interested in. In that regard, you can consider trying to extend the various definitions of the SVD to your operator, provided that you carefully think about all spaces involved. I assume at least one "operator SVD" exists and has been studied extensively... For instance, I can imagine trying to extend the variational definition of the SVD... and the algorithms for computing the SVD probably make good sense in a function space, too...
- simpaticoder 1y agoThe author asserts vectors are functions, specifically a function that takes an index and returns a value. He notes that as you increase the number of indices, a vector can contain an arbitary function (he focuses on continuous, real-valued functions). It's fun to simulate one thing with another, but there is a deeper and more profound sense in which vectors are functions in Clifford Algebra, or Geometric Algebra. In that system, vectors (and bi-vectors...k-vectors) are themselves meaningful operators on other k-vectors. Even better, the entire system generalizes to n-dimensions, and decribes complex numbers, 2-d vectors, quaternions, and more, essentially for free. (Interestingly, the primary operation in GA is "reflection", the same operation you get in quantum computing with the Hadamard gate)
- elbear 1y agoThanks for mentioning this. I think I encountered Geometric Algebra in a video related to making video games. It's this one (Why you can't multiply vectors): https://www.youtube.com/watch?v=htYh-Tq7ZBI https://www.youtube.com/watch?v=htYh-Tq7ZBI
- layer8 1y agoWell, yeah, function spaces are an example of vector spaces: https://en.wikipedia.org/wiki/Vector_space#Function_spaces https://en.wikipedia.org/wiki/Vector_space#Function_spaces
- imtringued 1y agoYeah, and it's such a boring thing to write about. Given an vector space V with (+, ), you can define the vector space over functions F whose codomain is V and where F.+ and F. both take two functions as argument and return another function applying V.+ or V.* on the result. All the linear algebra properties come from the original vector space. Hence it is boring.
- dragonwriter 1y ago* is used on HN for italic delimiters, you need to escape it with a backslash to avoid that.
- dang 1y agoThis previous thread was also good: Functions are vectors - https://news.ycombinator.com/item?id=36921446 https://news.ycombinator.com/item?id=36921446 - July 2023 (120 comments)
- EGreg 1y agoOnly functions on a finite domain are vectors. Functions on a countable domain are sequences.
- ttoinou 1y agoWhy is this being downvoted ? Could a downvoter elaborate ?
- teiferer 1y agoBecause it makes little sense. Vector spaces can have infinite dimension, so the "only" in the first sentence does not belong there. The second sentence is also odd. How do you define "sequence"? Are there no finite sequences?
- ttoinou 1y agoI think it is "vector" taken in the way the author wrote about it / showed illustrations in the article. For the second sentence, he's right, we could also write (wrongly) an article titled "Functions are Sequences" and (try to) apply what we know about dealing with countable sequences to functions
- nh23423fefe 1y agosequence is typically defined as a function from N -> A, thus countable domain
- jschveibinz 1y agoAn engineering, signal processing extension/perspective: An infinite sequence approximates a general function, as described in the article (see the slider bar example). In signal processing applications, functions can be considered (or forced) to be bandlimited so a much lower-order representation (i.e. vector) suffices: - The subspace of bandlimited functions is much smaller than the full L^2 space - It has a countable orthonormal basis (e.g., shifted sinc functions) - The function can be written as (with sinc functions): x(t) = \sum_{n=-\infty}^{\infty} f(nT) \cdot \text{sinc}\left( \frac{t - nT}{T} \right) - This is analogous to expressing a vector in a finite-dimensional subspace using a basis (e.g. sinc) Discrete-time signal processing is useful for comp-sci applications like audio, SDR, trading data, etc.
- QuesnayJr 1y agoFull $L_2$ also has a countable orthonormal basis. Hermite functions are one example.
- MITSardine 1y agoThis idea is also at the core of the Finite Element Method (and other Galerkin methods, such as DG) which are probably the most widespread methods to approximate the solutions of partial differential equations (PDEs) used everywhere in physics and engineering. Basically, define a finite dimensional function space V_N of dimension N, in such a way that you could grow N to be arbitrarily large. Solve not the PDE (originally defined over an infinite dimensional function space V, such as H^1), but its discretization as if it dealt only with functions in V_N rather than all functions. The PDE is then simply a linear system, easy to solve. And you can prove, for instance in the case of elliptic PDEs, that the solution to the discrete problem is the orthogonal projection of the true solution of the PDE (in V) onto V_N (Céa's Lemma). Finally, you can produce estimations of the error this projection incurs as a function of N, and thus give theoretical guarantees that the algorithm converges to the true solution as N goes to infinity. (N in this case is the number of vertices in a mesh that is used to define the basis functions of V_N)
- 77pt77 1y agoAny basic liniear algebra course should talk about this, at least in the finite dimensional case. Polynomials come to mind.
- ttoinou 1y agoFinite degree polynomials are vectors yes. Polynomials is a typical example you study when learning about linear algebra. Doesn't say anything about real functions in general though, I don't think any linear algebra course should make the analogies made in this article, that'd be confusing
- 77pt77 1y agoI think they should. You don't need to go into intricacies like metrics and approximations because that's more like analysis. You can however talk about infinite dimensional vectors spaces and talk about projections onto finite subspaces.
- mouse_ 1y agoI love the prerequisites section. Every technical blog post should start with this.
- bmitc 1y agoI will need to read through the rest of the article later, but the initial intuition building is a bit sloppy. None of those vectors drawn in the initial examples belong to the same vector space. Vectors need to emanate from the same origin to be considered as part of the same vector space.
- Tainnor 1y ago> Vectors need to emanate from the same origin to be considered as part of the same vector space. That's absolutely not the case.
- bmitc 1y ago> That's absolutely not the case. Can you elaborate? Which vector space(s) do these vectors belong to, as drawn? If anything, this just highlights the sloppiness of these types of illustrations. Things aren't precise enough, in my opinion, for the illustrations to do anything except confuse.
- Tainnor 1y agoIn the mathematical sense, a vector is nothing but an object that can be added to other objects and scaled by some field (with some reasonable properties attached). In physics, a vector is often more specifically something with magnitude and direction. This still doesn't mean that it needs to be anchored at the origin. Vectors that are anchored at the origin are IIRC called position vectors, but mathematically, if you translate them away from the origin they're still the same vector.
- bmitc 1y agoThe question is one of identification between the drawing and the vector space. See my comment here: https://news.ycombinator.com/item?id=44491474 https://news.ycombinator.com/item?id=44491474. The graph of a function f: X -> Y is the set {(x, f(x)) | x in X}. It is much more clear and precise to associate elements of the graph with vectors such that the 0 vector is identified with the R^2 origin, and then points in R^2 are identified with vectors. Then there is a mapping between vectors in this vector space to the graph, i.e., to points (x, f(x)). > In physics, a vector is often more specifically something with magnitude and direction. Physics is sloppy. :) This is not a general description of a vector, where vector is an element of a general vector space. Not all vector spaces have a norm, which is required for magnitude to make any sense. > but mathematically, if you translate them away from the origin they're still the same vector. Right, and you cannot always translate vectors without more machinery, such as parallel transport.
- ttoinou 1y agoThe author seems to be a great educator and computer scientist, much respect to his work. But from what I can gather, although I'd love to study more infinite sized matrices, he proved / showed nothing in this article. What he wrote is not true at all, they are only analogies and not rigorous maths. Functions are not vectors. But finite polynomials are vectors yes, this is trivial.
- gizmo686 1y agohttps://thenumb.at/Functions-are-Vectors/#proofs https://thenumb.at/Functions-are-Vectors/#proofs It's not a particularly interesting proof, but the author does prove that real valued functions are vectors. The bulk of the article is less about proofs, and more about showing how the above result is useful.
- ttoinou 1y agoVectors in the way he talks about in the beginning. With indices (and then extending to "In higher dimensions, vectors start to look more like functions!"). Of course if you use the general meaning of every word, vectors are functions and functions are vectors, and this article shouldn't then have anything interesting to talk about. how the above result is useful It doesn't seem useful at all to me, the examples in the article are not that interesting. On the contrary it is more confusing than anything to apply linear algebra to real valued functions.
- Tainnor 1y agoI think you are confused about the analogies at the beginning at the article (although there is nothing technically wrong about them). Here's the definition of a vector space which agrees with the one everyone in mathematics use: https://thenumb.at/Functions-are-Vectors/#vector-spaces https://thenumb.at/Functions-are-Vectors/#vector-spaces From this it's fairly easy to prove (and done in the article) that the set of all functions R->R is a vector space.
- ttoinou 1y ago
- graycat 1y ago"Functions Are Vectors"? Let's see: Let A be a non-empty set, N the set of positive whole numbers, and X the set of all functions f f: A --> N with usual notation. Assume as is common, the scalars are the set of real numbers, but the set of complex numbers will also do. So, is X a vector space and, thus, each f in X a vector? No, since -f is not in X. Neither is (1/2)f. Some references (with TeX markup): Paul R.\ Halmos, {\it Finite-Dimensional Vector Spaces, Second Edition\/} linear algebra treated as functional analysis. Walter Rudin, {\it Real and Complex Analysis\/} with Lebesgue integration and, then, Banach and Hilbert vector spaces. Walter Rudin, {\it Functional Analysis\/} with Fourier theory. Jacques Neveu, {\it Mathematical Foundations of the Calculus of Probability\/} with random variables, that is, functions from a probability space to, usually, the set of real numbers with convergence results, building on the work A. Kolmogorov building on the work of H. Lebesgue.
- Tainnor 1y agoFunctions with arbitrary codomains are not vectors, but when your codomain is some field, they are. This is what the article is very explicitly about. I guess you can quibble that the title is imprecise, but it's just a title and the article makes it clear.
- bmacho 1y ago> Functions with arbitrary codomains are not vectors, Well, not with the operations pulled from the codomain at least.
- graycat 1y agoIn all those books I listed, and more, e.g., on axiomatic set theory and other foundations, never saw definition or mention of codomain. So, the term is obscure. Readers are supposed to guess at the meaning of obscure terms? What I did was follow, as in the references, long established convention, that for a function to be a vector at least it had to be in a vector space where (1) can multiply a function by a number (e.g., reals or complex) and (2) add two functions and still get a function in the vector space. To be general, I omitted metrics, inner products, topologies, convergence, probability spaces, and more. Or, as in the references I gave, math talks about vector spaces and vectors, and each vector is in a vector space. The references are awash in definitions of vector spaces with (1) and (2) and much more. Computing is awash in indexes for data, e.g., B-trees, SQL (structured query language) operations on relational data bases, addressing in central processors, collection classes in Microsoft's .NET, REDIS, and calling all such also functions confuses established material, conventions, and understanding.
- elbear 1y agoReally well explained. I managed to follow it for quite a while.