3 ms·

Thank you, this is a great point! I am in the category of someone who needs linear algebra in order to apply it for day-to-day stuff, hands on not blue sky. Cur

Thank you, this is a great point! I am in the category of someone who needs linear algebra in order to apply it for day-to-day stuff, hands on not blue sky. Currently my primary use case is image filtering but a bit down the line signal processing will come up.

dragontamer 4y ago
> Currently my primary use case is image filtering but a bit down the line signal processing will come up. Image filtering _is_ signal processing, two-dimensional signal processing to be precise. Traditionally, a college would take you through linear algebra -> differential equations -> signals and systems, to approach this subject. I found it easier to go through the reverse: start at signals-and-systems (to see what you have to learn), then work your way back down to linear algebra, and then work your way back up to signals and systems. --------------- From a "signals and systems" point of view, your image filtering functions are 99% going to just be a "kernel" applied to an image. https://en.wikipedia.org/wiki/Kernel_(image_processing) https://en.wikipedia.org/wiki/Kernel_(image_processing) IMO, its easier to start with a 1-dimensional version, where you perform kernels upon sound and/or RADAR signals rather than 2-dimensional images. https://en.wikipedia.org/wiki/Convolution#Visual_explanation https://en.wikipedia.org/wiki/Convolution#Visual_explanation You can see that the 1-dimensional version of the convolution applied between (data x kernel) is extremely simple and "obvious" to think about, given this GIF: https://upload.wikimedia.org/wikipedia/commons/6/6a/Convolution_of_box_signal_with_itself2.gif https://upload.wikimedia.org/wikipedia/commons/6/6a/Convolut... Where blue-box is the original signal, and red-box is the convolution-kernel, and the black-line is the output of blue convolve with red. From there, you generalize the 1-dimensional convolution, into a 2-dimensional convolution. To do so, you need to study linear algebra and matricies. But now that you're "focused" upon the convolution idea, as well as the idea of a "kernel", everything should be "more obvious" to you as you go through your studies. You can see that a "Matrix", in your specific field of study, represents a kernel to a discrete system. The image you want to manipulate is a 2-dimensional signal. A "matrix" is many different things to many different mathematicians / engineers. "Focusing" upon your particular application is key to learning as quickly as possible. (You can generalize later after you've mastered your particular field). Still, the study of signals / systems is a very generalized and large field. Mechanical engineers study this, because it turns out that an "impulse" that is "convoluted" with a "kernel" is descriptive of how a speed-bump affects your car's suspension system (!!!!). (EDIT: A youtube video demonstrating the same math for earthquakes vs buildings: https://www.youtube.com/watch?v=f1U4SAgy60c https://www.youtube.com/watch?v=f1U4SAgy60c) So studying signals-and-systems is still a very abstract goal of yours. It sounds like you need to focus upon the image-processing portions of signals-and-systems. --------- IMO, you'll find that there's probably very little linear algebra you actually need to learn for your particular path.
- bkcooper 4y ago
  I largely agree with your comments in this thread -- I'd been thinking about trying to express the same thing myself. I'd been guessing the motivation would be ML, where I feel that most people substantially overestimate how much they'd need. Signals processing, though, is one of the places where I actually think a decent understanding of some of the higher-level concepts in linear algebra is really helpful. Linearity itself comes to mind, and maybe it just reflects my physicist's education, but it's hard for me to imagine having a working understanding of Fourier transforms without getting the idea of changing bases. I feel like you're about halfway through a first linear algebra course before you'd get there. EDIT: that said, a good signals processing book probably covers a lot of this in sufficient depth if you can figure it out. The other catch-all comment I'd make is that linear algebra from a math class can look somewhat different from practical linear algebra on a computer. (That it's often a bad idea to directly invert a matrix for many computing applications is non-obvious from math class.) A book like Trefethen and Bau is great on that latter subject but is not a good starting point for OP.

dragontamer 4y ago

> Currently my primary use case is image filtering but a bit down the line signal processing will come up. Image filtering _is_ signal processing, two-dimensional signal processing to be precise. Traditionally, a college would take you through linear algebra -> differential equations -> signals and systems, to approach this subject. I found it easier to go through the reverse: start at signals-and-systems (to see what you have to learn), then work your way back down to linear algebra, and then work your way back up to signals and systems. --------------- From a "signals and systems" point of view, your image filtering functions are 99% going to just be a "kernel" applied to an image. https://en.wikipedia.org/wiki/Kernel_(image_processing) https://en.wikipedia.org/wiki/Kernel_(image_processing) IMO, its easier to start with a 1-dimensional version, where you perform kernels upon sound and/or RADAR signals rather than 2-dimensional images. https://en.wikipedia.org/wiki/Convolution#Visual_explanation https://en.wikipedia.org/wiki/Convolution#Visual_explanation You can see that the 1-dimensional version of the convolution applied between (data x kernel) is extremely simple and "obvious" to think about, given this GIF: https://upload.wikimedia.org/wikipedia/commons/6/6a/Convolution_of_box_signal_with_itself2.gif https://upload.wikimedia.org/wikipedia/commons/6/6a/Convolut... Where blue-box is the original signal, and red-box is the convolution-kernel, and the black-line is the output of blue convolve with red. From there, you generalize the 1-dimensional convolution, into a 2-dimensional convolution. To do so, you need to study linear algebra and matricies. But now that you're "focused" upon the convolution idea, as well as the idea of a "kernel", everything should be "more obvious" to you as you go through your studies. You can see that a "Matrix", in your specific field of study, represents a kernel to a discrete system. The image you want to manipulate is a 2-dimensional signal. A "matrix" is many different things to many different mathematicians / engineers. "Focusing" upon your particular application is key to learning as quickly as possible. (You can generalize later after you've mastered your particular field). Still, the study of signals / systems is a very generalized and large field. Mechanical engineers study this, because it turns out that an "impulse" that is "convoluted" with a "kernel" is descriptive of how a speed-bump affects your car's suspension system (!!!!). (EDIT: A youtube video demonstrating the same math for earthquakes vs buildings: https://www.youtube.com/watch?v=f1U4SAgy60c https://www.youtube.com/watch?v=f1U4SAgy60c) So studying signals-and-systems is still a very abstract goal of yours. It sounds like you need to focus upon the image-processing portions of signals-and-systems. --------- IMO, you'll find that there's probably very little linear algebra you actually need to learn for your particular path.

bkcooper 4y ago
I largely agree with your comments in this thread -- I'd been thinking about trying to express the same thing myself. I'd been guessing the motivation would be ML, where I feel that most people substantially overestimate how much they'd need. Signals processing, though, is one of the places where I actually think a decent understanding of some of the higher-level concepts in linear algebra is really helpful. Linearity itself comes to mind, and maybe it just reflects my physicist's education, but it's hard for me to imagine having a working understanding of Fourier transforms without getting the idea of changing bases. I feel like you're about halfway through a first linear algebra course before you'd get there. EDIT: that said, a good signals processing book probably covers a lot of this in sufficient depth if you can figure it out. The other catch-all comment I'd make is that linear algebra from a math class can look somewhat different from practical linear algebra on a computer. (That it's often a bad idea to directly invert a matrix for many computing applications is non-obvious from math class.) A book like Trefethen and Bau is great on that latter subject but is not a good starting point for OP.