13 ms·
Wavelets allow researchers to transform and understand data
- mensetmanusman 5y agoWavelets are like localized Fourier transforms. Super useful in some compression algorithms.
- borodi 5y agoThey are very similar to a short term Fourier transform (STFT). Both of which permit one to analyze the spectrum of signal over time
- nickff 5y agoI agree that the discrete wavelet transform is analogous to the discrete Fourier transform, but they are calculated differently, and present very different information in a very different way. I think most people find DWT results much more confusing than FFT results. If you're very familiar with both of them, it can be easy to underestimate how confusing each is to a newcomer.
- TimMurnaghan 5y agoNice to see some love for wavelets. They're great for many more things. Robust volatility measurement, galerkin methods for numerical solutions to differential equations in finance. Just please don't use a Haar wavelet for anything other than teaching.
- ur-whale 5y ago>Nice to see some love for wavelets. Yeah, I agree. Wavelets were all the rage in the late 80's and 90's and they seem to have fallen out of fashion. As a matter of fact, It's kind of strange that applied math techniques be subject to fashion. I think there is quite a lot to be done looking at deep nets in terms of wavelets for example.
- nrr 5y agoMathematicians are human just the same and are wont to get their hands on shiny toys too from time to time. There's some value in seeking out novelty. Apropos deep nets, with the explosion in machine learning in the past few years, I've been seeing a lot of research interest statements change to meet that. In particular, there's an awful lot of numerical linear algebra being done now. I suspect that things will come full-circle soon enough, and those tools developed in numerical linear algebra (via their connections to functional analysis) will make their way to harmonic analysis. (This is notwithstanding the fact that compressed sensing is picking up a little momentum as a research area in applied mathematics and other disciplines that study signal processing. Wavelets, curvelets, shearlets, chirplets, etc. will likely see some action there too.)
- kragen 5y agoIt's rational to seek out novelty. What's more likely: that you'll be the first to discover a momentous consequence of a theorem published last week, or that you'll be the first to discover a momentous consequence of a theorem published by Euler?
- east2west 5y agoCould you provide some examples of recent numerical linear algebra inspired by machine learning? The only numerical linear algebra related to machine learning, in particular deep learning, is matrix multiplication due to convolution. I am curious what else in numerical linear algebra is impacting machine learning.
- ur-whale 5y ago> The only numerical linear algebra related to machine learning, in particular deep learning, is matrix multiplication I suspect the OP is not talking about how deep nets are implemented, but rather how people are trying to understand how and why they work so well, or how to reverse-engineer knowledge out of a trained net, or how to train them faster, etc ... In that space, you need quite a bit more than matrix multiplication.
- eutectic 5y agoI though Haar wavelets were good for text and other hard-edged images?
- nickff 5y agoHear wavelets can be extremely useful when the signal of interest is square-shaped, or fast processing is required.
- dr_kiszonka 5y agoCan you use wavelets in analyzing behavioral data (clicks, decisions, etc.)? Intuitively, wavelets, and SP in general, seem like something behavioral folks could benefit from, but I can't come up with any good use cases that would justify going through all the associated math.
- amcoastal 5y agoWavelets have also been used in the physical sciences. Here they are used to translate sea surface patterns into water depths: https://www.mdpi.com/2077-1312/8/10/772 https://www.mdpi.com/2077-1312/8/10/772
- VikingCoder 5y agoSo, dumb question... If you were training some deep learning model... ...should you be trying a few Wavelet transforms on your inputs, and feeding those in to your model, too, to see if your model performs better with wavelet inputs?
- nickff 5y agoIt very much depends on your inputs, but it’s likely worth a shot. Note that wavelet transforms are generally much slower than fast Fourier transforms.
- actusual 5y agoInstead of saying "I'm going to try this, maybe it will work", you should instead be asking if wavelet transforms are appropriate given the domain you are building a model for. Don't just transform data in the hopes that it will magically work.
- ericjang 5y agoHypothetically, if the computer could try both of these experiments at no cost to you, and tell you whether it improved things or not, does asking whether wavelets are appropriate for the domain even matter?
- taneq 5y agoOr if you do, write down the results and publish them even if they’re negative. That’s how science is meant to work.
- cinntaile 5y agoI guess the overarching question is... How do you determine if they are good for your application and how do you choose which family of wavelets to apply?
- aaaaaaaaaaab 5y ago>Don't just transform data in the hopes that it will magically work. You’ve just described modern machine learning.
- graycat 5y agoThe referenced article has: "Wavelets came about as a kind of update to an enormously useful mathematical technique known as the Fourier transform. In 1807, Joseph Fourier discovered that any periodic function — an equation whose values repeat cyclically — could be expressed as the sum of trigonometric functions like sine and cosine." Due to the "periodic", that math is Fourier series, not the Fourier transform. Given almost any periodic function, the Fourier transform won't exist. The math of Fourier theory is done carefully in two of the W. Rudin texts: Fourier series is in his Principles of Mathematical Analysis, and Fourier transforms is in his Real and Complex Analysis. For more, his Functional Analysis covers the related distributions. See also the "celebrated" Riesz–Fischer theorem, IIRC in his Real and Complex Analysis. As I recall, wavelets have some good completeness properties. Power spectra are of interest, and for that see the Wiener–Khinchin theorem. For the relevant statistics of estimating power spectra, see the work of Blackman and Tukey. The referenced article also has "That’s because Fourier transforms have a major limitation: They only supply information about the frequencies present in a signal, saying nothing about their timing or quantity." Phase gives some information on timing, and the power spectra, on quantity. The article also has "Whenever you have a particularly good match, a mathematical operation between them known as the dot product becomes zero, or very close to it." A dot product value of zero means orthogonality, that is, in the usual senses, neither signal is useful for saying any much about the other. Of course, in the case of stochastic processes, the orthogonality of a dot product (inner product) is not the same as probabilistic independence. For computations, of course, see the many versions of the fast Fourier transform, in response to a question from R. Garwin, rediscovered by Tukey, first programmed by Cooley, later given various developments and many applications. At one point early in my career, these topics got the company I was working for "sole source" on a US Navy contract and got me a nice, new high end Camaro, a nice violin, a nice piano for my wife, some good French food, a sack full of nice Nikon camera equipment, and a nice stock market account! My annual salary was 6+ the cost of the Camaro.
- mr_luc 5y agoSo if I'm understanding this right -- if I have a stream of numbers coming in forming a squiggly line, and I have a bucket of these wavelet shapes, I can pick up wavelets and stretch and squeeze and resize them and overlay them on my line, and ... use them to characterize that squiggly line? Working as feature recognition and also serving as a way to compress it? So instead of 20k data points, I have a sequence like 'mexican hat, mexican hat', and maybe elements in the sequence are different sizes and overlap? (a) if my intuition is super wrong I'd love for HN to correct it, heh. (b) long shot, but anyone have links to code? It's HN after all -- maybe some commenters in this thread are aware of cool/idiomatic/simple/etc uses of wavelet code in open source software. (I know of a bunch of cool uses they've been put to, from JPEGs to the spacex fluid dynamics presentation about using wavelet compression on gpu, but I've personally never used them as a tool for anything, and it'd be fun to learn about them with code!)
- maliker 5y agoYes. A mathematician explained it to me as an alternative to a fourier transform: instead of describing the function as a sum of sine waves, its a sum of mexican hats (or whatever basis function). And it turns out that’s a simpler representation in the case of function with sharp discontinuities. It’s also an alternative to a Taylor series, replacing sums of derivatives with the sum of the scaled basis function. Seemed like a pretty elegant explanation to me.
- a-dub 5y agoyes, this is how i understand them. it has been a long time, so i'm probably going to say something really stupid... but it's fun to try and hopefully someone will correct me if i'm wrong, my understanding is that one place that they're useful is in terms of time-frequency uncertainty that occurs with regular fourier transforms. for example, if you're doing spectral analysis and looking for low frequencies you need a wider analysis window (a full cycle has a long period for low frequencies), but that wide analysis window now covers a lot of time, so if you pick up a low frequency component, you don't know where it is in that big wide window. wavelets instead fit these families of arbitrary basis functions that can be more compact, so you can use a narrower window and therefore have less time uncertainty for the things you find within it. ...most applications sort of go from there, where you want to warp the equivalent of the x-axis in the frequency domain in various ways and perhaps be able to either use a narrower analysis window for better time resolution or say, a smaller number of basis functions for reconstruction in lossy compression where some bands are less important than others and resolution in those bands can be discarded.