17 ms·

Breakthrough in inverse Laplace transform procedures

233 points by pabo 6y ago

The_rationalist 6y ago
What could be the use cases? Also visually it seems so simple, like for the Mish activation function https://github.com/digantamisra98/Mish/blob/master/README.md https://github.com/digantamisra98/Mish/blob/master/README.md It seems to be advanced maths but I wonder why the designer (he already know in advance the desired form of the function in order to give him a desirable property (in both cases being more smoothed / continuous / centered)) does not draw graphically the desired function and let a software solve, find automatically the best approximation of the function? EDIT: well it seems to be a general function approximator so my point doesn't apply (but still apply for the new activation functions in machine learning)
rollulus 6y ago
A little ELI5 for those who haven't had Laplace transforms at school, from someone who only had a Laplace 101 course, so for what's it worth: Laplace transforms allow you to convert differential equations into easier equations, and back: the differentials and integrals become multiplications and divisions. So you can take a differential equation, transform it into the Laplace domain, manipulate it, and convert it back. And that's cool because differential equations tend to appear everywhere, for instance to model springs, electrical circuits with caps and coils, the surface of a soap bubble in a metal rod, etc. A sibling is the z-transform, which is like the digital version. This one is used for instance to design digital audio filters. I'm sure some math wizards here can elaborate and correct me.
- cm2187 6y ago
  And don’t forget those like me for whom school is a distant memory! What are the domains where this new method can be applied? Is it mostly physics simulations and the likes?
  - timclark 6y ago
    I got taught them in a course on linear systems which was a pre-requisite course to control theory. Lots of electrical circuits, mechanical systems and electro-mechanical systems can be modelled using laplace transforms if they are linear systems. I did an electrical and electronic engineering degree and we got to skip the tedious differential equation solving lectures that the mechanical, civil and chemical engineers had to attend because of Monsieur Laplace.
  - srean 6y ago
    Control systems, signal filters (noise attenuation), modeling epidemics, modeling queues, modeling reliability of repairable systems, modeling recurrent events(such as failures), renewal processes, modeling inventory plans, probability in general (because of the connection with moment generating function) ...
- GolDDranks 6y ago
  I don't have any real understanding over the Laplace transform, but I understand Fourier transform well enough that it makes sense to me. Back then, I saw an claim that Laplace transform is a generalization of Fourer transform in the sense that it transforms a function not only to a space of frequencies and phases of sine waves, but to a larger space of parameters of exponentials. Note that the parameter space of the sine waves is subset of the (complex) parameter space of exponentials. Is this claim correct?
  - beagle3 6y ago
    Yes. The Fourier transform characterizes on a circle (usually the unit circle in the complex domain) at different frequencies. It is properly defined for periodic signals. The Laplace transform takes any exponential spiral in the complex plane, and reduces to the Fourier transform if you only care about the unit circle. I appreciate that doesn’t make things clearer unless you already have some understanding of integral transforms in the complex plane (in which case, you probably know this already). However, I have never met a simple intuitive explanation of the Laplace transform, - and actually no meaningful explanation that doesn’t involve integrals.
    - durbat 6y ago
      With Fourier you can analyze oscillatory characteristics of function (frequency and phase). With Laplace you can also analyze amplification/attenuation.
      - tilt_error 6y ago
        The Fourier transform, well, transforms a time-based phenomenon such as an alternating current sine wave into a frequency spectrum where you can observe the frequency spectrum components of the signal. A pure nice sine becomes a spike (delta function) located at a specific frequency. Music, as we observe it through our ears and can view it on an oscilloscope becomes moving spikes (lots of them :) in the frequency spectrum where the "amplitude" at a given frequency relates to the "amount" of that frequency in the music. The behaviour of a filter is much easier to describe in the frequency spectral domain than it would be in the time domain. Now to the direct current (DC) view. This cannot be handled by the Fourier transform -- at least the DC-part of the signal cannot be transformed to the frequency domain. As shown in the article, there were "steps", "ramps" and such. A typical scenario would be to describe what happens in your amplifier during startup, to describe how electrical circuits are behaving during startup before reaching the "running" state. The Laplace transform will handle these types of scenarios, and can thus be used to study (or describe) systems during other types of transitions than the "steady state" when you are up and running. Regarding filters, the Fourier transform describes things going on at the unit circle, while the Laplace transform can be used to study both the interior and exterior of the plane. In this sense, creating filters relates to locate "poles" and "zeros" in the plane (amplification and attenuation) which can be observed on the unit circle as the behaviour on periodic signals.
- taneq 6y ago
  Sounds right to my (very very rusty) recollection. Laplace transforms are a magic trick that let you easily solve some kinds of differential equations.
  - phkahler 6y ago
    >> Laplace transforms are a magic trick that let you easily solve some kinds of differential equations. To mathematicians I don't think they're so much magic. When I took differential equations class it was frustrating that they went too fast for me to fully digest what was "really" going on. It didn't feel out of reach, but something I needed to look at a couple different ways but didn't have time (or the internet) to do so. Think I'm gonna checkout 3blue1brown after this - he can probably close that gap for me.
    - scythmic_waves 6y ago
      > Think I'm gonna checkout 3blue1brown after this - he can probably close that gap for me. You might like this lecture from MIT's OCW: [1]. It's my favorite source for motivating the Laplace transform. It's a bit difficult to make this concept "simple", and this resource assumes that you already have some familiarity with the following concepts: infinite series, power series, radius of convergence, and (indefinite) integration. The tl;dw is that the Laplace transform is a generalization of a power series. [1] https://www.youtube.com/watch?v=sZ2qulI6GEk https://www.youtube.com/watch?v=sZ2qulI6GEk Edit: I also wrote up a form of this video elsewhere if anyone's interested. It's kinda long though, and I didn't want to spam this thread with it.
- kpmah 6y ago
  I've had some problems understanding the Laplace transform. Maybe somebody here can point me towards some material. I have an interest understanding how IIR filters are designed, and I always get stuck at this part in DSP books. The Laplace transform is used, but as well as finding the mathematics difficut I don't really understand why it is being used at all. I think it is trying to replicate the effect of an analog circuit?
  - Gibbon1 6y ago
    You can describe a circuit by it's time domain behavior. Or you can describe the circuit by it's frequency domain behavior. Both are valid and congruent. The thing is a lot of questions are easy to answer in the frequency domain. For instance, you want to know if a circuit with feedback will oscillate. Hard to answer using time domain equations. But in the frequency domain there is a simple constraint. If for all frequencies where the the gain is greater than one the phase shift is less than 180 degrees, circuit won't oscillate. This is obviously rather useful. Also a point with a lot of 'books' the authors get caught up in describing how something is done that they never explain why something is done. I've found often the answer is simple yet opaque and frustratingly never talked about.
    - pantulis 6y ago
      This. I remember having adequate cursory knowledge of Fourier Transform to the point of understanding the value of FFT algorithms, but the Laplace Transform was explained like hell so I failed my robotics classes.
      - TheOtherHobbes 6y ago
        If you have an electronic circuit, you can model each element with a differential equation. E.g. voltage across a capacitor is modelled as the integral of current, voltage across an inductor is dI/dt. This is a useful fact for a simple circuit in a classroom, but the differential equations for any circuit with more than a few components soon become insanely complex. With the Laplace transform you (more or less) replace an integral with 1/s and a differential with s, plus some constants derived from the component values. Then you can simplify for s, and use the Inverse Laplace Transform to convert the final expression in s into an expression in t. You have now solved an insanely complex differential equation with some basic algebra, and your final expression in t - with component constants, and some exponentials that appear after the inverse transform - accurately models how the circuit responds over time. There's also a related fairly simple trick for converting the s-domain representation into a frequency/phase plot which tells you how the circuit operates in the frequency domain. And another related fairly simple trick for converting the continuous s-domain into the z-domain for DSP calculations over a sampled time series. Because the same theory also applies in other domains - spring/mass systems, and so on - you can use the same technique there too.
- barbecue_sauce 6y ago
  >from someone who only had a Laplace 101 course Laplace transforms are an entire course?
  - jusssi 6y ago
    From my time at the uni, I wish we'd had a proper course that covered Laplace (and the important special cases, e.g. Fourier and z) transforms properly. Instead, the coverage was interspersed to general math courses and to the courses that needed to apply them.
  - Koshkin 6y ago
    Appears as an exercise on page 505 of this (excellent and freely downloadable, at least for now) book: https://link.springer.com/book/10.1007%2F978-3-319-01195-0 https://link.springer.com/book/10.1007%2F978-3-319-01195-0.
  - steve76 6y ago
    Laplace transforms are an entire industry. They are much more than just a simple trick. To hell with the passivity of myself in nature. Time to grow up. Everything really is PID control. I will have zero surprise if gravitational applications, popping a particle into existence and using that event to get all the data in the universe, use Laplace. Good measurement of - relativity to jolt - momentum to acceleration - capacitance of viscosity - resistance from elasticity is all you just really need for total sovereignty of everything. Quantum theory would give you the odds of not messing up which would be something like 99.9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999% Part of my education was done with a nod and wink by people with some pretty high up security clearances, as in, we have all this here on the shelf. Foreign treaties prevent us from announcing it. Please for the love of god wake up and figure it all out yourself it's really not that difficult to do.
- contravariant 6y ago
  Something also worth mentioning is that it isn't just useful when dealing with differentiation / anti-differentiation but also when dealing with convolution.
- steve76 6y ago
  I think of Laplace as being the non-passive approach: https://en.wikipedia.org/wiki/Coherent_control#Controllability https://en.wikipedia.org/wiki/Coherent_control#Controllabili...
cesarosum 6y ago
rollulus gave a good summary of Laplace transforms and what they do. For some more context, they appear regularly in applied probability (e.g. finance, insurance, physical models including dams). A typical problem is dealing with sums of non-negative random variables. Let's say you want the distribution of n independent copies of a non-negative random variable with distribution function F. The hard way is the n-fold convolution or essentially evaluating an n-dimensional integral. The easy way is using the Laplace transform of F and simply raising it to the power of n. The result isn't always invertible analytically, but you can almost always invert it numerically and this is why techniques like the one outlined in the paper are so important. This is a fantastic post and I thoroughly recommend reading it and the 2019 paper that summarises all their work for several reasons: 1. Very clear exposition of previous work and their own. 2. Clear evaluation metrics. 3. They've even made it easy for you to replicate their work and results.
jiggawatts 6y ago
I tried to get the Mathematica version to do something useful, but in typical "mathematician minimalist style" it squeezed everything into one enormous, terse, but useless blob. The code on GitHub works as a demo only. Even after trying a few different syntax variations for about twenty minutes, I couldn't figure out how to feed the "ILT" function something that would spit out what I expect. In case the original authors ever come across this article: There is a standard for writing Mathematica modules! Please take a look at some other modules available online, and see how they split the code into small, reusable functions. Even the name is too short. In the Mathematica naming convention it should be called "InverseLaplaceTransformCME[...]" or something like that. Ideally, use the same calling convention as the built-in function, documented here: https://reference.wolfram.com/language/ref/InverseLaplaceTransform.html https://reference.wolfram.com/language/ref/InverseLaplaceTra... This would allow your function to be a drop-in replacement, allowing users to switch between the symbolic and approximate versions trivially. You may even want to contact Wolfram Research! They just implemented a new "Asymptotics" module that includes approximate inverse Laplace transforms as a feature. See: https://reference.wolfram.com/language/guide/Asymptotics.html https://reference.wolfram.com/language/guide/Asymptotics.htm... They might add your approach into the 12.2 release, which would mean that many thousands of people could automatically benefit from your hard work!
- gspr 6y ago
  > in typical "mathematician minimalist style" it squeezed everything into one enormous, terse, but useless blob You're complaining that it's both too minimalist and too enormous?
  - whatshisface 6y ago
    Imagine War and Peace, but every word is abbreviated like it's written in shorthand.
    - gspr 6y ago
      Fair enough. But also not a problem if the intended recipient is well-versed in that shorthand. An important difference is that a novel doesn't typically require a specialized skillset, but a mathematical development does.
turbinerneiter 6y ago
This is how papers should be, link to GitHub, interactive demo. Awesome stuff.
- peterburkimsher 6y ago
  I feel like this is what Tim Berners-Lee imagined the World Wide Web to be: sharing knowledge and research with interactive media and hypertext, instead of printed papers. It found new applications outside academia, but this site is probably close to the original idea.
- koheripbal 6y ago
  It's actually pretty exciting to see how well it performs interactively. This might really spark some interesting breakthroughs that I am not smart enough to predict. Core math breakthroughs like this have huge and unpredictable knock-on advancements...
MaxBarraclough 6y ago
Excellent presentation of the material. Not really my, well, domain (sorry), so my only contribution is that there's a spelling error in the dropdown: it refers to the Heaviside step function as the 'Heavyside' function.
signa11 6y ago
for an introduction to the whole topic of laplace-transforms: the venerable 3b1b https://www.youtube.com/watch?v=6MXMDrs6ZmA https://www.youtube.com/watch?v=6MXMDrs6ZmA i found this (https://johnflux.com/2019/02/12/laplace-transform-visualized/ https://johnflux.com/2019/02/12/laplace-transform-visualized...) to be pretty cool as well.
punnerud 6y ago
One of the big breakthroughs is Machine Learning/Neural Networks (NN) is to use the derivative of the error to update the weights of the network (backpropagation). Thinking if CME could be used to avoid local min/max in some way, to speed up the training process.
dzdt 6y ago
This is an interesting promotion of an applied math result. From their promotional material it looks promising, though the unusual promotional approach makes me worry. The actual paper is at https://www.sciencedirect.com/science/article/pii/S0166531619302457/pdfft?md5=7de20355de7ca3e047e645006fa812f2&pid=1-s2.0-S0166531619302457-main.pdf https://www.sciencedirect.com/science/article/pii/S016653161.... This is a pretty obscure journal. The paper is pretty "soft" -- lots of numerical testing of their approach vs. other well-known approaches and not very much theoretical analysis of convergence rates or such. The main claim seems to be that their approach has better numerical properties for discontinuous functions and that it can be effectively implemented to high order using double precision arithmetic.
- mratsim 6y ago
  I'd rather have an applied paper have tests, comparisons and source code than lots of theory and being hard to reproduce because "implementation details" don't appear in the paper. Thanks the authors for putting the code out there for anyone to reproduce and not fall into the unreproduceable "science" that is plaguing us at the moment[1]. [1]: http://polaris.imag.fr/arnaud.legrand/teaching/2016/mosig_smpe_2_reproducible_research.pdf http://polaris.imag.fr/arnaud.legrand/teaching/2016/mosig_sm...
- JorgeGT 6y ago
  Why do you say they are not associated? They seem part of a research group of the Technical University of Budapest, looking at the papers affiliations. As a researcher I really appreciate the promotion effort, some years ago I came across a similar "landing page" for a numerical technique that helped me a lot: http://people.ece.umn.edu/users/mihailo/software/dmdsp/ http://people.ece.umn.edu/users/mihailo/software/dmdsp/, Trying to put together how a new numerical method works scouring for papers with different nomenclatures, different sets of authors, different implementations etc. is often a huge pain. I wish these "landing pages" became a standard, or that a standard repository for them became available. Something like, this is our technique, these are the relevant papers, and here is some demo code.
  - dzdt 6y ago
nisuni 6y ago
Inverting the Laplace transform is a central problem in computational physics, since it connects imaginary-time results (easier to obtain numerically) to real-time response. Over the years a number of approaches have been developed for the inverse Laplace transform, such as MaxEnt, GIFT and many others. I would love to see how this new approach fares against those.
credit_guy 6y ago
Here's a little ELI5 about the Laplace and inverse Laplace transform, and why the inverse transfrom is fiendishly difficult, and therefore why this result is extraordinarily important. Imagine you win the Megabucks lottery. The win is one hundred million dollars. You go to claim your money, but you are told you can choose between the full amount given in monthly payments over 20 years, or a lump sum. But the lump sum is not the full $100MM, it is the present value of the monthly payments discounted at a rate of 5%. To discount an amount received 10 years from now at the 5% rate, you simply divide by 1.05 ^ 10, which is very close to exp(0.05 x 10). If you actually calculate this present value using the exponential function, you say that you use "continuously compounded rates". So, for any stream of future cashflows one can calculate the present value by multiplying the cashflows with appropriate discount factors (of the type exp(-r t)) and adding them up. For different discounting rates r you obtain different present values. This present value as a function of r is the Laplace transform of the cashflow stream as a function of t. The inverse Laplace transform is solving the riddle: if I tell you the present value (PV) of some cashflows for any (positive) discount rate you want, can you calculate the cashflows? Why is this a difficult problem? Because it is "ill-conditioned". Imagine the following two cashflow streams: in the first you get $1MM every year for the next 10 years and another $1MM one hundred years from now. In the second you also get $1MM annually for the first ten years but the last $1MM is 101 years from now. For a zero discount rate the value of both cashstreams is $11MM. For a 5% they are both around $9MM and different by about $300, which is about 0.003%. For any discount rate the PV's will be very very close. In some cases "in real life" this closeness could be below machine precision level. If someone gives you 2 sets of inputs where their Laplace transforms are different by less than the machine precision levels for all values of the discount rate, then there is no hope to tell them apart knowing only their trasforms only, at least not if you don't use some multiple precision libraries. That should give you an intuition why the inverse Laplace transform is nasty. All hope is not lost though. First of all, in a typical application the Laplace transform of a function is known in closed (analytical) form, so you can actually use multiple precision libraries if you so wish. I have seen cases where people were using precision of 2000 digits in Mathematica for this. It's slow as hell, but it gets the job done. Separately, you are free to calculate the Laplace transform at any "discount rate", including complex values. If you are smart about how to choose these values, you can come up with good recipes for the Laplace transform. For hundreds of years now, the general wisdom was that various inverse numerical Laplace transform algorithms have strengths and weaknesses, but no single one is universally good. Maybe this one will be, and if so it will be indeed revolutionary.
vmchale 6y ago
this is a lot of energy for an inverse laplace transform
nebukadnezar 6y ago
How does this new algorithm compare to https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1355451 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1355451 ?
nabla9 6y ago
If you are familiar with Fourier transform, then Fourier transform: sinusoidals Laplace transform: sinusoidals + exponentials Here is nice video explaining it: https://www.youtube.com/watch?v=n2y7n6jw5d0 https://www.youtube.com/watch?v=n2y7n6jw5d0
dls2016 6y ago
I've never understood the use of the Laplace transform. Perhaps that's due to my mathematical exposure (theoretical qualitative analysis of pdes). Since the Laplace transform lacks the duality of the Fourier transform, it doesn't seem to have a place in research mathematics. But I probably think of a dozen fundamental uses of the Fourier transform, from Bourgain spaces to evaluating oscillatory integrals. And if you're working on some manifold with curvature then you generally need to be familiar with the eigenfunctions of the Laplacian on that manifold... not the basis of the Laplace transform. I also know a bit of signal processing\numerical analysis, and I'm not familiar with any practical uses of the Laplace transform there. I don't believe it's used in the numerical solution of pdes or odes, whereas spectral methods are a huge area of study and (until recently, I think) were used in the GFS weather model. And most time series analysis tools either apply the Fourier transform or bail out of this approach and use statistical tools. My version of Greenspun's 10th rule goes: any sufficiently complex program includes an FFT. Can anyone help me out here? Is there a problem/theorem the Laplace transform solves/proves which the Fourier transform doesn't?
- chessweb01 6y ago
  https://duckduckgo.com/?t=ffab&q=applications+laplace+transform&atb=v100-1&ia=web https://duckduckgo.com/?t=ffab&q=applications+laplace+transf...
  - dls2016 6y ago
    har har But seriously, do you have a favorite example where the Laplace transform is used to prove a theorem or used in practice to solve a problem? I'm familiar with the undergraduate differential equations examples. But there are plenty of things taught at the undergraduate level which are tractable and helpful to build intuition but either a) aren't important from a research perspective or b) aren't used in practice. The Fourier transform has both.
    - SAI_Peregrinus 6y ago
      All the time in AC circuits. Especially for anything RF-related. It's vastly, vastly easier to work in the (complex) frequency domain. Antenna and filter design are pretty much all done in the s-domain.
GolDDranks 6y ago
Just spitballing here, about an application of the Laplace transform. We have a product that allows the users to use machine learning in a semi-automatized way, without deeply understanding hyperparameter optimization, model testing, selection and evaluation and such. There was some talk about supporting the prediction of time-series data. I have absolutely no knowledge of how time-series data should be pre-processed and what kind of algorithms are common or applicable in general. (I'm not in charge of the R&D of the data-science-y features) However, it seems like Laplace transform as a pre-processing step ticks a lot of the checkboxes. As a superset of Fourier, it supports periodic changes in time series, and being about exponentials, it also allows for growth (or decreasing) over time, allowing to transform a time series to data that is more applicable to classical ML algorithms. Is Laplace transform actually used for such usecases?
- cogman10 6y ago
  IDK, but Fourier, and specifically the more specialized, the DCT certainly is. Part of the reason for this is because the algorithms to go from discrete data points into a wave form are fairly well known and fast. DCT is the foundation for most Lossy encoding formats. Using it for time series data makes a lot of sense, especially if you are optimizing for storage space.
hilles 6y ago
Hi! One of the authors here. Thank you for the exposure and feedback! We really appreciate it. About the code. We have added comments and simple running examples to the code on github. Hopefully that helps make the code more accessible to everyone. About the contribution. Classic numerical inverse Laplace transformation methods work in some cases but fail in others, while the CME method always gives a good approximation at low computational cost. We recommend it for general use when you just want to invert a function numerically without spending effort to figure out what methods might be applicable.
- no_identd 6y ago
  This story is six days old. Don't expect a lot of replies to your comment to come, but know that your comment & more importantly the improvements (and ofc the result!) get appreciated.

233 points by pabo 6y ago

What could be the use cases? Also visually it seems so simple, like for the Mish activation function https://github.com/digantamisra98/Mish/blob/master/README.md https://github.com/digantamisra98/Mish/blob/master/README.md It seems to be advanced maths but I wonder why the designer (he already know in advance the desired form of the function in order to give him a desirable property (in both cases being more smoothed / continuous / centered)) does not draw graphically the desired function and let a software solve, find automatically the best approximation of the function? EDIT: well it seems to be a general function approximator so my point doesn't apply (but still apply for the new activation functions in machine learning)

rollulus 6y ago

A little ELI5 for those who haven't had Laplace transforms at school, from someone who only had a Laplace 101 course, so for what's it worth: Laplace transforms allow you to convert differential equations into easier equations, and back: the differentials and integrals become multiplications and divisions. So you can take a differential equation, transform it into the Laplace domain, manipulate it, and convert it back. And that's cool because differential equations tend to appear everywhere, for instance to model springs, electrical circuits with caps and coils, the surface of a soap bubble in a metal rod, etc. A sibling is the z-transform, which is like the digital version. This one is used for instance to design digital audio filters. I'm sure some math wizards here can elaborate and correct me.

cm2187 6y ago
And don’t forget those like me for whom school is a distant memory! What are the domains where this new method can be applied? Is it mostly physics simulations and the likes?
- timclark 6y ago
  I got taught them in a course on linear systems which was a pre-requisite course to control theory. Lots of electrical circuits, mechanical systems and electro-mechanical systems can be modelled using laplace transforms if they are linear systems. I did an electrical and electronic engineering degree and we got to skip the tedious differential equation solving lectures that the mechanical, civil and chemical engineers had to attend because of Monsieur Laplace.
- srean 6y ago
  Control systems, signal filters (noise attenuation), modeling epidemics, modeling queues, modeling reliability of repairable systems, modeling recurrent events(such as failures), renewal processes, modeling inventory plans, probability in general (because of the connection with moment generating function) ...
GolDDranks 6y ago
I don't have any real understanding over the Laplace transform, but I understand Fourier transform well enough that it makes sense to me. Back then, I saw an claim that Laplace transform is a generalization of Fourer transform in the sense that it transforms a function not only to a space of frequencies and phases of sine waves, but to a larger space of parameters of exponentials. Note that the parameter space of the sine waves is subset of the (complex) parameter space of exponentials. Is this claim correct?
- beagle3 6y ago
  Yes. The Fourier transform characterizes on a circle (usually the unit circle in the complex domain) at different frequencies. It is properly defined for periodic signals. The Laplace transform takes any exponential spiral in the complex plane, and reduces to the Fourier transform if you only care about the unit circle. I appreciate that doesn’t make things clearer unless you already have some understanding of integral transforms in the complex plane (in which case, you probably know this already). However, I have never met a simple intuitive explanation of the Laplace transform, - and actually no meaningful explanation that doesn’t involve integrals.
  - durbat 6y ago
    With Fourier you can analyze oscillatory characteristics of function (frequency and phase). With Laplace you can also analyze amplification/attenuation.
    - tilt_error 6y ago
      The Fourier transform, well, transforms a time-based phenomenon such as an alternating current sine wave into a frequency spectrum where you can observe the frequency spectrum components of the signal. A pure nice sine becomes a spike (delta function) located at a specific frequency. Music, as we observe it through our ears and can view it on an oscilloscope becomes moving spikes (lots of them :) in the frequency spectrum where the "amplitude" at a given frequency relates to the "amount" of that frequency in the music. The behaviour of a filter is much easier to describe in the frequency spectral domain than it would be in the time domain. Now to the direct current (DC) view. This cannot be handled by the Fourier transform -- at least the DC-part of the signal cannot be transformed to the frequency domain. As shown in the article, there were "steps", "ramps" and such. A typical scenario would be to describe what happens in your amplifier during startup, to describe how electrical circuits are behaving during startup before reaching the "running" state. The Laplace transform will handle these types of scenarios, and can thus be used to study (or describe) systems during other types of transitions than the "steady state" when you are up and running. Regarding filters, the Fourier transform describes things going on at the unit circle, while the Laplace transform can be used to study both the interior and exterior of the plane. In this sense, creating filters relates to locate "poles" and "zeros" in the plane (amplification and attenuation) which can be observed on the unit circle as the behaviour on periodic signals.
taneq 6y ago
Sounds right to my (very very rusty) recollection. Laplace transforms are a magic trick that let you easily solve some kinds of differential equations.
- phkahler 6y ago
  >> Laplace transforms are a magic trick that let you easily solve some kinds of differential equations. To mathematicians I don't think they're so much magic. When I took differential equations class it was frustrating that they went too fast for me to fully digest what was "really" going on. It didn't feel out of reach, but something I needed to look at a couple different ways but didn't have time (or the internet) to do so. Think I'm gonna checkout 3blue1brown after this - he can probably close that gap for me.
  - scythmic_waves 6y ago
    > Think I'm gonna checkout 3blue1brown after this - he can probably close that gap for me. You might like this lecture from MIT's OCW: [1]. It's my favorite source for motivating the Laplace transform. It's a bit difficult to make this concept "simple", and this resource assumes that you already have some familiarity with the following concepts: infinite series, power series, radius of convergence, and (indefinite) integration. The tl;dw is that the Laplace transform is a generalization of a power series. [1] https://www.youtube.com/watch?v=sZ2qulI6GEk https://www.youtube.com/watch?v=sZ2qulI6GEk Edit: I also wrote up a form of this video elsewhere if anyone's interested. It's kinda long though, and I didn't want to spam this thread with it.
kpmah 6y ago
I've had some problems understanding the Laplace transform. Maybe somebody here can point me towards some material. I have an interest understanding how IIR filters are designed, and I always get stuck at this part in DSP books. The Laplace transform is used, but as well as finding the mathematics difficut I don't really understand why it is being used at all. I think it is trying to replicate the effect of an analog circuit?
- Gibbon1 6y ago
  You can describe a circuit by it's time domain behavior. Or you can describe the circuit by it's frequency domain behavior. Both are valid and congruent. The thing is a lot of questions are easy to answer in the frequency domain. For instance, you want to know if a circuit with feedback will oscillate. Hard to answer using time domain equations. But in the frequency domain there is a simple constraint. If for all frequencies where the the gain is greater than one the phase shift is less than 180 degrees, circuit won't oscillate. This is obviously rather useful. Also a point with a lot of 'books' the authors get caught up in describing how something is done that they never explain why something is done. I've found often the answer is simple yet opaque and frustratingly never talked about.
  - pantulis 6y ago
    This. I remember having adequate cursory knowledge of Fourier Transform to the point of understanding the value of FFT algorithms, but the Laplace Transform was explained like hell so I failed my robotics classes.
    - TheOtherHobbes 6y ago
      If you have an electronic circuit, you can model each element with a differential equation. E.g. voltage across a capacitor is modelled as the integral of current, voltage across an inductor is dI/dt. This is a useful fact for a simple circuit in a classroom, but the differential equations for any circuit with more than a few components soon become insanely complex. With the Laplace transform you (more or less) replace an integral with 1/s and a differential with s, plus some constants derived from the component values. Then you can simplify for s, and use the Inverse Laplace Transform to convert the final expression in s into an expression in t. You have now solved an insanely complex differential equation with some basic algebra, and your final expression in t - with component constants, and some exponentials that appear after the inverse transform - accurately models how the circuit responds over time. There's also a related fairly simple trick for converting the s-domain representation into a frequency/phase plot which tells you how the circuit operates in the frequency domain. And another related fairly simple trick for converting the continuous s-domain into the z-domain for DSP calculations over a sampled time series. Because the same theory also applies in other domains - spring/mass systems, and so on - you can use the same technique there too.
barbecue_sauce 6y ago
>from someone who only had a Laplace 101 course Laplace transforms are an entire course?
- jusssi 6y ago
  From my time at the uni, I wish we'd had a proper course that covered Laplace (and the important special cases, e.g. Fourier and z) transforms properly. Instead, the coverage was interspersed to general math courses and to the courses that needed to apply them.
- Koshkin 6y ago
  Appears as an exercise on page 505 of this (excellent and freely downloadable, at least for now) book: https://link.springer.com/book/10.1007%2F978-3-319-01195-0 https://link.springer.com/book/10.1007%2F978-3-319-01195-0.
- steve76 6y ago
  Laplace transforms are an entire industry. They are much more than just a simple trick. To hell with the passivity of myself in nature. Time to grow up. Everything really is PID control. I will have zero surprise if gravitational applications, popping a particle into existence and using that event to get all the data in the universe, use Laplace. Good measurement of - relativity to jolt - momentum to acceleration - capacitance of viscosity - resistance from elasticity is all you just really need for total sovereignty of everything. Quantum theory would give you the odds of not messing up which would be something like 99.9999999999999999999999999999999999999999999999999999999999999999999999999999999999999999% Part of my education was done with a nod and wink by people with some pretty high up security clearances, as in, we have all this here on the shelf. Foreign treaties prevent us from announcing it. Please for the love of god wake up and figure it all out yourself it's really not that difficult to do.
contravariant 6y ago
Something also worth mentioning is that it isn't just useful when dealing with differentiation / anti-differentiation but also when dealing with convolution.
steve76 6y ago
I think of Laplace as being the non-passive approach: https://en.wikipedia.org/wiki/Coherent_control#Controllability https://en.wikipedia.org/wiki/Coherent_control#Controllabili...

cesarosum 6y ago

rollulus gave a good summary of Laplace transforms and what they do. For some more context, they appear regularly in applied probability (e.g. finance, insurance, physical models including dams). A typical problem is dealing with sums of non-negative random variables. Let's say you want the distribution of n independent copies of a non-negative random variable with distribution function F. The hard way is the n-fold convolution or essentially evaluating an n-dimensional integral. The easy way is using the Laplace transform of F and simply raising it to the power of n. The result isn't always invertible analytically, but you can almost always invert it numerically and this is why techniques like the one outlined in the paper are so important. This is a fantastic post and I thoroughly recommend reading it and the 2019 paper that summarises all their work for several reasons: 1. Very clear exposition of previous work and their own. 2. Clear evaluation metrics. 3. They've even made it easy for you to replicate their work and results.

jiggawatts 6y ago

I tried to get the Mathematica version to do something useful, but in typical "mathematician minimalist style" it squeezed everything into one enormous, terse, but useless blob. The code on GitHub works as a demo only. Even after trying a few different syntax variations for about twenty minutes, I couldn't figure out how to feed the "ILT" function something that would spit out what I expect. In case the original authors ever come across this article: There is a standard for writing Mathematica modules! Please take a look at some other modules available online, and see how they split the code into small, reusable functions. Even the name is too short. In the Mathematica naming convention it should be called "InverseLaplaceTransformCME[...]" or something like that. Ideally, use the same calling convention as the built-in function, documented here: https://reference.wolfram.com/language/ref/InverseLaplaceTransform.html https://reference.wolfram.com/language/ref/InverseLaplaceTra... This would allow your function to be a drop-in replacement, allowing users to switch between the symbolic and approximate versions trivially. You may even want to contact Wolfram Research! They just implemented a new "Asymptotics" module that includes approximate inverse Laplace transforms as a feature. See: https://reference.wolfram.com/language/guide/Asymptotics.html https://reference.wolfram.com/language/guide/Asymptotics.htm... They might add your approach into the 12.2 release, which would mean that many thousands of people could automatically benefit from your hard work!

gspr 6y ago
> in typical "mathematician minimalist style" it squeezed everything into one enormous, terse, but useless blob You're complaining that it's both too minimalist and too enormous?
- whatshisface 6y ago
  Imagine War and Peace, but every word is abbreviated like it's written in shorthand.
  - gspr 6y ago
    Fair enough. But also not a problem if the intended recipient is well-versed in that shorthand. An important difference is that a novel doesn't typically require a specialized skillset, but a mathematical development does.

turbinerneiter 6y ago

This is how papers should be, link to GitHub, interactive demo. Awesome stuff.

peterburkimsher 6y ago
I feel like this is what Tim Berners-Lee imagined the World Wide Web to be: sharing knowledge and research with interactive media and hypertext, instead of printed papers. It found new applications outside academia, but this site is probably close to the original idea.
koheripbal 6y ago
It's actually pretty exciting to see how well it performs interactively. This might really spark some interesting breakthroughs that I am not smart enough to predict. Core math breakthroughs like this have huge and unpredictable knock-on advancements...

MaxBarraclough 6y ago

Excellent presentation of the material. Not really my, well, domain (sorry), so my only contribution is that there's a spelling error in the dropdown: it refers to the Heaviside step function as the 'Heavyside' function.

signa11 6y ago

for an introduction to the whole topic of laplace-transforms: the venerable 3b1b https://www.youtube.com/watch?v=6MXMDrs6ZmA https://www.youtube.com/watch?v=6MXMDrs6ZmA i found this (https://johnflux.com/2019/02/12/laplace-transform-visualized/ https://johnflux.com/2019/02/12/laplace-transform-visualized...) to be pretty cool as well.

punnerud 6y ago

One of the big breakthroughs is Machine Learning/Neural Networks (NN) is to use the derivative of the error to update the weights of the network (backpropagation). Thinking if CME could be used to avoid local min/max in some way, to speed up the training process.

dzdt 6y ago

This is an interesting promotion of an applied math result. From their promotional material it looks promising, though the unusual promotional approach makes me worry. The actual paper is at https://www.sciencedirect.com/science/article/pii/S0166531619302457/pdfft?md5=7de20355de7ca3e047e645006fa812f2&pid=1-s2.0-S0166531619302457-main.pdf https://www.sciencedirect.com/science/article/pii/S016653161.... This is a pretty obscure journal. The paper is pretty "soft" -- lots of numerical testing of their approach vs. other well-known approaches and not very much theoretical analysis of convergence rates or such. The main claim seems to be that their approach has better numerical properties for discontinuous functions and that it can be effectively implemented to high order using double precision arithmetic.

mratsim 6y ago
I'd rather have an applied paper have tests, comparisons and source code than lots of theory and being hard to reproduce because "implementation details" don't appear in the paper. Thanks the authors for putting the code out there for anyone to reproduce and not fall into the unreproduceable "science" that is plaguing us at the moment[1]. [1]: http://polaris.imag.fr/arnaud.legrand/teaching/2016/mosig_smpe_2_reproducible_research.pdf http://polaris.imag.fr/arnaud.legrand/teaching/2016/mosig_sm...
JorgeGT 6y ago
Why do you say they are not associated? They seem part of a research group of the Technical University of Budapest, looking at the papers affiliations. As a researcher I really appreciate the promotion effort, some years ago I came across a similar "landing page" for a numerical technique that helped me a lot: http://people.ece.umn.edu/users/mihailo/software/dmdsp/ http://people.ece.umn.edu/users/mihailo/software/dmdsp/, Trying to put together how a new numerical method works scouring for papers with different nomenclatures, different sets of authors, different implementations etc. is often a huge pain. I wish these "landing pages" became a standard, or that a standard repository for them became available. Something like, this is our technique, these are the relevant papers, and here is some demo code.
- dzdt 6y ago

nisuni 6y ago

Inverting the Laplace transform is a central problem in computational physics, since it connects imaginary-time results (easier to obtain numerically) to real-time response. Over the years a number of approaches have been developed for the inverse Laplace transform, such as MaxEnt, GIFT and many others. I would love to see how this new approach fares against those.

credit_guy 6y ago

Here's a little ELI5 about the Laplace and inverse Laplace transform, and why the inverse transfrom is fiendishly difficult, and therefore why this result is extraordinarily important. Imagine you win the Megabucks lottery. The win is one hundred million dollars. You go to claim your money, but you are told you can choose between the full amount given in monthly payments over 20 years, or a lump sum. But the lump sum is not the full $100MM, it is the present value of the monthly payments discounted at a rate of 5%. To discount an amount received 10 years from now at the 5% rate, you simply divide by 1.05 ^ 10, which is very close to exp(0.05 x 10). If you actually calculate this present value using the exponential function, you say that you use "continuously compounded rates". So, for any stream of future cashflows one can calculate the present value by multiplying the cashflows with appropriate discount factors (of the type exp(-r t)) and adding them up. For different discounting rates r you obtain different present values. This present value as a function of r is the Laplace transform of the cashflow stream as a function of t. The inverse Laplace transform is solving the riddle: if I tell you the present value (PV) of some cashflows for any (positive) discount rate you want, can you calculate the cashflows? Why is this a difficult problem? Because it is "ill-conditioned". Imagine the following two cashflow streams: in the first you get $1MM every year for the next 10 years and another $1MM one hundred years from now. In the second you also get $1MM annually for the first ten years but the last $1MM is 101 years from now. For a zero discount rate the value of both cashstreams is $11MM. For a 5% they are both around $9MM and different by about $300, which is about 0.003%. For any discount rate the PV's will be very very close. In some cases "in real life" this closeness could be below machine precision level. If someone gives you 2 sets of inputs where their Laplace transforms are different by less than the machine precision levels for all values of the discount rate, then there is no hope to tell them apart knowing only their trasforms only, at least not if you don't use some multiple precision libraries. That should give you an intuition why the inverse Laplace transform is nasty. All hope is not lost though. First of all, in a typical application the Laplace transform of a function is known in closed (analytical) form, so you can actually use multiple precision libraries if you so wish. I have seen cases where people were using precision of 2000 digits in Mathematica for this. It's slow as hell, but it gets the job done. Separately, you are free to calculate the Laplace transform at any "discount rate", including complex values. If you are smart about how to choose these values, you can come up with good recipes for the Laplace transform. For hundreds of years now, the general wisdom was that various inverse numerical Laplace transform algorithms have strengths and weaknesses, but no single one is universally good. Maybe this one will be, and if so it will be indeed revolutionary.

vmchale 6y ago

this is a lot of energy for an inverse laplace transform

nebukadnezar 6y ago

How does this new algorithm compare to https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1355451 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1355451 ?

nabla9 6y ago

If you are familiar with Fourier transform, then Fourier transform: sinusoidals Laplace transform: sinusoidals + exponentials Here is nice video explaining it: https://www.youtube.com/watch?v=n2y7n6jw5d0 https://www.youtube.com/watch?v=n2y7n6jw5d0

dls2016 6y ago

I've never understood the use of the Laplace transform. Perhaps that's due to my mathematical exposure (theoretical qualitative analysis of pdes). Since the Laplace transform lacks the duality of the Fourier transform, it doesn't seem to have a place in research mathematics. But I probably think of a dozen fundamental uses of the Fourier transform, from Bourgain spaces to evaluating oscillatory integrals. And if you're working on some manifold with curvature then you generally need to be familiar with the eigenfunctions of the Laplacian on that manifold... not the basis of the Laplace transform. I also know a bit of signal processing\numerical analysis, and I'm not familiar with any practical uses of the Laplace transform there. I don't believe it's used in the numerical solution of pdes or odes, whereas spectral methods are a huge area of study and (until recently, I think) were used in the GFS weather model. And most time series analysis tools either apply the Fourier transform or bail out of this approach and use statistical tools. My version of Greenspun's 10th rule goes: any sufficiently complex program includes an FFT. Can anyone help me out here? Is there a problem/theorem the Laplace transform solves/proves which the Fourier transform doesn't?

chessweb01 6y ago
https://duckduckgo.com/?t=ffab&q=applications+laplace+transform&atb=v100-1&ia=web https://duckduckgo.com/?t=ffab&q=applications+laplace+transf...
- dls2016 6y ago
  har har But seriously, do you have a favorite example where the Laplace transform is used to prove a theorem or used in practice to solve a problem? I'm familiar with the undergraduate differential equations examples. But there are plenty of things taught at the undergraduate level which are tractable and helpful to build intuition but either a) aren't important from a research perspective or b) aren't used in practice. The Fourier transform has both.
  - SAI_Peregrinus 6y ago
    All the time in AC circuits. Especially for anything RF-related. It's vastly, vastly easier to work in the (complex) frequency domain. Antenna and filter design are pretty much all done in the s-domain.

GolDDranks 6y ago

Just spitballing here, about an application of the Laplace transform. We have a product that allows the users to use machine learning in a semi-automatized way, without deeply understanding hyperparameter optimization, model testing, selection and evaluation and such. There was some talk about supporting the prediction of time-series data. I have absolutely no knowledge of how time-series data should be pre-processed and what kind of algorithms are common or applicable in general. (I'm not in charge of the R&D of the data-science-y features) However, it seems like Laplace transform as a pre-processing step ticks a lot of the checkboxes. As a superset of Fourier, it supports periodic changes in time series, and being about exponentials, it also allows for growth (or decreasing) over time, allowing to transform a time series to data that is more applicable to classical ML algorithms. Is Laplace transform actually used for such usecases?

cogman10 6y ago
IDK, but Fourier, and specifically the more specialized, the DCT certainly is. Part of the reason for this is because the algorithms to go from discrete data points into a wave form are fairly well known and fast. DCT is the foundation for most Lossy encoding formats. Using it for time series data makes a lot of sense, especially if you are optimizing for storage space.

hilles 6y ago

Hi! One of the authors here. Thank you for the exposure and feedback! We really appreciate it. About the code. We have added comments and simple running examples to the code on github. Hopefully that helps make the code more accessible to everyone. About the contribution. Classic numerical inverse Laplace transformation methods work in some cases but fail in others, while the CME method always gives a good approximation at low computational cost. We recommend it for general use when you just want to invert a function numerically without spending effort to figure out what methods might be applicable.

no_identd 6y ago
This story is six days old. Don't expect a lot of replies to your comment to come, but know that your comment & more importantly the improvements (and ofc the result!) get appreciated.