4 ms·
The question arises: what is the scope of compressive sensing? What you show is l1 regularization. This has been around far longer than the term 'compressive se
by shas3 13y ago
The question arises: what is the scope of compressive sensing? What you show is l1 regularization. This has been around far longer than the term 'compressive sensing'. Even the theory underlying it pre-dates 'compressive sensing' [1]. I think the 'compressive' 'sensing' aspect comes about due to the possibility of literally, 'compressively sensing', not just in using 'l1-regularization'.
Further, an ambiguity arises in the concept of the interactive graphics in the OP website: the graphical explanation shown holds not just for 'l1' regularization, but also for say, 'l2' regularization (and other regularizations). The 'bar' could also represent a scaled version of the l2 norm. It is not easily apparent why it should only be 'l1'.
When it comes to graphical intuition for compressive sensing, I find the 'l2-ball' (intersects 'low-dimensional measurements' at dense points ) vs. 'l1-ball' (intersects 'low-dimensional measurement' equations on sparse points in the plane, thus 'promoting sparsity', see for example, [2]) to be more lucid.
Since you already have the excellent interactive web-site, can you possibly include the l1-ball vs. l2-ball [2] graphic?
[1] R. Tibshirani, "The Lasso page "Regression shrinkage and selection via the lasso". Journal of the Royal Statistical Society, Series B 58 (1): 267–288.
[2] R. Baraniuk, "Compressive sensing." IEEE signal processing magazine 24.4 (2007).
Edit:
Another suggestion: The OP website format is a great way to demonstrate the utility of regularizations in optimization, which has wide applications in machine learning, data interpolation, etc.[3]
[3] http://www.mit.edu/~9.520/spring07/Classes/rlsslides.pdf http://www.mit.edu/~9.520/spring07/Classes/rlsslides.pdf