3 ms·

Interesting. The problem of finding a low-rank-approximation (in Frobenius norm) to a given matrix is well understood and solved in reasonable time (truncated

by FabHK 8y ago

Interesting. The problem of finding a low-rank-approximation (in Frobenius norm) to a given matrix is well understood and solved in reasonable time (truncated SVD, as stated in this article). However, the particular characteristic of this NMF approach seems to be that we require W, H >= 0 (and that's what makes it harder), but it's not clear why, what purpose it serves. The original paper linked [1] mentions that it makes interpretation easier (the vectors of W can be interpreted as facial features, or a form of "eigenface"), but it's not clear to me why you can't take a vector with negative elements after norming to [0,1]. Final note: The article contains quite a few paragraphs from the paper more or less verbatim. [1] https://arxiv.org/abs/1401.5226 https://arxiv.org/abs/1401.5226

svantana 8y ago
Some quantities are non-negative in their nature, such as energy, intensity, number of occurrences etc. When modeling data that is assumed to be a sum of such non-negative terms (i.e. the terms cannot cancel out), it can make sense to enforce non-negativity.
stabbles 8y ago
For example: you measure concentrations of `p` gases at `n` different places. You collect the results in a matrix A where A_ij is the concentration (a value between 0 and 1) of gas `i` at station `j`. Now you factorize A ≈ B * C in the sense that ||A - B * C|| is small in your favourite matrix norm. Here B is of size `p × k` with normalized columns and C is `k × n` and of course k is very small. Then the columns of B form an approximate basis for the column space of A -- so B compresses the data of A well. Now, if also B is element-wise positive, you might interpret the columns as concentrations (values between 0 and 1 due to normalization). And you could plot them. It's not mathematical, but it's also not a crazy interpretation.
ivan_ah 8y ago
Intuitively, the purpose of the positive coefficients constraint is to allow for the interpretation as a mixture (admixture?) of components. This constraint gives the nice properties of W. In principle there is no reason why you wouldn't allow negative coefficients in H, but if you want the W to learn additive components, then you need the non-negative constraint.
jononor 8y ago
The non-negative constraint is so that components will only add together. One can never subtract a feature, or add a negative feature. For instance complex sounds can be decomposed into many smaller sound "atoms". Usually this is done in usually spectrogram representation split in small sections of time. With NMF, the weight matrix becomes how much each of the atoms is added to form the sound mixture. The weights can then be used for classification, similarity or synthesis. Allowing to add a negative amount of a sound atom would make interpretation harder. What does it mean for an anti-sound to be present? Destructive interference of waveforms is possible, but quite rare to encompass a whole "atom" in a realistic acoustic scenario. And when not combined with its inverse it just sounds like a sound to the human ear. So its contribution is not always negative either. NMF interpretation is straightforward in comparison.

by FabHK 8y ago

svantana 8y ago

Some quantities are non-negative in their nature, such as energy, intensity, number of occurrences etc. When modeling data that is assumed to be a sum of such non-negative terms (i.e. the terms cannot cancel out), it can make sense to enforce non-negativity.

stabbles 8y ago

For example: you measure concentrations of `p` gases at `n` different places. You collect the results in a matrix A where A_ij is the concentration (a value between 0 and 1) of gas `i` at station `j`. Now you factorize A ≈ B * C in the sense that ||A - B * C|| is small in your favourite matrix norm. Here B is of size `p × k` with normalized columns and C is `k × n` and of course k is very small. Then the columns of B form an approximate basis for the column space of A -- so B compresses the data of A well. Now, if also B is element-wise positive, you might interpret the columns as concentrations (values between 0 and 1 due to normalization). And you could plot them. It's not mathematical, but it's also not a crazy interpretation.

ivan_ah 8y ago

Intuitively, the purpose of the positive coefficients constraint is to allow for the interpretation as a mixture (admixture?) of components. This constraint gives the nice properties of W. In principle there is no reason why you wouldn't allow negative coefficients in H, but if you want the W to learn additive components, then you need the non-negative constraint.

jononor 8y ago

The non-negative constraint is so that components will only add together. One can never subtract a feature, or add a negative feature. For instance complex sounds can be decomposed into many smaller sound "atoms". Usually this is done in usually spectrogram representation split in small sections of time. With NMF, the weight matrix becomes how much each of the atoms is added to form the sound mixture. The weights can then be used for classification, similarity or synthesis. Allowing to add a negative amount of a sound atom would make interpretation harder. What does it mean for an anti-sound to be present? Destructive interference of waveforms is possible, but quite rare to encompass a whole "atom" in a realistic acoustic scenario. And when not combined with its inverse it just sounds like a sound to the human ear. So its contribution is not always negative either. NMF interpretation is straightforward in comparison.