4 ms·
From page 6, when they describe their synthetic textbook dataset: "Consider the matrix A = np.array([[1, 2], [2, 4]]). We can check if this matrix is singular
by cafaxo 3y ago
From page 6, when they describe their synthetic textbook dataset:
"Consider the matrix A = np.array([[1, 2], [2, 4]]). We can check if this matrix is singular or nonsingular using the determinant function. [...]"
No. The determinant is not a suitable way to do that. A proper way to numerically measure singularity would be to compute the condition number of the matrix (the ratio of its largest to smallest singular value).
- aix1 3y agoAm I misreading this, or is it really about a 2x2 matrix? (In which case computing the determinant involves just two multiplications and one subtraction.)
- cafaxo 3y agoThey define a generic "is_singular" function and test it with a 2x2 matrix. The problem with the determinant is not about performance. It is just useless for determining if a matrix is singular. The thing that gives it away is that the determinant is influenced by a rescaling of the matrix: det(s A) = s^n det(A) where A is a n x n matrix As an example, would you say that [[1e-10, 0], [0, 1e-10]] is singular? It has condition number 1.
- renonce 3y agoIt does work in theory and for integer numbers. Such an algorithm might be used in practice when numerical stability is concerned, but condition number can certainly be defined somewhere in another textbook and you just need more context to tell it to use another method.
- cafaxo 3y agoEdit: Sorry, I completely messed up my original answer here. A better version: Let's say we are in a setting where we only work with integers. A matrix is invertible iff its determinant is invertible in the underlying ring. The only invertible elements in Z are -1 and 1. So, the code is also incorrect in the integer setting. Here, we should not check for 0, but for -1 or 1.
- thaumasiotes 3y agoIf I'm reading you correctly, you'd also need to flip the response to the check: where the original test for 0 determines that a matrix is singular if it does find 0, the new test for ±1 should determine that a matrix is singular if it does not find ±1?
- cafaxo 3y agoYes, exactly.
- TwentyPosts 3y agoSaying that the determinant is useless to determine whether a matrix is incorrect, or misleading at best. This is purely a matter of numerical stability. Of course [[1e-10, 0], [0, 1e-10]] is nonsingular, and its determinant is 1e-20, which does not equal zero. Yes, when it comes to floating point issues we might want to use something else, and that's a valid complaint when it comes to NumPy code, but from a theoretical perspective the determinant is an excellent tool to determine singularity.
- cafaxo 3y agoOf course. Theoretically, the determinant answers the binary question "singular" or "nonsingular". Numerically, such a binary answer is pretty useless. Here, we need a measure of how singular/nonsingular a matrix is relative to the numerical precision we are working with.
- civilized 3y agoWhile you're correct that condition number is a more robust numerical method for arbitrary matrices, the determinant is certainly suitable for many matrices. For small matrices with small integer values such as this one, there is no issue with the determinant. There is no one bulletproof general method to approximate mathematical calculations with floating point numbers. More context is generally required, including the actual problem that is being approximated, to determine if a method is reliable. Painting this as a black and white situation where the determinant is wrong and the condition number is right gives a misleading picture of how we evaluate numerical methods for fit-to-purpose.