4 ms·
I don't know anything about algebraic geometry or category theory or haskell, but I know some stuff about the normal boring kind of linear algebra, and in some
by ftxbro 4y ago
I don't know anything about algebraic geometry or category theory or haskell, but I know some stuff about the normal boring kind of linear algebra, and in some places the author relies on statements like "since the eigenvalues are real and positive we can conclude that it is a symmetric positive definite matrix" which I know are just wrong (it wouldn't have to be symmetric). This is particularly bad because the rest of the argument (for example in 6.2.2) explains why it's so cool that we've shown it's symmetric (we haven't).
- deleted 4y ago[deleted]
- matheist 4y agoFor example, [[1, 1], [0, 1]] is a non-symmetric matrix with positive real eigenvalues.
- adamnemecek 4y agoYou are right, I will rectify that. The statement still holds but I need to provide a justification.
- adamnemecek 4y agoJust FYI when you lookup visualizations of the attention matrices, they tend to be somewhat symmetric.
- east2west 4y agoI think that sentence is confusing but mathematical conclusion still stands. For quadratic forms, we are only concerned with symmetric matrices, or the symmetric part of any real matrix, because we can always replace any square matrix in a quadratic form, x^T A x, by x^T (A + A^T)/2 x, the symmetric part of the matrix, and the quadratic form still retains the original value. Then, for symmetric matrices, their eigenvalues being positive implies that the matrices are positive definite. Symmetric square matrices have real eigenvalues; one can prove this by taking the complement of quadratic forms.