4 ms·

> Purist test The short answer is. Yes, but if your PL treats it that way you will wind up in trouble [0]. The long answer is: https://www.youtube.com/watch

• tomjakubowski 6y ago
  Besides needing to change the order of multiplication (y = xM vs y = Mx) and use the right transposition of the matrix, do any other actual consequences come from the choice of "row vector" vs "column vector"?
  • dnautics 6y ago
    Mathematically speaking, no. They are isomorphic duals. But ergonomics are important to prevent errors. I personally think Matrix-in-front is nice because it evokes the idea that the matrix is a function acting on the vector. In the real world, When you start doing matrix-vector, if you aren't careful with row-major vs column-major storage you can introduce performance regressions. Let's say the architecture can do 256 bit memory move into 4xf64 mmx registers, you have to think carefully about what is going where before proceeding. Some architectures might be able to do an arbitrary stride so that those f64s can be nonadjacent in a simultaneous move. I suspect not handling this correctly is part of what holds back really good fp8, fp16 performance, and I suspect for example the TPUs struggle with this. Of course optimizing for matrix-vector can be different for optimizing for matrix-matrix, since the column vs row memory access preference for the front matrix is exactly the opposite of the back matrix.
• jpeloquin 6y ago
  > numpy also treat them [vectors] as 1xN matrices I may be misunderstanding something, but in numpy at least a vector, a 1×N array ("row vector"), and a N×1 array ("column vector") behave differently. vec = numpy.array([1,2,3]) assert vec.shape == (3,) assert vec.ndim == 1 row = np.array([[1,2,3]]) assert row.shape == (1,3) assert row.ndim == 2 col = np.array([[1], [2], [3]]) assert col.shape == (3,1) assert col.ndim == 2 And a vector, unlike a 1×N or N×1 array, can be multiplied to either side of any matrix with compatible cardinality: assert all(vec @ numpy.eye(3) == numpy.array([1, 2, 3])) assert all(numpy.eye(3) @ vec == numpy.array([1, 2, 3])) Matlab / Octave, in contrast, forces the user to pick a 1×N or N×1 matrix to represent a vector. Its arrays are always ≥ 2D. So you're probably still right that people get confused translating this to Python. Just wanted to clarify the numpy side of things. The JuliaCon talk you linked does a great job of explaining what different fields mean by "vector"; thanks for linking it.
  • dnautics 6y ago
    I just tried this, I stand corrected, but I remember this (the last clause) not working in numpy. It's possible my memory is flawed: >>> import numpy as np >>> vec = np.array([1, 2, 3]) >>> mtx = np.eye(3) >>> np.dot(vec, mtx) array([1., 2., 3.]) >>> np.dot(mtx, vec) array([1., 2., 3.]) It's possible though that I was remembering not that it doesn't execute, but rather that if you do mess up the multiplication order and do it in the wrong direction with anything besides trivial diagonal matrices, you won't get a type error, you will get an incorrect but valid result, which is arguably worse than a type error, and way more frustrating, especially if start by testing eye, which is the easiest thing to do.
    • jpeloquin 6y ago
      I remember a problem similar to the one you describe when using a dot product both to substitute for a fast loop and for its normal mathematical purpose, in the same operation: points = np.array([[0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0]]) M = np.array([[0.866, 0.5, 0], [-0.5, 0.866, 0], [0, 0, 1]]) # rotation points @ M # works M @ points # raises ValueError, also a different transform Regarding messing up the multiplication order, numpy by itself doesn't have the information to raise a relevant type error, sadly. As you probably know, based on your other comments, the important thing is whether the adjacent basis vectors (dimensions) match, not how they are written. That is, given M = M_ij (a_i ⊗ b_j), a vector v_i a_i must be multiplied on the left, a vector v_i b_i must be multiplied on the right, and a vector v_i c_i (with basis vector c) shouldn't be used at all. Numpy just does array math; it doesn't keep track of dimensions. Hopefully the code has accurate comments, or you're using a language with a decent type checker. However, I think you can track dimensional compatibility using xarray: import xarray as xr import numpy as np points = xr.DataArray([[0, 0, 0], [1, 0, 0], [1, 1, 0], [0, 1, 0]], dims=("idx", "a")) M = xr.DataArray([[0.154, 0.988, 0], [-0.988, 0.154, 0], [0, 0, 1]], dims=("a", "b")) # rotation points.dot(M) # works M.dot(points) # also works # M's a-dim is matched to point's a-dim, so both # results have same values regardless of # multiplication order, just transposed. assert np.all(points.dot(M).T == points.dot(M)) I only just tried xarray in response to your post, so not sure if there are pitfalls in practice, or if this is the best way to use it.