18 ms·

Column Vectors vs. Row Vectors

95 points by oumua_don17 4y ago

FrostKiwi 4y ago
As someone who's in love with OpenGL, man what a great piece and interesting piece of history.
- texaslonghorn5 4y ago
  Agree, this email format is also so easy to parse. It's like I'm dropped right into the action.
- greggman3 4y ago
  Not in love with GL as someone that's written GL drivers and re-implemented GL on top of modern APIs, but.... for me, the GL style (whatever you decide to call it) seems to make sense from a performance POV. I often want one axis or the translation. It's a single SIMD load since vs if they were stored the other way I'd have to load every 4th float. Also, while you do type them out in code different from math x x x x y y y y z z z z t t t t If you squint you can view them as columns xaxis = [ . . . . ] yaxis = [ . . . . ] zaxis = [ . . . . ] translation = [ . . . . ] matrix = [xaxis, yaxis, zaxis, translation] // 4 columns side by side
captainmuon 4y ago
I studied physics, and I did a bit of 3D graphics back in the day, and I could never remember the difference between column and row vectors (or co- and contravariant vectors, bra and ket vectors, ...) and when to use which. My guilty secret is that I don't care. For me a vector is just abstractly three or four numbers together. (Maybe with some defined transformation properties if I'm doing physics.) Whether a matrix is stored row-major or column-major is an implementation detail I try to hide. In fact I used to make indexing errors all the time. But now when I need to deal with vectors and matrices, I look up or decide on the "correct" way, hide everything behind Vector and Matrix classes, implement dot and cross product once, and never fall back to the raw vector elements if possible.
- chombier 4y ago
  If one thinks of vectors as the matrix of some linear map over the scalars (and similarly covectors as linear maps to the scalars), and assuming the usual composition order of maps then it seems there is no choice as to the column/row representation: vectors have to be (n x 1) and covectors have to be (1 x n) This doesn't say anything regarding their practical storage, but at least conceptually vectors/covectors are a bit more than just a collection of numbers. This distinction is pretty useful in practice when encoded in types to avoid implicitly mixing things that are conceptually different.
- IIAOPSW 4y ago
  I was with you until you said bra and ket didn't do it for you. Its right there in the name bra(c)ket. <bra|, |ket>. Logically equiv to any other notation, but impossible to fudge up. When they are multiplied in a way that lines up like <bra|ket>, you got an inner product (scalar measure of overlap). When they are multiplied wrong way around, you got a ketbra |i><j| (which is basically a neat way to represent a matrix element at indices i and j). Is the bra the row and the ket the column or vice versa? Don't matter. Just keep things facing whichever way they were originally facing and nothing will go wrong. In fact, you don't even need to think in terms of columns at all if you don't want. That's it. That's the whole system. For the engineers who never seen this, |i> |j> |k> work exactly like unit vectors i,j,k, but you aren't limited to the usual 3 dimensions and 3 letters. Writing |x> in a ket like this just means "I am using x as a label for a degree of freedom in some vector space, not as the literal value x". This lets you do cool shit, like if you got n dimensions you can write a vector as sum v_i |i> ranging from 0 to n. You can also do super useful stuff like merging two vector spaces together to represent a larger system (aka a tensor product). Eg if I have a vector v = sum v_i |i> representing one subsystem and another vector w = sum w_j |j> representing some other subsystem, I can construct the vector u = sum v_i w_j |i, j> representing the whole system (sum is over both i and j). (I know you already learned all this, I wrote it for everyone else's benefit). Unrelated, given the webs origin at cern, HTML should have used bras as start tags and kets as end tags. <HTML| <head|head> <body| ... |body> |HTML>.
  - azalemeth 4y ago
bruce343434 4y ago
Abstractly, a vector is neither a row nor a column. An n-dimensional vector is an ordered collection of n coordinates. An m by n matrix is an m-dimensional vector of n-dimensional vectors. But when you define a matrix in most programming languages you can only really write it like this: mat = [ [1,2,3], [4,5,6], [7,8,9] ] Note that the inner vectors are laid out as rows, and the outer vector (the matrix) is laid out as a column. Some languages have fancy syntax that omits the need for braces. But for those that don't, this is clearly an array of 3 arrays of 3 ints. It's only intuitive to think of this as an array of rows given how it's laid out visually. I also think in that context it would make more sense to index it as mat[row][col], or in other words row major indexing. Because of that, it also makes sense to store the mat row-major, that is, laid out in memory as [1,2,3,4,5,6,7,8,9] so that mat[row][col] becomes mem[row*3 + col]. This is also how C does it. I think it was a mistake for OpenGL to break with that C convention given how close the relationship between C and OpenGL is. Mathematicians be damned.
- IshKebab 4y ago
  Yeah having used Matlab extensively it's pretty clear that maths just got some things wrong. 1-based indexing is wrong. Matrix index ordering is wrong (try dealing with images in Matlab if you don't believe me). Mathematicians generally don't care because equations are basically pseudocode and they can make up new syntax and hand-wave edge cases away whenever they want.
- rocqua 4y ago
  Very true, and I think this is generally why C 'does it right'. But there are caveats. Firstly, almost no-one actually creates their matrices as literals. Instead they are build dynamically an usually incrementally. Nevertheless, people still think back to these very basic examples when trying to understand things, so this probably should be standard. Secondly, these decisions matter a lot for matrix vector multiplication. The 'C' way is very efficient for computing Ax which presumes vector x to be a Column vector. When you want to compute xA instead which presumes vector x to be a row vector, you want column-major ordering for efficiency. The same works for matrix-matrix multiplication AB where you want A to be row-major and B to be column-major. To sum up. Because of our writing direction, and the natural way to write 2-d arrays, it makes sense to have row-major ordering. That immediately means it also makes sense to use column vectors, generally speaking. There are also exceptions if you happen to know if a matrix will often be at the 'left' or 'right' side of a multiplication. Some more interesting notes that are less relevant: In C the matrix you defined has an incompatible type with how most matrices are stored (most people in C store rows contiguously, then define a matrix as an array of pointers to rows). int mat[3][3]; vs int *mat[3] These are meaningfully different. In the first case getting element `mat[1][1]` is just 'base-pointer + 4'. Whereas in the second case, getting element `mat[1][1]` is 'read pointer at base-pointer + 1 and add 1 to the added pointer`. Another common ways to use matrices is element-wise, where you refer to x[i][j]. Mathematicians expect i to be the row and j to be the column. Which matches C. Whilst Fortran does it the other way around.
pkrumins 4y ago
Can anyone explain why in physics you use a 3x3 matrix for rotation but in computer graphics you have a 4x4 with “1” in the lower right corner of the matrix.
- arbitrandomuser 4y ago
  Such 4x4 matrices allow for translations along with rotations (and skewing I think) into one linear transformation the 4th column can include the translation vector x,y,z and the last element is 1
- deleted 4y ago
  [deleted]
- GuB-42 4y ago
  These are called homogeneous coordinates, and as expected since you have 4x4 matrices, vectors are also 4D, usually in the form (x,y,z,1). The idea is that you can multiply each coordinate by a non-zero scalar and the vector represents the same point, it has several interesting mathematical properties. For computer graphics, the main advantage is that you can represent translation and projection in the matrix in addition to the other linear transformations like rotation and scaling. For projection, the 4th vector coordinate, usually named "w", is called the homogeneous coordinate and it is typically not 1 after going through the projection matrix. As a final step, it divides all the other coordinates, so (x,y,z,w) becomes (x/w,y/w,z/w,1), x/w and y/w are the 2D screen coordinates, and z/w goes into the Z-buffer.
- joppy 4y ago
  The 4x4 represents both a 3x3 matrix A and a translation t, so that the “affine transformation” applied to a vector is v -> Av + t. One could pass around pairs (A, t) instead, and figure out how to compose those pairs, but it turns out there’s a nice way of embedding them in a 4x4 matrix such that composition is just 4x4 matrix multiplication. You may opt into some other mathematical niceties too, which are handy for applying perspective transformations, but the main takeaway is that you get efficient compositions of affine transformations.
- rogual 4y ago
pseudostem 4y ago
> Recent mathematical treatments of linear algebra and related fields invariably treat vectors as columns (there are some technical reasons for this) As someone who is studying the subject, I would be greatful if someone can please explain the technical reasons for this. I do not understand why I can't write them as row vectors and do the same Linear Algebra trickery (which we do with column vectors) another way around.
- computerfriend 4y ago
  You can, but you'll be doing linear algebra in the dual space, with covectors, instead of in the regular space with vectors.
- sfpotter 4y ago
  There’s no real reason to think of column vectors as points in space rather than the other way around when you’re doing linear algebra. It’s just a convention. If you wanted, you could think of a row vector as a point in space and transpose everything. In that case, your dual space of linear functionals would consist of column vectors rather than row vectors.
- shanusmagnus 4y ago
  I second your request for a detailed explanation of the "technical reason" -- different people kept mentioning it, then saying that they would omit it. I want to know the technical / mathematical details for why, in isolation, row(1, 2, 3) is different from column(1, 2, 3). Like, all the way back down to the purest axioms or geometric intuitions. I understand why the "rules" of matrix multiplication make the distinction -- but what is underneath those rules? What does it mean?
  - mjburgess 4y ago
    By convention, (roughly), row vectors are functions and column vectors are data. [1, 2, 3][2 4 6] Means 1*2 + 2*4 + 6*3
  - dandanua 4y ago
dandanua 4y ago
It's common in mathematics to consider vectors as column vectors because the action of some linear operator represented by matrix A on a vector v is just the matrix product A*v. Mathematical conventions are really dependent on each other. If you change one — a lot of others become awkward. I wrote short notes about conventions in mathematics to understand them better: (pdf) https://github.com/dandanua/little-endian-vs-big-endian-in-quantum-computing/blob/master/txt.pdf https://github.com/dandanua/little-endian-vs-big-endian-in-q...
- nh23423fefe 4y ago
  The because seems wrong. I would argue vectors are columns because systems of linear equations are sentences which take up a row in a math paper. the matrix is simply the layout and so R^n is column on the other side of the equals. It's arbitrary.
  - dandanua 4y ago
    Well, from the historical point of view I think you're right. But in an abstract sense, matrix is a presentation of a linear operator in a specific basis. If we choose to write it as a transposed matrix, for example, then the matrix product rule would be different (it should correspond to the composition of linear operators).
    - nh23423fefe 4y ago
      But I think this mixes issues. Vectors are objects from some set (suppose R^n), linear operators some other set (suppose R^n -> R^n). There exists an operation function application which takes an operator and a vector and produces a vector. There is an operation function composition which takes 2 operators to make a third. There is a "functor" from vectors to coordinate vectors, linear operators to matrices, function application to multiplication, function composition to multiplication. The rule for matrix multiplication is constrained by the functor. So you can't choose to write it incorrectly or its not a "homomorphism". This is what I'm trying to express. I don't think you are "free to choose" in some sense. And I wouldn't say the orientation of the vector is caused by the multiplication rule over matrices. I think the "functor" is what is doing the choosing.
VikingCoder 4y ago
Yes and yes, to the Homogeneous coordinate being the first column or row. That way, you can have sparse vectors, or sparse matrices. It doesn't matter how many dimensions are specified for your object, stored in memory. It gets transformed the same. Have a point at the origin? [1] Done. It doesn't matter if it's a 2D, 3D, 7D. Have a 0-vector? [0] Done. Have a 2D point? [1, x, y] Have a 3D point? [1, x, y, z] Have a 1D vector? [0, x] I honestly don't know why we don't all do it this way.
albertzeyer 4y ago
As someone else said, when using einops, einsum, named tensors (named dimensions), it doesn't really matter. For most software frameworks, there is anyway not a diff between column vector vs row vector, it's just shape [dim] in any case, but it becomes relevant how you do matmul etc. with some other matrix [dim1, dim2] or some other tensor [dim1, dim2, dim3].
bugfix-66 4y ago
Fix the bug: Is the matrix row major or column major? https://bugfix-66.com/e74dcfb6a10cb71404f495922533d947e39fd79a4d363033161f812cd539a1d9 https://bugfix-66.com/e74dcfb6a10cb71404f495922533d947e39fd7...
bikenaga 4y ago
If f: X -> Y and g: Y -> Z are functions, their composite is the function from X to Z which most mathematicians would write as "g o f" (typographically that should be a small centered circle, not the letter "o" as here), defined by (g o f)(x) = g(f(x)). Notice that the function f, which is applied first, appears on the right and g, which is applied second, appears on the left. But if we draw a diagram of the composite, it looks like this: f g X ----> Y ----> Z In the diagram, the first applied function is on the left and the second applied function is on the right. Some people have felt that, for this reason, it would be better to write composites with the arguments on the left, like this: (x)f and [x(f)]g . However, most mathematicians follow the right-to-left convention (that's the way it seems to be in most programming languages, right?). Anyway, consider how this works if f and g are linear transformations and X, Y, and Z are finite-dimensional vector spaces, as in linear algebra. So we can take X = F^p, Y = F^n, and Z = F^m, where F is the field of scalars, and vectors are just "arrays of numbers". f: F^p -> F^n, g: F^n -> F^m You can represent f and g using matrices: f is represented by an n x p matrix B and g is represented by an m x n matrix A. In that case, function application is done by matrix multiplication. The composite g o f is represented by the product A B, which is an m x p matrix. Notice that the matrices are in the same order in "A B" as their functions are in "g o f". Inputs to g o f are p-dimensional vectors; where should a p-dimensional vector go in order that multiplication by A B should represent application of the function g o f? A B [p x 1 column vector] ---> okay [1 x p row vector] A B ---> wrong: you can't multiply (1 x p) * (m x p) In other words, once you commit to the standard convention for writing composites, you're committed to using column vectors as inputs to linear transformations written as matrices, or somewhere you'll probably have an inconsistency in the order of application. Or you can bite the bullet, write arguments to functions on the left, and then defaulting to row vectors would be okay. It's just notation.
thayne 4y ago
Interesting that this mentions people using row vectors in physics. In my physics classes (and math classes) vectors were always represented as columns.

95 points by oumua_don17 4y ago

FrostKiwi 4y ago

As someone who's in love with OpenGL, man what a great piece and interesting piece of history.

texaslonghorn5 4y ago
Agree, this email format is also so easy to parse. It's like I'm dropped right into the action.
greggman3 4y ago
Not in love with GL as someone that's written GL drivers and re-implemented GL on top of modern APIs, but.... for me, the GL style (whatever you decide to call it) seems to make sense from a performance POV. I often want one axis or the translation. It's a single SIMD load since vs if they were stored the other way I'd have to load every 4th float. Also, while you do type them out in code different from math x x x x y y y y z z z z t t t t If you squint you can view them as columns xaxis = [ . . . . ] yaxis = [ . . . . ] zaxis = [ . . . . ] translation = [ . . . . ] matrix = [xaxis, yaxis, zaxis, translation] // 4 columns side by side

captainmuon 4y ago

I studied physics, and I did a bit of 3D graphics back in the day, and I could never remember the difference between column and row vectors (or co- and contravariant vectors, bra and ket vectors, ...) and when to use which. My guilty secret is that I don't care. For me a vector is just abstractly three or four numbers together. (Maybe with some defined transformation properties if I'm doing physics.) Whether a matrix is stored row-major or column-major is an implementation detail I try to hide. In fact I used to make indexing errors all the time. But now when I need to deal with vectors and matrices, I look up or decide on the "correct" way, hide everything behind Vector and Matrix classes, implement dot and cross product once, and never fall back to the raw vector elements if possible.

chombier 4y ago
If one thinks of vectors as the matrix of some linear map over the scalars (and similarly covectors as linear maps to the scalars), and assuming the usual composition order of maps then it seems there is no choice as to the column/row representation: vectors have to be (n x 1) and covectors have to be (1 x n) This doesn't say anything regarding their practical storage, but at least conceptually vectors/covectors are a bit more than just a collection of numbers. This distinction is pretty useful in practice when encoded in types to avoid implicitly mixing things that are conceptually different.
IIAOPSW 4y ago
I was with you until you said bra and ket didn't do it for you. Its right there in the name bra(c)ket. <bra|, |ket>. Logically equiv to any other notation, but impossible to fudge up. When they are multiplied in a way that lines up like <bra|ket>, you got an inner product (scalar measure of overlap). When they are multiplied wrong way around, you got a ketbra |i><j| (which is basically a neat way to represent a matrix element at indices i and j). Is the bra the row and the ket the column or vice versa? Don't matter. Just keep things facing whichever way they were originally facing and nothing will go wrong. In fact, you don't even need to think in terms of columns at all if you don't want. That's it. That's the whole system. For the engineers who never seen this, |i> |j> |k> work exactly like unit vectors i,j,k, but you aren't limited to the usual 3 dimensions and 3 letters. Writing |x> in a ket like this just means "I am using x as a label for a degree of freedom in some vector space, not as the literal value x". This lets you do cool shit, like if you got n dimensions you can write a vector as sum v_i |i> ranging from 0 to n. You can also do super useful stuff like merging two vector spaces together to represent a larger system (aka a tensor product). Eg if I have a vector v = sum v_i |i> representing one subsystem and another vector w = sum w_j |j> representing some other subsystem, I can construct the vector u = sum v_i w_j |i, j> representing the whole system (sum is over both i and j). (I know you already learned all this, I wrote it for everyone else's benefit). Unrelated, given the webs origin at cern, HTML should have used bras as start tags and kets as end tags. <HTML| <head|head> <body| ... |body> |HTML>.
- azalemeth 4y ago

bruce343434 4y ago

Abstractly, a vector is neither a row nor a column. An n-dimensional vector is an ordered collection of n coordinates. An m by n matrix is an m-dimensional vector of n-dimensional vectors. But when you define a matrix in most programming languages you can only really write it like this: mat = [ [1,2,3], [4,5,6], [7,8,9] ] Note that the inner vectors are laid out as rows, and the outer vector (the matrix) is laid out as a column. Some languages have fancy syntax that omits the need for braces. But for those that don't, this is clearly an array of 3 arrays of 3 ints. It's only intuitive to think of this as an array of rows given how it's laid out visually. I also think in that context it would make more sense to index it as mat[row][col], or in other words row major indexing. Because of that, it also makes sense to store the mat row-major, that is, laid out in memory as [1,2,3,4,5,6,7,8,9] so that mat[row][col] becomes mem[row*3 + col]. This is also how C does it. I think it was a mistake for OpenGL to break with that C convention given how close the relationship between C and OpenGL is. Mathematicians be damned.

IshKebab 4y ago
Yeah having used Matlab extensively it's pretty clear that maths just got some things wrong. 1-based indexing is wrong. Matrix index ordering is wrong (try dealing with images in Matlab if you don't believe me). Mathematicians generally don't care because equations are basically pseudocode and they can make up new syntax and hand-wave edge cases away whenever they want.
rocqua 4y ago
Very true, and I think this is generally why C 'does it right'. But there are caveats. Firstly, almost no-one actually creates their matrices as literals. Instead they are build dynamically an usually incrementally. Nevertheless, people still think back to these very basic examples when trying to understand things, so this probably should be standard. Secondly, these decisions matter a lot for matrix vector multiplication. The 'C' way is very efficient for computing Ax which presumes vector x to be a Column vector. When you want to compute xA instead which presumes vector x to be a row vector, you want column-major ordering for efficiency. The same works for matrix-matrix multiplication AB where you want A to be row-major and B to be column-major. To sum up. Because of our writing direction, and the natural way to write 2-d arrays, it makes sense to have row-major ordering. That immediately means it also makes sense to use column vectors, generally speaking. There are also exceptions if you happen to know if a matrix will often be at the 'left' or 'right' side of a multiplication. Some more interesting notes that are less relevant: In C the matrix you defined has an incompatible type with how most matrices are stored (most people in C store rows contiguously, then define a matrix as an array of pointers to rows). int mat[3][3]; vs int *mat[3] These are meaningfully different. In the first case getting element `mat[1][1]` is just 'base-pointer + 4'. Whereas in the second case, getting element `mat[1][1]` is 'read pointer at base-pointer + 1 and add 1 to the added pointer`. Another common ways to use matrices is element-wise, where you refer to x[i][j]. Mathematicians expect i to be the row and j to be the column. Which matches C. Whilst Fortran does it the other way around.

pkrumins 4y ago

Can anyone explain why in physics you use a 3x3 matrix for rotation but in computer graphics you have a 4x4 with “1” in the lower right corner of the matrix.

arbitrandomuser 4y ago
Such 4x4 matrices allow for translations along with rotations (and skewing I think) into one linear transformation the 4th column can include the translation vector x,y,z and the last element is 1
deleted 4y ago
[deleted]
GuB-42 4y ago
These are called homogeneous coordinates, and as expected since you have 4x4 matrices, vectors are also 4D, usually in the form (x,y,z,1). The idea is that you can multiply each coordinate by a non-zero scalar and the vector represents the same point, it has several interesting mathematical properties. For computer graphics, the main advantage is that you can represent translation and projection in the matrix in addition to the other linear transformations like rotation and scaling. For projection, the 4th vector coordinate, usually named "w", is called the homogeneous coordinate and it is typically not 1 after going through the projection matrix. As a final step, it divides all the other coordinates, so (x,y,z,w) becomes (x/w,y/w,z/w,1), x/w and y/w are the 2D screen coordinates, and z/w goes into the Z-buffer.
joppy 4y ago
The 4x4 represents both a 3x3 matrix A and a translation t, so that the “affine transformation” applied to a vector is v -> Av + t. One could pass around pairs (A, t) instead, and figure out how to compose those pairs, but it turns out there’s a nice way of embedding them in a 4x4 matrix such that composition is just 4x4 matrix multiplication. You may opt into some other mathematical niceties too, which are handy for applying perspective transformations, but the main takeaway is that you get efficient compositions of affine transformations.
rogual 4y ago

pseudostem 4y ago

> Recent mathematical treatments of linear algebra and related fields invariably treat vectors as columns (there are some technical reasons for this) As someone who is studying the subject, I would be greatful if someone can please explain the technical reasons for this. I do not understand why I can't write them as row vectors and do the same Linear Algebra trickery (which we do with column vectors) another way around.

computerfriend 4y ago
You can, but you'll be doing linear algebra in the dual space, with covectors, instead of in the regular space with vectors.
sfpotter 4y ago
There’s no real reason to think of column vectors as points in space rather than the other way around when you’re doing linear algebra. It’s just a convention. If you wanted, you could think of a row vector as a point in space and transpose everything. In that case, your dual space of linear functionals would consist of column vectors rather than row vectors.
shanusmagnus 4y ago
I second your request for a detailed explanation of the "technical reason" -- different people kept mentioning it, then saying that they would omit it. I want to know the technical / mathematical details for why, in isolation, row(1, 2, 3) is different from column(1, 2, 3). Like, all the way back down to the purest axioms or geometric intuitions. I understand why the "rules" of matrix multiplication make the distinction -- but what is underneath those rules? What does it mean?
- mjburgess 4y ago
  By convention, (roughly), row vectors are functions and column vectors are data. [1, 2, 3][2 4 6] Means 1*2 + 2*4 + 6*3
- dandanua 4y ago

dandanua 4y ago

It's common in mathematics to consider vectors as column vectors because the action of some linear operator represented by matrix A on a vector v is just the matrix product A*v. Mathematical conventions are really dependent on each other. If you change one — a lot of others become awkward. I wrote short notes about conventions in mathematics to understand them better: (pdf) https://github.com/dandanua/little-endian-vs-big-endian-in-quantum-computing/blob/master/txt.pdf https://github.com/dandanua/little-endian-vs-big-endian-in-q...

nh23423fefe 4y ago
The because seems wrong. I would argue vectors are columns because systems of linear equations are sentences which take up a row in a math paper. the matrix is simply the layout and so R^n is column on the other side of the equals. It's arbitrary.
- dandanua 4y ago
  Well, from the historical point of view I think you're right. But in an abstract sense, matrix is a presentation of a linear operator in a specific basis. If we choose to write it as a transposed matrix, for example, then the matrix product rule would be different (it should correspond to the composition of linear operators).
  - nh23423fefe 4y ago
    But I think this mixes issues. Vectors are objects from some set (suppose R^n), linear operators some other set (suppose R^n -> R^n). There exists an operation function application which takes an operator and a vector and produces a vector. There is an operation function composition which takes 2 operators to make a third. There is a "functor" from vectors to coordinate vectors, linear operators to matrices, function application to multiplication, function composition to multiplication. The rule for matrix multiplication is constrained by the functor. So you can't choose to write it incorrectly or its not a "homomorphism". This is what I'm trying to express. I don't think you are "free to choose" in some sense. And I wouldn't say the orientation of the vector is caused by the multiplication rule over matrices. I think the "functor" is what is doing the choosing.

VikingCoder 4y ago

Yes and yes, to the Homogeneous coordinate being the first column or row. That way, you can have sparse vectors, or sparse matrices. It doesn't matter how many dimensions are specified for your object, stored in memory. It gets transformed the same. Have a point at the origin? [1] Done. It doesn't matter if it's a 2D, 3D, 7D. Have a 0-vector? [0] Done. Have a 2D point? [1, x, y] Have a 3D point? [1, x, y, z] Have a 1D vector? [0, x] I honestly don't know why we don't all do it this way.

albertzeyer 4y ago

As someone else said, when using einops, einsum, named tensors (named dimensions), it doesn't really matter. For most software frameworks, there is anyway not a diff between column vector vs row vector, it's just shape [dim] in any case, but it becomes relevant how you do matmul etc. with some other matrix [dim1, dim2] or some other tensor [dim1, dim2, dim3].

bugfix-66 4y ago

Fix the bug: Is the matrix row major or column major? https://bugfix-66.com/e74dcfb6a10cb71404f495922533d947e39fd79a4d363033161f812cd539a1d9 https://bugfix-66.com/e74dcfb6a10cb71404f495922533d947e39fd7...

bikenaga 4y ago

If f: X -> Y and g: Y -> Z are functions, their composite is the function from X to Z which most mathematicians would write as "g o f" (typographically that should be a small centered circle, not the letter "o" as here), defined by (g o f)(x) = g(f(x)). Notice that the function f, which is applied first, appears on the right and g, which is applied second, appears on the left. But if we draw a diagram of the composite, it looks like this: f g X ----> Y ----> Z In the diagram, the first applied function is on the left and the second applied function is on the right. Some people have felt that, for this reason, it would be better to write composites with the arguments on the left, like this: (x)f and [x(f)]g . However, most mathematicians follow the right-to-left convention (that's the way it seems to be in most programming languages, right?). Anyway, consider how this works if f and g are linear transformations and X, Y, and Z are finite-dimensional vector spaces, as in linear algebra. So we can take X = F^p, Y = F^n, and Z = F^m, where F is the field of scalars, and vectors are just "arrays of numbers". f: F^p -> F^n, g: F^n -> F^m You can represent f and g using matrices: f is represented by an n x p matrix B and g is represented by an m x n matrix A. In that case, function application is done by matrix multiplication. The composite g o f is represented by the product A B, which is an m x p matrix. Notice that the matrices are in the same order in "A B" as their functions are in "g o f". Inputs to g o f are p-dimensional vectors; where should a p-dimensional vector go in order that multiplication by A B should represent application of the function g o f? A B [p x 1 column vector] ---> okay [1 x p row vector] A B ---> wrong: you can't multiply (1 x p) * (m x p) In other words, once you commit to the standard convention for writing composites, you're committed to using column vectors as inputs to linear transformations written as matrices, or somewhere you'll probably have an inconsistency in the order of application. Or you can bite the bullet, write arguments to functions on the left, and then defaulting to row vectors would be okay. It's just notation.

thayne 4y ago

Interesting that this mentions people using row vectors in physics. In my physics classes (and math classes) vectors were always represented as columns.