5 ms·
For a multidimensional array: a matrix will be in stored single vector with metadata showing dimension. The presentation of the matrix (view) will then not matt
by robberbarron 6y ago
For a multidimensional array: a matrix will be in stored single vector with metadata showing dimension. The presentation of the matrix (view) will then not matter on the data contained therein.
For q, they are stored as lists of lists, which can mean that their memory is not contiguous. Some algorithms can slow down significantly as a result (eg dense), others might speed up (eg sparse)
However, there are libs in q such as qml which interface with LAPACK and are doing memory mapping so it is not entirely q, just an interface to a C implementation
http://althenia.net/qml http://althenia.net/qml
- moonchild 6y agoI had not seen qml. I must say, an array language interface to another language's math library is one of the funniest things I've ever seen.
- smabie 6y agoI'm sure Arthur Whitney would be upset!
- scottlocklin 6y agoJ has lapack bindings also; you pretty much can't beat lapack for raw matrix math speed.
- smabie 6y agoAre you sure they're not stores in contiguous memory? Q features a crazy number of optimizations for special cases. I would imagine a dense representation of rectangular nested lists is one of them. Q code uses flip (flips a list) and requires it to be rectangular. Moreover, I highly highly doubt column vectors are stored as a bunch of pointers to a single element list. I just can't imagine Whitney ever tolerating such obvious inefficiency. Looking at the link, Q is only 10% slower for big matrices. I can't imagine this being possible without major optimization on q's part since lapack is insanely optimized.