3 ms·

That remains unclear. Similar to q/k4/kdb, it's optimized for fin-tech database work; that is, operations on very long columns. Apl (esp. dyalog) and j have p

That remains unclear. Similar to q/k4/kdb, it's optimized for fin-tech database work; that is, operations on very long columns. Apl (esp. dyalog) and j have proper multidimensional arrays, which means that e.g. matrix ops are practical/easier. They also have better tacit programming support.

smabie 6y ago
What's the difference between nested lists and full multidimensional array support? It's not like q is bad at multiplying matrices.
- robberbarron 6y ago
  For a multidimensional array: a matrix will be in stored single vector with metadata showing dimension. The presentation of the matrix (view) will then not matter on the data contained therein. For q, they are stored as lists of lists, which can mean that their memory is not contiguous. Some algorithms can slow down significantly as a result (eg dense), others might speed up (eg sparse) However, there are libs in q such as qml which interface with LAPACK and are doing memory mapping so it is not entirely q, just an interface to a C implementation http://althenia.net/qml http://althenia.net/qml
  - moonchild 6y ago
    I had not seen qml. I must say, an array language interface to another language's math library is one of the funniest things I've ever seen.
    - smabie 6y ago
      I'm sure Arthur Whitney would be upset!
    - scottlocklin 6y ago
      J has lapack bindings also; you pretty much can't beat lapack for raw matrix math speed.
  - smabie 6y ago
    Are you sure they're not stores in contiguous memory? Q features a crazy number of optimizations for special cases. I would imagine a dense representation of rectangular nested lists is one of them. Q code uses flip (flips a list) and requires it to be rectangular. Moreover, I highly highly doubt column vectors are stored as a bunch of pointers to a single element list. I just can't imagine Whitney ever tolerating such obvious inefficiency. Looking at the link, Q is only 10% slower for big matrices. I can't imagine this being possible without major optimization on q's part since lapack is insanely optimized.

smabie 6y ago

What's the difference between nested lists and full multidimensional array support? It's not like q is bad at multiplying matrices.

robberbarron 6y ago
For a multidimensional array: a matrix will be in stored single vector with metadata showing dimension. The presentation of the matrix (view) will then not matter on the data contained therein. For q, they are stored as lists of lists, which can mean that their memory is not contiguous. Some algorithms can slow down significantly as a result (eg dense), others might speed up (eg sparse) However, there are libs in q such as qml which interface with LAPACK and are doing memory mapping so it is not entirely q, just an interface to a C implementation http://althenia.net/qml http://althenia.net/qml
- moonchild 6y ago
  I had not seen qml. I must say, an array language interface to another language's math library is one of the funniest things I've ever seen.
  - smabie 6y ago
    I'm sure Arthur Whitney would be upset!
  - scottlocklin 6y ago
    J has lapack bindings also; you pretty much can't beat lapack for raw matrix math speed.
- smabie 6y ago
  Are you sure they're not stores in contiguous memory? Q features a crazy number of optimizations for special cases. I would imagine a dense representation of rectangular nested lists is one of them. Q code uses flip (flips a list) and requires it to be rectangular. Moreover, I highly highly doubt column vectors are stored as a bunch of pointers to a single element list. I just can't imagine Whitney ever tolerating such obvious inefficiency. Looking at the link, Q is only 10% slower for big matrices. I can't imagine this being possible without major optimization on q's part since lapack is insanely optimized.