4 ms·

The instructor emphasized the performance benefit of Octave's optimized matrix operations. Did you use Colt's matrix operations, or were you able to match the p

The instructor emphasized the performance benefit of Octave's optimized matrix operations. Did you use Colt's matrix operations, or were you able to match the performance of vectorized Octave code with non-vectorized Clojure code?

rincewind 15y ago
I also implemented some of the exercises in clojure, and my naive implementation without colt or incanter ran orders of magnitude slower than my octave code. Then again i did not code it with performance in mind. I overloaded algo.generic for matrix operations, so the fixnum math was probably not inlined.
aheilbut 15y ago
I think the real magic comes from understanding the algorithms as matrix operations, and then implementing them by more or less just writing down the algebra.
- billswift 15y ago
  Can you give a source for learning about "algorithms as matrix operations"? I tried searching on that phrase in Google Scholar but didn't get anything useful.
  - oscilloscope 15y ago
    If you can express the code using the functions and operators of linear algebra, you can simply write the math and get fast, elegant code. May even run on a GPU. Fortran is frequently used for physics simulation. We used Mathematica, Maple and Matlab even in non-computing classes. See http://www.netlib.org/lapack/ http://www.netlib.org/lapack/ Also http://en.wikipedia.org/wiki/Array_programming http://en.wikipedia.org/wiki/Array_programming
    - billswift 15y ago
      Your first phrase captures what I assumed he meant, I was wondering if there is anything available that could help me recognize algorithms that could or should be expressed that way, and how to do it. Your Wikipedia link was helpful. I am working on a calculus refresher right now (I really need it) and am planning on working through an elementary linear algebra text once that is done, which is why I particularly noticed the comment. I have found that having an idea of applications helps retention, which is one reason I am trying to track down something more concrete.
      - aheilbut 15y ago
        Most of the textbooks on machine learning present things in that way (it's really the only way). For example, check out Elements of Statistical Learning (http://www-stat.stanford.edu/~tibs/ElemStatLearn/ http://www-stat.stanford.edu/~tibs/ElemStatLearn/).
  - jaylevitt 15y ago
    If, like me, you thought "linear algebra" meant "stuff without exponents", you should start here: http://www-math.mit.edu/~gs/papers/starting2matrices.pdf http://www-math.mit.edu/~gs/papers/starting2matrices.pdf It's the most accessible starter I've found to date. There is also a linear algebra group-learning thread somewhere on HN.
djacobs 15y ago
I used Colt. I'd be interested to see if Clojure parallelization and under-the-hood optimizations could match vectorization performance. Note, I haven't run official benchmarks yet. My comparisons have been more like "back propagation and gradient checking took about the same length of time in both".
spariev 15y ago
I believe Incanter uses Colt/Parallel Colt under the hood

rincewind 15y ago

I also implemented some of the exercises in clojure, and my naive implementation without colt or incanter ran orders of magnitude slower than my octave code. Then again i did not code it with performance in mind. I overloaded algo.generic for matrix operations, so the fixnum math was probably not inlined.

aheilbut 15y ago

I think the real magic comes from understanding the algorithms as matrix operations, and then implementing them by more or less just writing down the algebra.

billswift 15y ago
Can you give a source for learning about "algorithms as matrix operations"? I tried searching on that phrase in Google Scholar but didn't get anything useful.
- oscilloscope 15y ago
  If you can express the code using the functions and operators of linear algebra, you can simply write the math and get fast, elegant code. May even run on a GPU. Fortran is frequently used for physics simulation. We used Mathematica, Maple and Matlab even in non-computing classes. See http://www.netlib.org/lapack/ http://www.netlib.org/lapack/ Also http://en.wikipedia.org/wiki/Array_programming http://en.wikipedia.org/wiki/Array_programming
  - billswift 15y ago
    Your first phrase captures what I assumed he meant, I was wondering if there is anything available that could help me recognize algorithms that could or should be expressed that way, and how to do it. Your Wikipedia link was helpful. I am working on a calculus refresher right now (I really need it) and am planning on working through an elementary linear algebra text once that is done, which is why I particularly noticed the comment. I have found that having an idea of applications helps retention, which is one reason I am trying to track down something more concrete.
    - aheilbut 15y ago
      Most of the textbooks on machine learning present things in that way (it's really the only way). For example, check out Elements of Statistical Learning (http://www-stat.stanford.edu/~tibs/ElemStatLearn/ http://www-stat.stanford.edu/~tibs/ElemStatLearn/).
- jaylevitt 15y ago
  If, like me, you thought "linear algebra" meant "stuff without exponents", you should start here: http://www-math.mit.edu/~gs/papers/starting2matrices.pdf http://www-math.mit.edu/~gs/papers/starting2matrices.pdf It's the most accessible starter I've found to date. There is also a linear algebra group-learning thread somewhere on HN.

djacobs 15y ago

I used Colt. I'd be interested to see if Clojure parallelization and under-the-hood optimizations could match vectorization performance. Note, I haven't run official benchmarks yet. My comparisons have been more like "back propagation and gradient checking took about the same length of time in both".

spariev 15y ago

I believe Incanter uses Colt/Parallel Colt under the hood