3 ms·
I'm going to say the unthinkable here: In my Information Retrieval class, I got numpy/scipy set up and went about implementing homework assignments with it. H
by jimbokun 18y ago
I'm going to say the unthinkable here:
In my Information Retrieval class, I got numpy/scipy set up and went about implementing homework assignments with it.
However, no matter how much I tried to push as much as possible down into the matrix libraries implemented in C/C++, the surrounding Python code slowed everything down. I was having trouble getting everything to finish in time to hand in my homework by the deadline.
I talked to a classmate who was using Java, and not having any speed problems at all. The night before it was due, I rewrote the whole thing in Java and got it to finish running (I handed in a day late, but at least I had something to hand in.)
I'm sure there are tricks to make things faster in Python. (For example, I later figured out a method I was calling was running all Python code, and if I had called a different method, it would have dropped directly into the fast C code.) But with Java, I didn't have to think about performance. It was just fast.
Java almost certainly has a library for anything you might possibly want to do. "But," I hear you say, "that means I have to write my program in...Java! shudder"
And I empathize with you. Which is why now I'm doing a lot of experimenting with Clojure. Fast as Java, because it compiles to the JVM (as long as you follow as few guidelines.) Access to any Java library with no extra effort on your part. (One of my favorite moments on one of Rich Hickey's Clojure video is where he shows a macro that makes Java calls requiring FEWER parentheses than Java. He was pretty excited about that.) I found a Java open source matrix library that, while not nearly as pretty as Python, got the job done.
So, that's my totally radical recommendation. Clojure + whatever Java libraries you need to get your work done.
- jacobolus 18y ago> However, no matter how much I tried to push as much as possible down into the matrix libraries implemented in C/C++, the surrounding Python code slowed everything down. I was having trouble getting everything to finish in time to hand in my homework by the deadline. Did you profile your code? What was the slow bit?
- jimbokun 18y agoSorry, it's just been too long now to remember the details. I think it was the EM (expectation maximization) algorithm, though, which is hard to reduce entirely to matrix operations, at least for me it was. Any good implementations for EM in Python (plus bridged libraries) out there?
- herdrick 18y agoI've been wondering, are there Java libs as good as numpy/scipy? Sure, in general Java has more libraries but is that true when it comes to math? Oh, and great call on Clojure. I've been having fun with it too.