7 ms·

I do scientific computing, and Python is one language I never actually got around to learning for some reason. However, as a long-time hobby, I do have an inte

I do scientific computing, and Python is one language I never actually got around to learning for some reason. However, as a long-time hobby, I do have an interest in programming languages so I like exploring things like Haskell, Clojure, Lisp, etc. One language I'm really excited about for scientific computing though is Julia. From a language-design perspective, it's beautiful. It was actually thought out rather than kludged together. I've been trying to gradually use it more and more for my research, but the only problem I've found so far is the large mental context-switch I make going from my usual languages to Julia. It's hard to tell what Julia code will be the most performant because there's many ways of doing the same task. I saw someone in the comments on this page mention that you can hand-tune the LLVM generated output within the REPL itself. I imagine this would be very useful if I can get around to learning it (anyone know a good tutorial?)

michaelochurch 13y ago
What are your thoughts on Clojure's suitability as a go-to language for scientific computing (except in lower-level, high-performance scenarios that might recommend Julia)? I think it has serious potential in this field. It's not there yet, but it's getting there, and the core.matrix standardization helps a lot.
- collyw 13y ago
  Its functional. Looking at the way bioinformaticians code, that will make it too complicated for many of them. SQL seems too much for many of them.
- jamesjporter 13y ago
  I agree with collyw; Lisp is too much for most scientists, who just care about getting their research done and can't be bother to learn all this weird FP/paren stuff [1]. Supplying an obvious, familiar syntax that looks like math written on paper for, e.g., matrix operations, is critical. This is actually one of the core tensions in Julia development imho: balancing having a sane, well designed language (from a programmer's perspective) vs. having a tool that allows scientists to quickly and easily crank out results. [1]: Note; I am not insinuating that Lisp is bad, I like it personally. Just relaying the response you will get from most practicing scientists who are not trained as programmers.
  - michaelochurch 13y ago
    Just relaying the response you will get from most practicing scientists who are not trained as programmers. Is it that hard of a gap to cross? I was a math major who took a couple CS courses, and wouldn't say I was "trained as a programmer". I found Scheme pretty easy to get from the go. I tend to think that anyone who can get a PhD in the physical sciences can become a half-decent programmer-- if the desire is there, the intelligence and work ethic being established (one hopes, at least) by the degree.
    - jamesjporter 13y ago
      It's less a matter of ability and more a matter of having the time/motivation. If you took a bunch of practicing natural scientists and/or engineers and forced them somehow to enroll in a course teaching Scheme/Clojure/etc., would many of them do well? Almost certainly. If you gave them the choice between a Lisp and something like Julia or MATLAB to use in their everyday work when they need to do some computation? They'll probably choose the later because it's easy and familiar and doesn't have a huge up-front time investment; scientists tend to be very busy people.
xfax 13y ago
The big question for me is whether Julia will be able to maintain its "purity" as it gains adoption. R probably started out "beautiful" and "thought out" but has lost that edge with years of community driven development. It's also what make it so damn useful -- you can pretty much find anything on CRAN, often multiple implementations of it.
- michaelhoffman 13y ago
  Sorry, no. Even the core S language would not be described by very many users as "beautiful" according to modern standards.
- mbq 13y ago
  R is actually one of the most pure languages out there; it basically says "I have vectors; they can have missing values, be nested, and can have other vectors as attributes. And I have functions with lexical scoping. Now go and build the rest as you like." So people did this, one better, one worse -- but the core and beautiful stuff here is that all those approaches will work together and just do the job.
tokipin 13y ago
python is a deliberately-straightforward language. i don't see how anyone couldn't become highly proficient in it after writing one or two scripts
- leephillips 13y ago
  The straightforward design makes it fast to go from zero to having a working proficiency. But I'm not sure about "highly proficient": even experienced Pythoners get tripped up by things such as mutable function arguments, and it's often not clear why some simple-looking code is running slowly, and how to speed it up.
- benhoyt 13y ago
  Deliberately-straightforward -- agreed. But "highly proficient" after writing one or two scripts? That's quite a stretch. For instance, one of the questions I give in phone screens is for the candidate to write a program to count the number of occurrences of unique words in a text file. The "after writing one or two Python scripts" approach is something like this: counts = {} f = open('test.txt') lines = f.read().split('\n') for line in lines: for word in line.split(' '): if word: word = word.lower() if word in counts.keys(): counts[word] += 1 else: counts[word] = 1 f.close() count_items = [(count, word) for word, count in counts.items()] count_items.sort() for count, word in reversed(count_items): print word, count Whereas the "highly proficient" (and much simpler and more Pythonic) approach might look something like this: import collections counts = collections.Counter() with open('test.txt') as f: for line in f: for word in line.lower().split(): counts[word] += 1 for word, count in counts.most_common(): print word, count
  - yohanatan 13y ago
    Actually that's not 'highly proficient'. This is: def read_words(words_file): return [word for line in open(words_file, 'r') for word in line.split()] len(set(read_words('test.txt')))
epaladin 13y ago
Once all of Bioconductor is ported to Julia, I will probably shed some joyous tears. R is really nice for some things, because of how easy it is to work with table-like data, and operations like applying a function over a vector or a matrix, etc are cake. However, it's also slow, leaky (and sometime seems to use more memory after gc()), and often difficult to debug because so many things fail silently. If it wasn't for so many standard bioinformatics packages being available only for R, I'd probably be using Python. Once we stop running microarrays, I'll probably be using R much less.

michaelochurch 13y ago

What are your thoughts on Clojure's suitability as a go-to language for scientific computing (except in lower-level, high-performance scenarios that might recommend Julia)? I think it has serious potential in this field. It's not there yet, but it's getting there, and the core.matrix standardization helps a lot.

collyw 13y ago
Its functional. Looking at the way bioinformaticians code, that will make it too complicated for many of them. SQL seems too much for many of them.
jamesjporter 13y ago
I agree with collyw; Lisp is too much for most scientists, who just care about getting their research done and can't be bother to learn all this weird FP/paren stuff [1]. Supplying an obvious, familiar syntax that looks like math written on paper for, e.g., matrix operations, is critical. This is actually one of the core tensions in Julia development imho: balancing having a sane, well designed language (from a programmer's perspective) vs. having a tool that allows scientists to quickly and easily crank out results. [1]: Note; I am not insinuating that Lisp is bad, I like it personally. Just relaying the response you will get from most practicing scientists who are not trained as programmers.
- michaelochurch 13y ago
  Just relaying the response you will get from most practicing scientists who are not trained as programmers. Is it that hard of a gap to cross? I was a math major who took a couple CS courses, and wouldn't say I was "trained as a programmer". I found Scheme pretty easy to get from the go. I tend to think that anyone who can get a PhD in the physical sciences can become a half-decent programmer-- if the desire is there, the intelligence and work ethic being established (one hopes, at least) by the degree.
  - jamesjporter 13y ago
    It's less a matter of ability and more a matter of having the time/motivation. If you took a bunch of practicing natural scientists and/or engineers and forced them somehow to enroll in a course teaching Scheme/Clojure/etc., would many of them do well? Almost certainly. If you gave them the choice between a Lisp and something like Julia or MATLAB to use in their everyday work when they need to do some computation? They'll probably choose the later because it's easy and familiar and doesn't have a huge up-front time investment; scientists tend to be very busy people.

xfax 13y ago

The big question for me is whether Julia will be able to maintain its "purity" as it gains adoption. R probably started out "beautiful" and "thought out" but has lost that edge with years of community driven development. It's also what make it so damn useful -- you can pretty much find anything on CRAN, often multiple implementations of it.

michaelhoffman 13y ago
Sorry, no. Even the core S language would not be described by very many users as "beautiful" according to modern standards.
mbq 13y ago
R is actually one of the most pure languages out there; it basically says "I have vectors; they can have missing values, be nested, and can have other vectors as attributes. And I have functions with lexical scoping. Now go and build the rest as you like." So people did this, one better, one worse -- but the core and beautiful stuff here is that all those approaches will work together and just do the job.

tokipin 13y ago

python is a deliberately-straightforward language. i don't see how anyone couldn't become highly proficient in it after writing one or two scripts

leephillips 13y ago
The straightforward design makes it fast to go from zero to having a working proficiency. But I'm not sure about "highly proficient": even experienced Pythoners get tripped up by things such as mutable function arguments, and it's often not clear why some simple-looking code is running slowly, and how to speed it up.
benhoyt 13y ago
Deliberately-straightforward -- agreed. But "highly proficient" after writing one or two scripts? That's quite a stretch. For instance, one of the questions I give in phone screens is for the candidate to write a program to count the number of occurrences of unique words in a text file. The "after writing one or two Python scripts" approach is something like this: counts = {} f = open('test.txt') lines = f.read().split('\n') for line in lines: for word in line.split(' '): if word: word = word.lower() if word in counts.keys(): counts[word] += 1 else: counts[word] = 1 f.close() count_items = [(count, word) for word, count in counts.items()] count_items.sort() for count, word in reversed(count_items): print word, count Whereas the "highly proficient" (and much simpler and more Pythonic) approach might look something like this: import collections counts = collections.Counter() with open('test.txt') as f: for line in f: for word in line.lower().split(): counts[word] += 1 for word, count in counts.most_common(): print word, count
- yohanatan 13y ago
  Actually that's not 'highly proficient'. This is: def read_words(words_file): return [word for line in open(words_file, 'r') for word in line.split()] len(set(read_words('test.txt')))

epaladin 13y ago

Once all of Bioconductor is ported to Julia, I will probably shed some joyous tears. R is really nice for some things, because of how easy it is to work with table-like data, and operations like applying a function over a vector or a matrix, etc are cake. However, it's also slow, leaky (and sometime seems to use more memory after gc()), and often difficult to debug because so many things fail silently. If it wasn't for so many standard bioinformatics packages being available only for R, I'd probably be using Python. Once we stop running microarrays, I'll probably be using R much less.