3 ms·

Which implementation of Lisp is best when speed matters?

15 points by palish 20y ago

dfranke 20y ago
Probably SBCL.
- abstractbill 20y ago
  Seconded. Just be aware that its implementation of equalp for strings is dog-slow (compared to clisp at least).
- gibsonf1 20y ago
  Ok - we are diving in: Today my partner and I decided to drop Lispworks (and especially Windows) and go with SBCL and Ubuntu for our first server box which we'll host at our office. We are on schedule to go live in about 2 months. He had been looking into SBCL for a while, and is convinced its the way to go. If we find problems with it, we (or more likely he as he is technically amazing compared to my self-taught lisp hacking) will contribute to the source code.
- busy_beaver 20y ago
  If Scheme will do, Chicken, Bigloo, and Gambit all compile to C code as an intermediate step (which you then run through gcc or whatever to generate the binary). Bigloo will also generate JVM and .NET output (.NET is still experimental). I haven't worked much with Gambit, but have fooled around with both Chicken and Bigloo. Of the two, I found Chicken to be the friendliest (it also seems to have a somewhat more active community). Another possible advantage of Chicken is that it's directly supported by SWIG. There was some activity on the Chicken mailing list a few months back about porting some game libraries. I don't know what the current status of that might be.
gibsonf1 20y ago
Here's a discussion about the various Lisp options: http://programming.reddit.com/info/1edrw/comments http://programming.reddit.com/info/1edrw/comments We use Lispworks, but if there is a better (especially freer) option, we would switch in a heartbeat. (We don't use Allegro Cache - so that wouldn't hold us back from switching). So far our systems are running very fast with LW - no complaints on speed.
marketer 20y ago
The question should be: if you want speed, is it really worth using lisp? Maybe your time would be better spent writing a c module and then interfacing with the lisp interpreter. I'm no lisp hater, it's a great language that offers some neat abstractions. But to really optimize code, you have to be intimately familiar with how data structures are implemented within the interpreter. You can avoid this problem by using C.
- herdrick 20y ago
  Actually you're likely to write slower code if you start with C, instead of starting with a high level language and then after finding out what is too slow, writing those parts in C. Except it won't be too slow. Especially if you use one of the popular implementations of Common Lisp or Scheme. They're pretty fast. More to the point, you're killing yourself on productivity if you are approaching things from a performance-central perspective. My advice to you is to forget C.
  - marketer 20y ago
    It's sometimes difficult to fix performance issues in your app unless you know what's causing the slow-down in the interpreter, but that's usually a pain. Lisp is a constructive language, so a lot of time is spent doing memory allocation. Interpreters often optimize that by pre-allocation and such, but to diagnose that you need to know exactly when memory is allocated, etc.. AN example: I was using the python heap module to implement a priority queue with several hundred thousand values. The memory allocation caused by the dynamic growth of the list was crippling the performance. I re-implemented it as a binary heap in C, and it was asymptotically faster.
    - ecuzzillo 20y ago
      Lisp is compiled, not interpreted, most of the time. No one uses interpreted Lisp in production; it's all compiled. And it's compiled to very fast code, much of the time, particularly when you add type declarations.
    - shiro 20y ago
mattculbreth 20y ago
Well I know that Paul has Arc running on MzScheme, and I've found this site to be plenty fast. Not exactly what you asked, but if you haven't started coding you could consider that.
- dfranke 20y ago
  I've had bad experiences with PLT. I used it for my senior project and ended up having to rewrite in Common Lisp because I kept running into bugs in the interpreter. I'm guessing that Arc compiles down to a pretty well-tested subset of PLT, but if you start messing around with obscure libraries or really beat up hard on the macro system, you're asking for trouble.
papersmith 20y ago
The ones that compile to native code: - Lispworks (commercial) - Allegro CL (commercial) - SBCL (my pick) - CMUCL - OpenMCL (Mac only) - GNU CL (not standard-comforming, but works well with other GCC binaries) - Corman CL (commercial, windows only) This is just off of the top of my head, please add anything I've missed. Assuming you're doing web apps: http://groups.google.com/group/comp.lang.lisp/msg/352f440baa7aeb94 http://groups.google.com/group/comp.lang.lisp/msg/352f440baa7aeb94 They (cl-user.net) have pretty impressive results with a single Lispworks process in a production environment; ~3.3 requests/sec, ~5million requests/day, uptime 456 days. It seems that historically when you want performance+portability+paid-support you go for one of the commercial implementations. CMUCL used to be the choice among free implementations for performance. SBCL inherits the performance of CMUCL, and they reworte some of the layers to be more portable. They had some threading issues in FreeBSD, but they're now resolved and pretty solid after recently passing version 1.0. SBCL now has a pretty substantial user base, so you should be able to get help by just asking around.
- omouse 20y ago
  And now to choose which one to use! Fun! :P
chris_l 20y ago
What are you doing and which part is the speed bottleneck? As I've mentioned before somewhere on here, I'm building a blog search engine in Common Lisp. That is indeed performance critical as you can probably imagine and I've found SBCL to work very well.
palish 20y ago
Cool, thanks for all the great replies. Here's a little about what I'm doing. I've worked in the video game industry for the last two years, and I've been messing around with Direct3D/OpenGL for about 5. I believe wrapping these API's in an extremely high level language like Lisp might speed up creation of a video game by about an order of magnitude, especially if I expose only very high level graphical concepts to Lisp, instead of just exposing every API function. Speed is crucial. I've layered Ruby onto my engine, but the problem was that if I wanted 60 frames per second, I could only do 16,000 Ruby if-statements per frame. That wouldn't work at all. Even if the high level language I choose is only 4-10 times slower than C++, that would be fine. Ruby was about 100-200 times slower. Investigating Lisp, I only require two things: 1) Somehow, I expose C functions to Lisp. These functions take standard types as arguments (float, int, double, char ...). For example, I might expose, from C: (Edit: argh, I can't write a star on News.YC. The following should read: "const char (star)str") void print_to_game_console(const char str); Then in lisp, I'd then be able to do (print_to_game_console "Odelay!") Something along those lines. 2) Lisp only has access to what I expose. I won't get into the boring details, but I'm distributing the Lisp script files for anyone to edit and run on other's machines, so sandboxing Lisp somehow is also pretty crucial. As long as under no circumstances can Lisp access the file system, then everything else is fine. Thanks everyone! Shawn
- aristus 20y ago
  The folks who made Crash Bandicoot did something like that in AllegroCL about 10 years ago: http://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp http://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp
  - palish 20y ago
    Great! It worked for someone else, so I'm not insane! Now I just have to figure out how to do it.
    - aristus 20y ago
      They had some killer stuff in it, from what I can tell from interviews posted here & there. But when Sony bought them they drowned GOAL in the bathtub and ported to their own inhouse platform. Kind of sad.
vikram 20y ago
SBCL on linux. Lightening fast. In some benchmarks it even beat C. The thing to remember with lisp is that you'll need to work to get your program to be efficient. It's not too much work but it's definitely something you'll need to do.
- palish 20y ago
  Thanks, I'll remember! It seems like SBCL is the general consensus. I hope I can figure out a way to get it to interface with a C library.
hayeah2 20y ago
use assembling language then wait for Greenspun's 10th.
albertcardona 20y ago
When speed matters, what you need is a good hacker to make a smart-ass elegant program that goes around the bottle-necks. But I'll attempt not to avoid your question altogether: from my own personal testing, SBCL is fastest. My bias is my own benchmarks. Find your own set.

15 points by palish 20y ago

dfranke 20y ago

Probably SBCL.

abstractbill 20y ago
Seconded. Just be aware that its implementation of equalp for strings is dog-slow (compared to clisp at least).
gibsonf1 20y ago
Ok - we are diving in: Today my partner and I decided to drop Lispworks (and especially Windows) and go with SBCL and Ubuntu for our first server box which we'll host at our office. We are on schedule to go live in about 2 months. He had been looking into SBCL for a while, and is convinced its the way to go. If we find problems with it, we (or more likely he as he is technically amazing compared to my self-taught lisp hacking) will contribute to the source code.
busy_beaver 20y ago
If Scheme will do, Chicken, Bigloo, and Gambit all compile to C code as an intermediate step (which you then run through gcc or whatever to generate the binary). Bigloo will also generate JVM and .NET output (.NET is still experimental). I haven't worked much with Gambit, but have fooled around with both Chicken and Bigloo. Of the two, I found Chicken to be the friendliest (it also seems to have a somewhat more active community). Another possible advantage of Chicken is that it's directly supported by SWIG. There was some activity on the Chicken mailing list a few months back about porting some game libraries. I don't know what the current status of that might be.

gibsonf1 20y ago

Here's a discussion about the various Lisp options: http://programming.reddit.com/info/1edrw/comments http://programming.reddit.com/info/1edrw/comments We use Lispworks, but if there is a better (especially freer) option, we would switch in a heartbeat. (We don't use Allegro Cache - so that wouldn't hold us back from switching). So far our systems are running very fast with LW - no complaints on speed.

marketer 20y ago

The question should be: if you want speed, is it really worth using lisp? Maybe your time would be better spent writing a c module and then interfacing with the lisp interpreter. I'm no lisp hater, it's a great language that offers some neat abstractions. But to really optimize code, you have to be intimately familiar with how data structures are implemented within the interpreter. You can avoid this problem by using C.

herdrick 20y ago
Actually you're likely to write slower code if you start with C, instead of starting with a high level language and then after finding out what is too slow, writing those parts in C. Except it won't be too slow. Especially if you use one of the popular implementations of Common Lisp or Scheme. They're pretty fast. More to the point, you're killing yourself on productivity if you are approaching things from a performance-central perspective. My advice to you is to forget C.
- marketer 20y ago
  It's sometimes difficult to fix performance issues in your app unless you know what's causing the slow-down in the interpreter, but that's usually a pain. Lisp is a constructive language, so a lot of time is spent doing memory allocation. Interpreters often optimize that by pre-allocation and such, but to diagnose that you need to know exactly when memory is allocated, etc.. AN example: I was using the python heap module to implement a priority queue with several hundred thousand values. The memory allocation caused by the dynamic growth of the list was crippling the performance. I re-implemented it as a binary heap in C, and it was asymptotically faster.
  - ecuzzillo 20y ago
    Lisp is compiled, not interpreted, most of the time. No one uses interpreted Lisp in production; it's all compiled. And it's compiled to very fast code, much of the time, particularly when you add type declarations.
  - shiro 20y ago

mattculbreth 20y ago

Well I know that Paul has Arc running on MzScheme, and I've found this site to be plenty fast. Not exactly what you asked, but if you haven't started coding you could consider that.

dfranke 20y ago
I've had bad experiences with PLT. I used it for my senior project and ended up having to rewrite in Common Lisp because I kept running into bugs in the interpreter. I'm guessing that Arc compiles down to a pretty well-tested subset of PLT, but if you start messing around with obscure libraries or really beat up hard on the macro system, you're asking for trouble.

papersmith 20y ago

The ones that compile to native code: - Lispworks (commercial) - Allegro CL (commercial) - SBCL (my pick) - CMUCL - OpenMCL (Mac only) - GNU CL (not standard-comforming, but works well with other GCC binaries) - Corman CL (commercial, windows only) This is just off of the top of my head, please add anything I've missed. Assuming you're doing web apps: http://groups.google.com/group/comp.lang.lisp/msg/352f440baa7aeb94 http://groups.google.com/group/comp.lang.lisp/msg/352f440baa7aeb94 They (cl-user.net) have pretty impressive results with a single Lispworks process in a production environment; ~3.3 requests/sec, ~5million requests/day, uptime 456 days. It seems that historically when you want performance+portability+paid-support you go for one of the commercial implementations. CMUCL used to be the choice among free implementations for performance. SBCL inherits the performance of CMUCL, and they reworte some of the layers to be more portable. They had some threading issues in FreeBSD, but they're now resolved and pretty solid after recently passing version 1.0. SBCL now has a pretty substantial user base, so you should be able to get help by just asking around.

omouse 20y ago
And now to choose which one to use! Fun! :P

chris_l 20y ago

What are you doing and which part is the speed bottleneck? As I've mentioned before somewhere on here, I'm building a blog search engine in Common Lisp. That is indeed performance critical as you can probably imagine and I've found SBCL to work very well.

palish 20y ago

Cool, thanks for all the great replies. Here's a little about what I'm doing. I've worked in the video game industry for the last two years, and I've been messing around with Direct3D/OpenGL for about 5. I believe wrapping these API's in an extremely high level language like Lisp might speed up creation of a video game by about an order of magnitude, especially if I expose only very high level graphical concepts to Lisp, instead of just exposing every API function. Speed is crucial. I've layered Ruby onto my engine, but the problem was that if I wanted 60 frames per second, I could only do 16,000 Ruby if-statements per frame. That wouldn't work at all. Even if the high level language I choose is only 4-10 times slower than C++, that would be fine. Ruby was about 100-200 times slower. Investigating Lisp, I only require two things: 1) Somehow, I expose C functions to Lisp. These functions take standard types as arguments (float, int, double, char ...). For example, I might expose, from C: (Edit: argh, I can't write a star on News.YC. The following should read: "const char (star)str") void print_to_game_console(const char str); Then in lisp, I'd then be able to do (print_to_game_console "Odelay!") Something along those lines. 2) Lisp only has access to what I expose. I won't get into the boring details, but I'm distributing the Lisp script files for anyone to edit and run on other's machines, so sandboxing Lisp somehow is also pretty crucial. As long as under no circumstances can Lisp access the file system, then everything else is fine. Thanks everyone! Shawn

aristus 20y ago
The folks who made Crash Bandicoot did something like that in AllegroCL about 10 years ago: http://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp http://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp
- palish 20y ago
  Great! It worked for someone else, so I'm not insane! Now I just have to figure out how to do it.
  - aristus 20y ago
    They had some killer stuff in it, from what I can tell from interviews posted here & there. But when Sony bought them they drowned GOAL in the bathtub and ported to their own inhouse platform. Kind of sad.

vikram 20y ago

SBCL on linux. Lightening fast. In some benchmarks it even beat C. The thing to remember with lisp is that you'll need to work to get your program to be efficient. It's not too much work but it's definitely something you'll need to do.

palish 20y ago
Thanks, I'll remember! It seems like SBCL is the general consensus. I hope I can figure out a way to get it to interface with a C library.

hayeah2 20y ago

use assembling language then wait for Greenspun's 10th.

albertcardona 20y ago

When speed matters, what you need is a good hacker to make a smart-ass elegant program that goes around the bottle-necks. But I'll attempt not to avoid your question altogether: from my own personal testing, SBCL is fastest. My bias is my own benchmarks. Find your own set.