7 ms·
Better performance in App Engine with new Lisp language Clojure
- mcav 16y agoA question I've been meaning to ask but didn't want to make a completely new post about it: If you could write a server (not a plain HTTP webapp necessarily) in Clojure, Python, or Node.JS, provided you know all three, which would you choose and why?
- francoisdevlin 16y agoI'd go with Clojure if you need any of the following: 1. Concurrency 2. The most (largest qty of) available FOSS libs anywhere 3. Speed 4. The ability to extend the language 5. WORA 6. Text/string processing Python will win in scripting/*NIX integration, though. Don't really know enough about Node.JS
- icey 16y agoNode is pretty interesting and seems to have pretty good performance, but I wouldn't build anything critical on it until at least version 0.2. As far as I know, there are semi-frequent breaking changes. Ryan Dahl has said that API stability should come with version 0.2.
- caffeine 16y ago"I wouldn't build anything critical on it until at least version 0.2" Ha! It's a brave new world. It used to be you needed a whole version number under your belt before critical stuff could be built..
- andrewvc 16y agoAgreed, node development is going at a breakneck pace, but you should count on your code not working 6 months down the line without at least as few tweaks. BTW, the new Buffer objects are awesome, and provide massive speed increases with binary data.
- rglullis 16y agoI really have a hard time seeing Python losing to Clojure on (2), (5) and (6). What kind of software/library would you find on Clojure but not Python? How is Python not WORA? Where does Python lack in text processing?
- francoisdevlin 16y agoJava & the JVM give me 2 & 5. While Pythons text processing is great, I think functional composition & juxtaposition give Clojure and edge. Plus, strings are simply sequences, so the core language works well with them too.
- KirinDave 16y ago> What kind of software/library would you find on Clojure but not Python? Java has a lot of great libraries for which there are not terrific examples in Python. Clojure gets all of them. > Where does Python lack in text processing? Clojure handles large strings better than Python (unless you start using substrings, in which case an underlying java bug hits you). (oh, and Clojure is not a deliberately and comically crippled language).
- rufugee 16y agoHow is Python deliberately and comically crippled? I'm honestly curious.
- hugoc 16y agoI think he may be refering to the GIL.
- KirinDave 16y agoThat and certain deficiencies spawned by a desire for “simplicity.”
- technomancy 16y agoDon't forget one-line lambdas.
- strlen 16y ago> 2. The most (largest qty of) available FOSS libs anywhere Quick heads up. The ease of FFI (in Clojure's case with Java) is not unique to Clojure and it isn't necessarily a great thing. It's also an issue with Common Lisp: as FFI with C is easy, there's less incentive to develop idiomatic libraries for the language. E.g., Common Lisp lacks a standard modern way to network I/O, but it's largely been ignored as larger users of the language e.g., ITA could always build and standardize upon an internal C-based extension. That being said, Clojure is great for reasons beyond running on the JVM: a unique approach to concurrency, incorporation of ML-family features, first order data structures beyond lists (maps, vectors) and the fact it's a Lisp-1 with a great macro facility. I'd love to see Clojure also go beyond the JVM and onto CLR, Parrot and LLVM. Interestingly enough the other prominent "next generation Lisp", arc, is also a Lisp-1 with true macros (as opposed to Scheme's hygienic macros).
- prospero 16y agoIt's very common to write an idiomatic wrapper for a useful Java library. Your complaint seems to be that those wrappers won't survive the transition onto another platform. I don't really see why that's a problem. Why target another platform if not for its libraries? If Clojure is exactly the same on the JVM as it is on LLVM, why should it ever leave the JVM?
- mahmud 16y agoCommon Lisp lacks a standard modern way to network I/O http://common-lisp.net/project/usocket/ http://common-lisp.net/project/usocket/
- draegtun 16y agoThe most (largest qty of) available FOSS libs anywhere Do you literally mean anywhere or compared to other GAE languages? If so do you have a rough estimate on how many Java/Clojure FOSS libraries/modules/packages there are?
- mattculbreth 16y agoI actually did this purely for fun. Could have done Python but I've been meaning to learn Clojure anyway. Check out http://github.com/mattc58/animate http://github.com/mattc58/animate for the beginning of my web server in Clojure.
- jacquesm 16y agoSuper, thanks! I've asked if there was some kind of a reference web application in clojure that I could study before to show me 'how it's done', but got no takers, so I'll use your code as a study object if you're ok with that.
- mattculbreth 16y agoSure absolutely. It's been a good learning experience. As soon as I finish the blogging features of it I'll start blogging about it. :)
- jacquesm 16y agoThis is something I've been noticing as long as I've been prorgramming. If you are programming a tool then the tool will be a requirement for building the tool. For instance, you can't really build an assembler for a new architecture from scratch without some form of assembler and so on. The bootstrapping sequence seems to be endless, you could be blogging about writing your blogging tool if you had it already. We always seem to need some scaffolding underneath it, just one more turtle on the stack. No more hand calculating branches these days, /me is happy ;)
- mattculbreth 16y agoThis is all for fun and for great justice, so I'm cool reinventing the wheel.
- weavejester 16y agoMark McGranaghan's Ring library might be a good place to start. The Ring spec is the de facto base library for web applications in Clojure, much as Servlets are in Java, and Rack is in Ruby.
- kixxauth 16y agoKeep in mind the loading request scenario for running GAE apps that are implemented on the JVM. http://code.google.com/appengine/kb/java.html#What_Is_A_Loading_Request http://code.google.com/appengine/kb/java.html#What_Is_A_Load...
- psranga 16y agoI've looked off and on into using App Engine's Java interface. But even the "hello world" app looked much more bureaucratic than the Python equivalent (it's due to Google trying to do things the Java way). Do you still have to deal with Java's baggage even if you run Clojure on App Engine? How much can you abstract away? Any libs?
- francoisdevlin 16y agoOccasional you do have to deal with Java's baggage in Clojure. The best example I can think of right now is the build process. The consensus is to use Maven or the Clojure tool Leiningen. The good news is that you also get Java's "Deployment for Dummies" solutions (just copy the JAR/WAR and you're good to go), though.
- jwr 16y agoLeiningen is actually pretty cool. It uses Maven under the hood (but you don't have to know that). The nice thing is that if you suddenly decide you need to use a Java library (say, one of the Apache Commons libs), you just add one line to your project.clj file, do "lein deps" and boom, you can use the library in your project. Having written quite a bit of software in Common Lisp, this is a welcome change.
- aerique 16y agoHuh?, how's this easier than adding a package entry to the depends-on form in an asdf-file in CL?
- LaPingvino 16y agoYou can add jars from any JVM language and use it directly with the great Clojure Java-interop.
- weavejester 16y agoLeiningen only uses Maven's dependency resolution API. Everything else is written in Clojure.
- andreyf 16y agoOn Google App Engine, Google's cloud computing platform, developers can officially choose between two languages -- Python and Java -- and both run on the Java Virtual Machine (JVM). If they've got Python running on the JVM, why aren't they contributing back to Jython (or releasing their own Python->JVM bytecode compiler)?
- rbanffy 16y agoThis is rather unexpected. AFAIK, the Python they have there did not run on top of JVMs.
- dustingetz 16y agoarticle implication (GAE's python support is actually Jython) is incorrect http://jython.xhaus.com/jython-on-google-appengine-why-bother/ http://jython.xhaus.com/jython-on-google-appengine-why-bothe...
- sgk284 16y agoWhere are you getting that quote from? They don't run Python on the JVM.
- csytan 16y agoI think the article has been changed after andreyf quoted it.
- waratuman 16y agoWhere do the 30% and 50% numbers come from?
- staunch 16y agoI took it to be an educated guess, not a statement based on hard data.
- zach 16y agoSo I was in an App Engine workshop at PyCon and asked an engineer about running Clojure on App Engine, which resulted in a confusing exchange. Then I realized they thought I was talking about Google's Closure Library. Argh! I guess this wasn't on the radar. This shows a lot of promise, and it would be great if App Engine could become (in a way) Clojure's Heroku.
- caffeine 16y agoIt doesn't really say much about performance...
- jbellis 16y agoIt doesn't say anything about performance. I checked find-in-page in case I somehow missed it. :)
- mace 16y agoThe blog referenced in this post has more details on their use of Clojure: http://www.hackers-with-attitude.com/search/label/Clojure http://www.hackers-with-attitude.com/search/label/Clojure
- defdac 16y agoThis is a very readworthy slide from the CTO of freiheit and the project: http://www.slideshare.net/smartrevolution/how-a-clojure-pet-project-turned-into-a-fullblown-cloudcomputing-webapp http://www.slideshare.net/smartrevolution/how-a-clojure-pet-... I'm curious about the "how to get rid of Eclipse"-part. How do you "Go to implementation" and "Find usages" in Clojure without a 7 key-combination in Emacs? I would love to take a Clojure-IDE for a spin.
- evgen 16y agoNetBeans with Enclojure is probably the best option for those who do not already know Emacs (or Counterclockwise if you are already in the Eclipse ecosystem.) I use Emacs for Erlang and Python coding, but for some reason the swank-clojure setup never really clicked for me. I keep at it and I am sure at some point I will not think that my Clojure dev environment is not more of a hinderance than a help, but if you are not already using Emacs I would probably suggest you start off by looking at the list here: http://www.bestinclass.dk/index.php/2010/03/clojure-ides-the-grand-tour-getting-started/ http://www.bestinclass.dk/index.php/2010/03/clojure-ides-the...
- technomancy 16y ago> How do you "Go to implementation" and "Find usages" in Clojure without a 7 key-combination in Emacs? Go to implementation is M-. with SLIME in Emacs. It can even open up Clojure source in jar files on the classpath and let you edit (and save!) them from there. Find usages is a little more complicated: C-c C-w c, but not too bad.
- projectileboy 16y agoLa Clojure for IntelliJ is getting pretty good, and is a nice option if you have multi-language application. I did a little work on some of the original code, some of which also survives as the "Clo-Jet" plugin, if you want something lighter-weight.
- jheriko 16y agoInteresting, but I cringed when I read this: "Writing less code requires less time." Seriously? Then again, it /is/ a "to-do" app...
- btilly 16y agoSeriously. Studies from a wide variety of languages over several decades have found that average lines of code/developer is reasonably close to constant independent of what language the developer is using. Therefore languages and libraries that result in less code to get your work done generally result in higher productivity.
- ShardPhoenix 16y agoI'd be interested in seeing similar studies with relatively modern languages, though. The only ones I've seen have been comparing things like C, Pascal, and Cobol.
- va_coder 16y agoJVM apps on appengine start up slow. I encourage anyone looking into using a JVM app on appengine to put together a simple app, wait 10 minutes after deployment, and then hit the url and see how long it takes to load. It bothers me that people don't mention this.
- Quiark 16y agoIt's called "loading request" and Google mentions it in the docs. They claim that with the number of users rising, relatively few people will actually experience it. But the ~10 seconds it takes to load my clojure app surely is not pleasant.
- swannodette 16y agoBecause if your server is getting hammered the JVM is going to be warmed up and really fast.
- wynand 16y agoPosts like this really make me want to move to Clojure. Python is now my go-to language, especially because of Scipy, NetworkX and bridges like RPy. I would definitely put in the time if someone with mathematical modelling experience can support the idea that I'll be more productive in Clojure. Anyone? I really would love to see a list of testimonials and anti-testimonials of people from various fields that switched to Clojure and either got much more done or got burned. Sorry, I realise that this is a wee bit off the main topic.
- icey 16y agoYou should spend a little time playing with Clojure. I don't know if you'll be more productive in it or not; people seem to either prefer lisps strongly or dislike them strongly. If you like lisp then you'll have no problems being tremendously productive, especially with libraries like Incanter available.
- jacquesm 16y agoI very much like the elegance of lisp code, but I'm still a 'reader', not a 'writer'. I hope to change that one of these days, but as long as that magic 'click' hasn't happened I'm not confident enough to jump in and start some major project. Maybe that's what I should lose, the paralysis of starting out down the wrong track and simply do stuff wrong until I find a better way.
- corruption 16y agoDo note, python has much better support for a huge variety of modelling techniques. I looked at incanter and it's obvious that it's in it's infancy c.f. R or numpy/scipy and associated libraries. At the moment my vote would be for python or clojure + r.
- mark_l_watson 16y agoI am using Clojure a lot right now on a customer's project, a cool language for sure. That said, I have doubts about Clojure + AppEngine because of the loading request times. I can get a Java + AppEngine app to do about a 1 to 2 second loading request time by not using JDO and minimizing dependencies. Unfortunately, loading the Clojure JARs (or the JRuby JARs) increase the loading request time. Sometimes when I go to TheDeadline, I see almost a 10 second delay on the first page load. Does not happen often, but it does happen.