7 ms·
Switching from Common Lisp to Julia
- bluefox 9y agoThe author mentions trouble with portability and reliance on standard guarantees, and then goes on to talk about Julia, a single-implementation language that is "rapidly evolving" with breaking changes and no standard to speak of?
- __s 9y agoYou don't need to support multiple implementations when writing a library when there's only one Whereas to ignore CL & stick to SBCL will be working to fracture the community-- perhaps necessary, but CL is already a niche community
- sigjuice 9y agoIf Julia gets more popular, it is very likely that other Julia implementations will get developed.
- int_19h 9y agoNot necessarily. R doesn't have other implementations, for example, and Julia still has some way to go before it catches up with R popularity-wise. Or consider Python - it does have several implementations, but the vast majority of the ecosystem is centered around one, and many libraries don't work on the others in practice (especially anything written in other languages using the extensibility API).
- ACow_Adonis 9y agoWell, technically it can be argued that R is itself an implementation (of the language S). It's just that the alternative implementation has become defacto and replaced the original...
- kgwgk 9y agoThere are other R implementations, see https://radfordneal.wordpress.com/2013/07/24/deferred-evaluation-in-renjin-riposte-and-pqr/ https://radfordneal.wordpress.com/2013/07/24/deferred-evalua... for some pointers.
- int_19h 9y agoHow many of these are used in production?
- flavio81 9y ago>will be working to fracture the community-- The CL community knows one uses the implementation one needs. Lispers are using ECL whenever they need to embed with C code, ABCL to interact with JVM, CCL when they're with macs, this is not fragmentation at all, they are all compatible with the CL stanfard.
- coldtea 9y agoThat's because the "single-implementation language that is "rapidly evolving" with breaking changes and no standard to speak of" still offers better guarantees that the respective CL ecosystem.
- reikonomusha 9y agoI still don’t know what these guarantees are. The Lisp ecosystem is extremely well understood and very stable.
- bitwize 9y agoSingle implementation > standard clusterfucks. Look at Java, Perl, Python, Ruby, Php, etc., vs. CL, Scheme, C, C++...
- deleted 9y ago[deleted]
- pjmlp 9y agoAny language that actually matters in the market reaches a point where it has multiple implementations.
- eesmith 9y agoCPython, PyPy, Jython, IronPython, MicroPython, Nuitka, ..
- reikonomusha 9y agoYet most of those don’t actually play well with on another, with various major and minor incompatibilities.
- eesmith 9y agoAre there any languages where the different implementations "play well with each other"? Perhaps C? I remember all the hassles of working with different Fortran compilers (and C++ compilers) simply because of different function name mangling in the .o files. Which major incompatibilities are you thinking of between CPython and PyPy, which are the two I use most often? Are they more severe than the incompatibilities between the different Common Lisp, which the author describes? (I regard C extensions as implementation-specific features outside of the Python language. The different Common Lisp implementations also have implementation-specific features which are not portable.) MicroPython is deliberately not Python compatible (differences at http://docs.micropython.org/en/latest/pyboard/genrst/index.html http://docs.micropython.org/en/latest/pyboard/genrst/index.h... ) but the others have a goal of not having major incompatibilities.
- varjag 9y ago> The standard does not guarantee that this gives you an array of double-float: it may (if the implementation provides them), otherwise you get an array of element type T. This turned out to be a major difficulty for implementing portable scientific code in Common Lisp. Wouldn't sticking to say SBCL (that does give you specialized arrays) be equivalent to using a one-implementation language that does the same?
- frivoal 9y agoFor yourself, maybe. If you intend to share the code you write (or use code written by others), then implementation differences start to matter.
- coliveira 9y agoCommon Lisp has standard ways to define that code must run on particular implementations that support a certain feature. There is nothing wrong with developing libraries supported by just a few compilers.
- varjag 9y agoIn case with lack of specialized float arrays, the worst that is going to happen is performance degradation. It's not like they are some non-standard extension.
- nextos 9y agoSlightly off-topic, but what's your preferred stack for developing data-intensive applications that involve a lot of preprocessing, heavy statistics and machine learning plus a web front-end? Julia sounds quite appealing as a replacement for MATLAB or R with some Dylan-like semantics plus types and really efficient code generation on LLVM. I wish the Racket - Chez merger lead to something that targeted LLVM to be able to do front-end and back-end stuff using Scheme. A JVM-centric stack is one of my preferred alternatives. Clojure is great for data preprocessing and manipulation, plus ClojureScript for coding all front-end. Datomic, core.spec, core.logic, anglican, plumatic.plumbing just to name a few are a joy to use. Then there's Scala, which is also a great asset, and tons of fantastic Java / Scala libraries like Stanford NLP, Markov Logic Networks (Tuffy, RockIT...), Factorie, Deeplearning4j, etc. Sadly, Scala-Clojure interop is not very good. Python is the other obvious option, with tons of good libraries, including a fantastic data analysis ecosystem built around NumPy, SciPy, Matplotlib and Pandas. Plus most deep learning libraries targeting Python first. I just feel the language doesn't scale that well, although things like Numba or Cython help.
- lasfter 9y agoHave you considered Lua? You can develop web apps with Lapis [1] and do your data analysis with Torch [2]. The tradeoffs compared to Python: LuaJIT is much faster, which should ease your scaling worries, but the ecosystem is not as developed. [1] http://leafo.net/lapis http://leafo.net/lapis [2] http://torch.ch/ http://torch.ch/
- aldanor 9y agoIt’s really not fair to compare CPython with LuaJIT. If you compare LuaJIT with PyPy or Numba, the “much faster” argument will simply not be true. What’s different is that both of the above only cover a part of the language, but for specialised (e.g. numeric) application that’s often not a problem.
- whyever 9y agoI think LuaJIT is generally faster, not only because Python is harder to optimize.
- reikonomusha 9y ago1. The standard doesn’t guarantee double float arrays will be efficient. Most implementations do it (CCL SBCL CMUCL ECL LW ACL SCL ...), except for a few unpopular implementations. Seven do what you want, but a couple don’t. What gives? Julia has neither a standard nor more than one implementation. 2. No efficient parametric/ad hoc polymorphism. You can get far with inlining. You can also use libraries specifically built with this in mind. But you’re right, it’s not a built-in feature. But scientific codes are generally not super polymorphic anyway. Look at all the FORTRAN. But besides that, Lisp let’s you build the language you want to use, and have it interoperate for the rest of the ecosystem. If you are surprised that building a language is work, then Lisp may not be the choice for you. If you’re willing to design the language you want for numerical computing, Lisp has the essential data types and primitives to allow you to do that, even portably. If you want off-the-shelf features of Julia, then using Julia is more optimal than Lisp. If you want a language that will readily adapt without changing under your feet, without risking your programs not working 2, 10, or 25 years from now, then Lisp might be a good choice.
- baldfat 9y agoFor DSL (DOmain Specific Languages) I use R for my data work. For general computing I use to be a Python and Lua person but now I use Racket, because I can create my own DSL for what I am doing. https://beautifulracket.com/ https://beautifulracket.com/ I totally get the reason why he would go to Julia for doing data jobs. Data structures and already done libraries are very valuable.
- lucozade 9y agoScientific programming is heavily polymorphic. You just need to look at the interface to most well used libraries like BLAS. Fortran and C conventionally deal with this by adding the relevant type signature to the function name. Scientific programming is also, regularly, polymorphic in 2, (often parametric) types. This heavily motivated the design choices in Julia esp the multiple dispatch infrastructure that is descended from Dylan.
- reikonomusha 9y ago
- flavio81 9y agoNote: I generally applaud Julia, it was designed taking many good choices. However, I really don't understand how the author complains that CL doesn't "have" certain features that in fact are available by choosing a suitable CL implementation. But no, he prefers switching to a language with no standard (yet) and only one implementation...
- baldfat 9y agoHe mentioned that they have them it is that they are just not updated or standardized since there are one person developing the library.
- ACow_Adonis 9y agoKeeping in mind that Julia appears to be very very close to common lisp, some minor differences that you appear to lose going over: - 1) lisp 1 vs lisp 2 - 2) Matlab syntax (y tho?), infix, and sygils everywhere - 3) Clos and metaobject protocol and ability to adjust these as easily at run-time You do appear to gain some things though in all honesty. I'm trying to program up an dataframe/analytics type program/library in Common Lisp as we speak, and I'll admit, getting that stuff right and efficient is hard work. Gives you immediate respect for anyone who has done the same in other languages. Additionally: - 1) Parametric types and dispatch on them - 2) Potentially inlining and compiler optimisations and integration around their generic functions. My gut says SBCL would need some compiler magic and metaobject protocol type stuff to do the same, and it would no longer be standard Common Lisp. That's not necessarily a bad thing, there are some warts around in-builts/objects/generic functions in Common Lisp, but it would be a fracturing of the community to update/change them.
- reikonomusha 9y agoYou’d only be able to inline if you know that your call-site is monomorphic. Also, there’s a Lisp library [0] for inlining generic functions that give you huge efficiency gains for certain use-cases (especially monomorphic ones). I couldn’t imagine this being a _library_ in any other language. [0] https://github.com/guicho271828/inlined-generic-function https://github.com/guicho271828/inlined-generic-function
- ChrisRackauckas 9y agoJulia does this automatically, it you force it with @inline
- nerdponx 9y ago1) lisp 1 vs lisp 2 Eh. Emacs vs Vim. 2) Matlab syntax (y tho?), infix, and sigils everywhere The target audience is scientific and mathematical users, not necessarily "programmers", and certainly not "software engineers". Matlab syntax, infix, and sygils read like math notation. The target use-case is mathematical and scientific programming. This is a benefit in my opinion. Also, with respect to sigils, I can think of at least one popular sigil in Common Lisp: the #' reader macro for accessing functions! So much for Lisp-1 vs Lisp-2. 3) Clos and metaobject protocol and ability to adjust these as easily at run-time Fair, but do you really need CLOS with a delightful type system and multiple dispatch?
- flavio81 9y ago>"The standard does not guarantee that this gives you an array of double-float: it may (if the implementation provides them), otherwise you get an array of element type T." Well, many Common Lisp implementations, which includes most of the famous ones like SBCL, CCL, and ABCL, will give you exactly... ... an array of double-float!! So where is the problem? >"However, this gets worse: while you can tell a function that operates on arrays that these arrays have element type double-float, you cannot dispatch on this, as Common Lisp does not have parametric types." Well, there are many options. First, let me reiterate that an array of a certain element type, stays of that element type. Example: CL-USER> (defparameter *a* (make-array 0 :element-type 'double-float )) *A* CL-USER> (type-of *a*) (SIMPLE-ARRAY DOUBLE-FLOAT (0)) I'm going to give the simplest, quick&dirty options: If you're using arrays of different element-types, option A is write your function and just use the ETYPECASE function to do the dispatch according to the element type, so you can ensure that you select the code correct to the element type. Use the DECLARE declaration specifier so the Lisp compiler knows which type is your array /array elements and thus the machine language code produced is optimal. Option B is simple, just define classes or structs, one for each array type you intend to use; and then take advantage of CLOS dispatch so invoking the generic function dispatches to the correct code for each array. These are two options which don't even need any macro solutions; there are many more options as well, i'm just proposing two. Option C, more elegant, could be perhaps using Fare's "LIL" (Lisp Interface Library)? https://common-lisp.net/~frideau/lil-ilc2012/lil-ilc2012.html https://common-lisp.net/~frideau/lil-ilc2012/lil-ilc2012.htm... I really think Julia is a nicely designed language, and is a language I often tell people to take a look at. However, if the author already has a program in Common Lisp, and has good experience of Common Lisp (which, by the way, means the author might be a quite skilled programmer), why not dig deeper into the facilities that Common Lisp brings to solve those problems? NOTE: This is an edited version of the comment i left on the page.
- _ph_ 9y agoIt is my biggest complaint with Common Lisp, that while the language supports efficient optimization based on type information, it is often a bit cumbersome to provide them. However, if you target is to write high performance numeric code, you should consider SBCL first of all. It is excellent in using type-information and has a very good type-inferencer, so the task is a bit easier, and it produces excellent code.
- ChrisRackauckas 9y agoThe problem I have with Lisp for scientific computing is that libraries matter. You really want to standardize not only what you are calling but also your notation because most of the people that you want to use the libraries are scientific domain experts and not "programmers": they may know like 20 functions and that's it (and that's fine!). Lisps tend to love the "you can build anything in it! You can make all of your own syntax in it!" and I think that goes too far. Julia lets you do this with macros, but then there's a good convention to not overuse this. There has to be a balance between being too structured (which hurts innovation) and too dynamic (which makes it hard to learn someone's new library because everything is too different), and I think Julia strikes the right balance.