20 ms·

Dylan: the harsh realities of the market

172 points by kryptiskt 12y ago

eudox 12y ago
The article hits rather close to home, as someone who, for better or worse, is committed to improving Common Lisp's ecosystem. I often feel this existential dread of "Is it worth it? Whose lives will it change? Will I spend years labouring in obscurity for nothing?". And while the answer to all those questions is probably not what I'd like to hear, I still do it. This is why ecosystems are, for most people, more valuable than intrinsic language features: Tribalism along the lines of "We have X thousand people backing us up"/"X thousand devs can't go wrong". People don't care about monads or macros, they care about feeling like they're part of a large community.
- pnathan 12y ago
  As someone also invested in Common Lisp, it.... hit really close to home.
- seanmcdirmid 12y ago
  Its even worse for the PL designer inventing something new without even a small ecosystem in place; I ask myself everyday "is it worth it?"
  - freyrs3 12y ago
    Have to figure it takes 10-20 years to grow an industrial strength language. It takes a lot of faith in an idea or stubbornness ( probably both ) to go through that struggle. I admire people with that kind of dedication and I wish exploring the language design space wasn't such an enormous investment.
  - sitkack 12y ago
    It is worth it. And your case it isn't the language per se, but spreading the ideas. The language is an ephemeral vehicle.
  - tetha 12y ago
    The important question isn't necessarily: Is it worth it, without the ecosystem? The more interesting questions is: What does this language add? For example, I need to deal with some unix process management tasks. I'm not crazy enough to do them in straight C, so I'm using python for it's strong C bindings, so I can use system calls in C with python level control flow. However, there's a lot to be desired, because dealing with all this concurrency and inherent raciness of the system in a language like python is a royal pain in the rearside. Similar things with java and performant programming. Some of my major programs at work don't use the java ecosystem outside of log4j, junit, apache commons lang and apache commons collections. They use java, because java is darned fast without being as brittle as C or as nuts as C++. So java adds a lot of really cheap development security. Designing yet another language that puts the existing features together with just a tiny difference doesn't add anything. I've enjoyed looking at experimental languages, and I've enjoyed tinkering around with ideas myself some time ago. But by now, I've accepted I'm not smart or talented enough to make something worthwile, and it will be some time until an interesting new breakthrough occurs.
    - seanmcdirmid 12y ago
      My languages tend to be very innovative or even inventive; e.g. I recently posted this to HN: http://research.microsoft.com/en-us/people/smcdirm/managedtime.aspx http://research.microsoft.com/en-us/people/smcdirm/managedti... My problem is quite the opposite: when the languages are so different from what already exists, people (and even myself) have trouble thinking about how adoption would even occur, at least in the short term.
      - tetha 12y ago
        A lot of languages have made one major mistake: It's not easy to get a development environment going, and some even wanted me to not use vim. The first step to some kind of adoption is that I can get it running with an "aptitude install crazy-foo-language-dk", maybe after adding a repository. After that, crazy-foo or crazy-fooc should be interpreter and/or compiler, and that's that.
        12y ago
- bainsfather 12y ago
  Hi eudox, I would love to use CL for my projects - I've read (most of) Norvig's and Graham's books, written small programs etc. I really like most of the language, especially the 'break into debugger on error'. But, when I came to write CL for a project that needed gui, web scraping, using APIs etc (i.e. I wanted to connect it to the outside world) I was stumped. Unlike in e.g. python, there was no nice gui library that I could just use. Now maybe people will reply telling me that something exists, but compared to python, the difference in ecosystem is massive. (possibly some commercial CL environment has this, but I am unwilling to pay for something that is closed). I really want to use CL for more than just toy programs, but I don't/can't. So, when you are asking yourself "Is it worth it? Whose lives will it change?" - the answer is mine, but only if you think you are able to fix this. Good luck.
  - crazypyro 12y ago
    I've had good luck using Haskell for similar things that you are describing. Hackage has a good variety of packages that do similar things to beautifulsoup, numpy, pandas, etc. Generally, I've found them to be less built up, but they still provide a decent level of abstraction. Cabal hell is real though.
  - codygman 12y ago
    Regarding CL with some quick googling (searched "common lisp html parser" and "common lisp http client") I found: Html Parser: http://www.cliki.net/cl-html-parse http://www.cliki.net/cl-html-parse Http client: http://weitz.de/drakma/ http://weitz.de/drakma/ I'm not sure why you are/were stumped? I could search for qt bindings for you, but I'm sure they exist and are pretty mature.
    - bainsfather 12y ago
      Thanks for that. Maybe I will try again some time. I'm not saying it is impossible, rather that it is hard work compared to e.g. python. I recall looking through a few promising google results and finding e.g. that the library is no longer maintained, has bits missing etc. With python, when I use e.g. beautiful soup, it just works - I lose almost no time getting setup. When doing a project that needs many external libraries, that makes a big difference. e.g. the first item on http://www.cliki.net/GUI http://www.cliki.net/GUI has its last news item from 2007, and the mailing list gives 'not found'. But there is a new(er) github page with the comment "I cleaned up the library just in case I needed GUI in Lisp, but it turned out that I did not. Hence, the primary extent of my testing is running test-gtk:gtk-demo application. Bug reports and/or patches are welcome." Since the parent was talking about ecosystems etc, I thought my experiences were worth mentioning - "Python novice uses python to get stuff done" but "CL novice gets lost in old/unmaintained libraries, gives up and sticks with c++ & python". I mean no disrespect to the people who write these libraries etc., I am just describing what I found as a novice in CL.
nostrademons 12y ago
Dylan was my favorite language back in college. I remember following PG's essays to Lisp, and then Lisp to all the newer dialects like Dylan or Goo. Dylan had it all: a metaobject protocol, generic functions, optional static typing, infix macros. I even got started working on an Eclipse plugin for it, which I ended up shelving after like 3 weeks when I lost interest. Unfortunately, there are large network effects to programming languages, and the stuff that really makes you productive - libraries and tooling - Dylan just lacked. It wasn't practical to write anything larger than an ICFP contest entry in it. So I went from Dylan to Python, which lacks many of the really cool language features and is a lot slower, but at least comes with so many batteries included that you can whip up a prototype for anything really quickly.
- wisty 12y ago
  I don't really think it's purely tooling that's the issue. If it were so easy to use Dylan, then the tooling would be easy to write. I think it's performance. "Scripting languages" tear compiled languages apart when it comes to iterating. Write a "hello world" web app page in Python and Scala, then see which one takes 30 seconds and about a gig of RAM to actually display in the browser. On the other hand, if you needed performance scaled across millions of users, you needed C or C++. The big languages are C and C++ and Fortran (if you need performance), a long laundry list of "scripting" languages, and Java. Java is the only one that actually gets a spot because of its tooling, most other languages only got tooling because people loved the languages. There's a lot of compiled languages coming out (or re-emerging) these days (Scala, Haskell, Swift, Julia) with funky features that require compilation that don't get you closer to the metal. I suspect that's largely because computers now have the RAM and CPU power to actually make them fun to use.
  - joshvm 12y ago
    The combo of C(++) and Python is very strong. A friend of mine works in a research HPC lab (think around 16000 cores); they use Python for the IO and interface glue and C for the heavily parallel science code. It's the best of both worlds.
  - nostrademons 12y ago
enduser 12y ago
I appreciate this article by someone who is serious about contributing useful solutions to the world--not just the social aspect of programming--and appreciates a language that empowers him to develop those useful solutions as readily as possible. When I'm not programming I like to get some distance from my work and hang out with people who have diverse interests. When I'm serious about programming I use Common Lisp. When I'm serious about connecting with other people I use English. Many people seem to confound these pursuits and end up with languages that compromise weakly between talking to people and talking to computers. For me, programming is about solving business problems ASAP in a manner that is amenable to a long series of minor improvements over many years. Having a stable language standard with language improvements happening as add-on libraries is a huge win. My old code keeps working, so I can stay focused on improvements instead of bailing water. Also Lisp has the seemingly magical property of being one of the easiest languages to read, understand, and reason about by programmers who have the aptitude to learn it--and it scares the pants off of people who don't. With all of the "expert programmer" pretenders out there it's helpful as an employer to have something that separates the serious programmers from the pretenders.
- tempodox 12y ago
  I fully agree. Maybe it's just me, but that sounds much more hands-on to me than the typical HN comment.
loup-vaillant 12y ago
> Algebraic types? Dependent types? You'll never see them. They're too ... research-y. They stink of academe, which is: they stink of uselessness-to-industry. One may think that because closures are finally entering the mainstream (after what, 5 decades?), we have hope for those things to come as well. But then I saw Swift. Built-in support for an Option type, so one can avoid null pointer exceptions. At the same time, this languages manages to recognize the extreme usefulness of algebraic data types, without using them in their full generality. Like, why bother with a generic feature when we can settle for an ad-hoc one? I'd give much to know what went so deeply wrong in our industry that we keep making such basic mistakes.
- cwzwarich 12y ago
  > But then I saw Swift. Built-in support for an Option type, so one can avoid null pointer exceptions. At the same time, this languages manages to recognize the extreme usefulness of algebraic data types, without using them in their full generality. Like, why bother with a generic feature when we can settle for an ad-hoc one? Swift has the syntax to define arbitrary algebraic datatypes, even if it doesn't yet work in the beta versions of the compiler.
  - tel 12y ago
    It seems to be likely that 1.0 will hit before this really holds true. In Beta 6 I'm told you can easily crash the compiler with recursive generic ADTs and most have to travel "through" some heap type to compile at all. Furthermore, Swift doesn't support enough laziness/deferral/coalgebraic formulation to have, say, an infinite stream type without breaking GCD. These will probably be fixed in time, but Swift's ADT support is still pretty experimental to say the least.
    - tempodox 12y ago
      Wow, where did you get Beta 6? I only have 5.
      - 12y ago
rayiner 12y ago
Dylan is also, unfortunately, an example of "worse is better" in action. The extant Dylan implementations were incredibly ambitious. CMU's d2c built on the experience with CMUCL. Harlequin Dylan (AKA OpenDylan) was "Dylan all the way down" with a sophisticated native code compiler and an IDE written in Dylan. Multithreading, generational GC, etc. And what filled the dynamic language niche? Interpreted languages like Ruby and Python that have yet to achieve 1970's levels of implementation sophistication. But simplicity made them agile and portable and allowed resources to be spent on libraries.
- lispm 12y ago
  Dylan wasn't really a 'dynamic language' in the classical sense. It was thought to bring the power of dynamic languages to application developers. Dylan was made obsolete by Objective-C at Apple, not Ruby or Python. Outside Apple Dylan was made obsolete by Java. Some of the early Dylan users complained that too much time was wasted on the IDEs, while the language implementation and libraries were lacking.
nickbauman 12y ago
If you want to gauge a language look at one thing and one thing only. Who are these people using it and what are they using it for? The answer to that question is more important than anything else on the list after that. Amen.
- xplothar 12y ago
  Good one. Used this heuristic to select Erlang and very happy since that time :-)
AndrewKemendo 12y ago
Something I never quite understood about languages was why they are restricted to one or another domain. For example the author writes: Algebraic types? Dependent types? You'll never see them. They're too ... research-y Why can't those features be baked into C++ or Java?
- eudox 12y ago
  Well, you can add ADT's to C/C++ with a macro system: https://github.com/eudoxia0/cmacro/blob/master/t/good-macros/adt.c https://github.com/eudoxia0/cmacro/blob/master/t/good-macros...
- judk 12y ago
  Java has a fixed assembly-language level syntax. Look to Scala or F#
- peterkelly 12y ago
  I think C++ is full now.
  - Symmetry 12y ago
    People keep saying that yet the standards committee keeps nailing new tentacles on. http://c2.com/cgi/wiki?ExtraLegsOntoaDog http://c2.com/cgi/wiki?ExtraLegsOntoaDog
- theseoafs 12y ago
  Well as far as I know neither language is clever enough to allow arbitrary proofs to be expressed at compile-time. (I hear that C++'s template system is Turing-complete, which may or may not be true, but even if it is, that doesn't mean that dependent types could be expressed in a way that's easy to write/read.) So you'd have to essentially rewrite a big part of the compiler if you were to "bake dependent types" into C++ or Java, which is of course easier said than done. Not to mention that dependent types are extraordinarily complicated and have only been implemented in research systems.
BruceM 12y ago
I'm the Bruce that he mentioned in the post. For better or worse, I've been pushing Dylan forward heavily over the last few years and am effectively the primary maintainer. Over the last couple of years, we've made a lot of progress. We've completely revived the documentation from 1990s era FrameMaker files and have it published via a pretty modern system. We've converted from SVN to Git and moved to GitHub. We've done 4 actual releases. We've improved our platform portability. We've provided some basic debugging integration with LLDB. We've fixed some long standing issues in the compiler and tool chain. We've improved the GC integration on all platforms. But there's a lot to do. We need to fix our Unicode support. We need to improve the type system in at least minor ways if not major ways. We need to improve how parse failures are handled as the errors are not always friendly. We need more libraries. Some of this is really easy, some isn't. But for pretty much everything, there are bite-sized pieces of work that could be done in a couple of hours/week that would lead to significant gains. I've wanted to just flat out use Dylan for something and have built some small prototypes with it and while they've worked out well enough, the actual projects themselves didn't go anywhere (unrelated to the use of Dylan). I think this blog post was triggered by a comment that I'd made publicly yesterday that I'm feeling rather discouraged at this point. There was also a private email that I sent to 19 people who have been involved with Dylan recently, but the author of this post didn't get that email. I view Dylan, not as a language from the past, but as a stepping ladder towards building a better language for the future. We don't have to get bogged down in a lot of the minutiae involved in creating a new language as a lot of the work has been done. We get to focus on things at a different level and those things are just as important. People bring up Goo often when Dylan comes up. Goo is interesting, but the implementation is nothing close to being industrial enough to survive an encounter with the real world. I came to Dylan because I saw the mess that Scala and other languages were. I didn't like where they were going and following some people on Twitter like https://twitter.com/milessabin https://twitter.com/milessabin and others seems to show that I'm not alone. And that's why I'll probably keep at it with Dylan. I want a better future and I'm going to keep trying to build it.
- justinpombrio 12y ago
  It sounds like you care not only about Dylan but also about advancing programming language construction in general. Have you considered writing, e.g., blog posts about design decisions you make as you work on Dylan? Maybe it would help other people learn from your work.
Fice 12y ago
You could think of it as putting a low-pass filter on some of the good ideas from the ’60s and ’70s, as computing spread out much, much faster than educating unsophisticated people can happen. In the last 25 years or so, we actually got something like a pop culture ... So I think the lack of a real computer science today, and the lack of real software engineering today, is partly due to this pop culture. A Conversation with Alan Kay, ACM Queue, 2004, https://queue.acm.org/detail.cfm?id=1039523 https://queue.acm.org/detail.cfm?id=1039523
- djur 12y ago
  The history of mass computing involves numerous 'bottleneck' events where an increase in usage was bought with a curtailment of hardware and software quality. You had the first blast of cheap minis available outside military and academic environments. Then you had cheap home computers with limited BASIC implementations. That was followed by web apps scripted with half-assed '90s JavaScript and served from cramped mass hosting servers. Then you had smartphones -- iOS using a somewhat spruced-up but constrained version of the decades-old NeXTSTEP, and Android using an outdated and pared-down version of Java. This is pretty much what "Worse Is Better" is about. Cheap, readily-available software that runs on cheap, readily-available hardware is always going to have a huge head start.
  - Joeri 12y ago
    Unix is another example of a worse reimplementation of an earlier system to fit the hardware constraints of the day. The thing is though, the security model of multics would be a much better fit for today's security needs, but we don't have it because the hardware that could run it was too expensive 40 years ago, which seems crazy when I think about it. Sometimes it feels like the industry as a whole is no longer ambitious, that building fundamentally better systems is no longer considered important. It's nice that you can run unix on your phone, but i would like to run something better than unix on my desktop. Where are the OS's that are ambitious enough to eventually turn into scarlett johansson in the movie 'Her'?
cschep 12y ago
just say fuck! we're all thinking it!
georgegeorges 12y ago
If you're going to rant, you might want to actually have a clear point to make. > college kids on comp.lang.lisp asking for the answers for problem-set 3 on last night's homework Surely not during the Naggum days. CLL was a hostile wasteland. > That is the lesson of perl and python and all these other languages. They're not good for anything. They suck. And they suck in libraries and syntax and semantics and weirdness-factor and everything. What? How have you not heard of CPAN? There is not a single language in the world that can touch Perl's libraries. I'm not sure why you feel the need to toss either Perl or Python under the bus to make some petty point about Dylan's lack of popularity. Python replaced Scheme at MIT. It's time to move on. I know I have. You have to have your head pretty far up your own ass to not see how much Common Lisp sucks. It's a language designed by committee, and it looks like it. I've used Erlang too. For everything Erlang does well, there are countless areas that make you want to bang your head against the desk. Languages don't matter. Platforms matter. APIs matter. Playing nicely with the rest of the world fucking matters. Common Lisp wouldn't.
- eudox 12y ago
  > It's a language designed by committee, and it looks like it. This means nothing. When a design succeeds, we say it was designed by the community; when it fails, by committee. You can produce examples for any language: In Common Lisp the committee that designed it and the community of previous Lisp users that backed it. For Python, the community of users and Guidos' mailing list pals. There is nothing inherently superior or inferior about design by community or committee, respectively.
  - rtpg 12y ago
    I think a good reference for what people mean by "designed by committee" is to look at the reference spec for a language. I remember picking up the Common Lisp reference book in my college library. It was one of the thickest books in the entire CS section. Obviously much thicker than any books on Scheme, but even on the same level as "The C++ Programming Language". And the latter spends a lot more time explaining things than the Lisp book did. CL is a huge and complex language, and I'm still not sure why there are 80 different ways to loop. It's a confusing language, with not nearly as much orthogonality as it could have.
rdtsc 12y ago
> These languages are defined, right there in the dictionary. Erlang: see 'career wrecker.' Please. Someone, wreck my career some more. Unlike Dylan Erlang was created by a company for a purpose with very clear goals and it did and still excels at meeting those goals, and nothing out there gets close to the qualities it has. Not everyone needs those qualities, but sometimes nothing will do. Erlang is at the core of many solid industrial applications -- mobile to internet gateways, message queues, trading systems, large databases, handling millions of concurrent connection and billions of messages per day for WhatsApp. What does Dylan do? This is the second time I heard about Dylan. I've played with Mercury, Prolog, Nimrod, Curry (Haskell + Logic programming) and other rather obscure languages but haven't heard about Dylan much. Some languages just don't make it, sometimes it is just luck. However I don't like the disparaging and angry remarks thrown around at other languages and ecosystems. That does nothing to promote Dylan it only pushes people away.
- Tomte 12y ago
  > What does Dylan do? I'm totally unfair here, because I really don't know much about Dylan, but my view maybe explains a bit why Dylan does not even "win" with people like me, who really like those strange languages like Lisp or ML. To me, Dylan was never about creating something new and great. It was taking Common Lisp and "fixing" its syntax. Unfortunately, I can't stand Lisp without prefix notation and lots of parentheses. And people who hate prefix notation and lots of parentheses aren't interested too much in Common Lisp. The resulting set of people interested in Dylan was... well, I've heard of Bruce. Andreas Bogk was doing some heavy advocacy in Germany, to the point where people just didn't want to listen anymore. Let's say about ten people in the world cared. And I just don't see how that set could grow by orders of magnitude.
  - nostrademons 12y ago
    Dylan started out with a prefix syntax; the infix syntax was added on late in its development at Apple, because it was believed the market wasn't willing to adopt a prefix-syntaxed language. Turns out it didn't matter very much. Dylan also standardized and simplified a lot of the dark, twisty corners of Common Lisp. Generic functions became a core part of the language, rather than something bolted on by ad-hoc macro packages. The collections API was unified to use generic functions (so no more mapc/mapcar/mapcan/mapl/maplist/mapcon mess), and the language syntax's infix operators were all defined in terms of generic functions as well, so you could eg. define a matrix type and have + and * work on it as expected. The superclass linearization was fixed to be monotonic. Macros are hygienic. It's a Lisp-1, and everything in the language is an object bound to a name. Honestly, I think its lack of success came down to a Worse-is-Better problem. Dylan is a really nice language, designed by a committee with a combined total of over 2 centuries of Common Lisp experience. But it doesn't excel in any one domain. It's a large language, fairly complicated, which tried to mainstream a number of features with complex interactions and limited applicability. To an outsider, who doesn't understand those features or how they might be useful, it's too hard to pick up in a weekend. And so simpler languages like Scheme, Go, Python, or Erlang steal its thunder - they lack the total power of Dylan, but you can point to Scheme and say "Small composable building blocks", or Go and "Fast concurrency with quick, simple deployment", or Python and "Prototype anything, with batteries included", or Erlang and "Reliable message-passing concurrency." If you point to Dylan and try to describe it in one sentence, you get "Generic functions all the way down", which means nothing to the average programmer. It's much the same problem that Scala faces, except that Scala features JVM interop and so Scala advocates can say "Tomorrow's language features with yesterday's Java code."
yarrel 12y ago
I love Dylan. We should all be using it instead of C++ and Java. Particularly C++. Each time I type "friend" in C++ or have to use the STL, I miss Dylan.
davidw 12y ago
There are a lot of things that strike me as being wrong about this essay. Most importantly, to understand the success or failure, you have to think about it, and its economics, like any other product on the market. Granted, programming languages have different characteristics as products than lighthouses or tv shows or telephones, but they do share things with them that we can learn from. This is a somewhat dated attempt of my own to share a bit of that thinking: http://www.welton.it/articles/programming_language_economics http://www.welton.it/articles/programming_language_economics A few other things: * Like rdtsc says, Erlang does not fit the mold in a lot of ways. It had a large corporate sponsor from the get-go, which was good for it in some ways (money for developers), and perhaps bad in others: lots of production code early on means it's not possible to change stuff that is less than optimal. * As per my article, new/small/unpopular languages need a niche, a beachhead if they are to gain traction. You can't create a new language and platform from scratch with an ecosystem as big as Java's (indeed, piggybacking on the JVM is a popular strategy because of this), so you'd better have one thing where you absolutely kick ass. Erlang has this in spades, for instance. Ruby had Rails. Tcl had Tk and a few other "killer apps". PHP was way easier to get started with than mod_perl, back in the day. I don't see this for Dylan, particularly, but then I don't know much about it, so maybe it's there somewhere, and BruceM will figure it out and the language will gain a following.
jbergens 12y ago
You must find the corporate developer who cares ;-) http://steve-yegge.blogspot.se/2010/12/haskell-researchers-announce-discovery.html http://steve-yegge.blogspot.se/2010/12/haskell-researchers-a... And then maybe try to get more like him/her.
pjmlp 12y ago
Very interesting read. I have a soft spot for Dylan from magazine articles back in the Newton days. Here we have a Lisp like language, with a more approachable syntax for the average Joe/Jane developers, AOT compilation and Apple kills it. I think it is also important to bring out the paper from Erik Meijer about using Visual Basic to hook the typical enterprise developer into FP (via LINQ). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.118.2025 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.118....
ZenoArrow 12y ago
I don't know that much about Dylan, but isn't there some consensus that Julia is the "new Dylan"?
RichardFord 12y ago
I really like Dylan, but it has a huge problem that never gets discussed - verbosity. The syntax is entirely too verbose for a modern language. My suggestion would be to first complete the Intellij plugin and then actually change the syntax so it can compete with the scripting languages.

172 points by kryptiskt 12y ago

eudox 12y ago

The article hits rather close to home, as someone who, for better or worse, is committed to improving Common Lisp's ecosystem. I often feel this existential dread of "Is it worth it? Whose lives will it change? Will I spend years labouring in obscurity for nothing?". And while the answer to all those questions is probably not what I'd like to hear, I still do it. This is why ecosystems are, for most people, more valuable than intrinsic language features: Tribalism along the lines of "We have X thousand people backing us up"/"X thousand devs can't go wrong". People don't care about monads or macros, they care about feeling like they're part of a large community.

pnathan 12y ago
As someone also invested in Common Lisp, it.... hit really close to home.
seanmcdirmid 12y ago
Its even worse for the PL designer inventing something new without even a small ecosystem in place; I ask myself everyday "is it worth it?"
- freyrs3 12y ago
  Have to figure it takes 10-20 years to grow an industrial strength language. It takes a lot of faith in an idea or stubbornness ( probably both ) to go through that struggle. I admire people with that kind of dedication and I wish exploring the language design space wasn't such an enormous investment.
- sitkack 12y ago
  It is worth it. And your case it isn't the language per se, but spreading the ideas. The language is an ephemeral vehicle.
- tetha 12y ago
  The important question isn't necessarily: Is it worth it, without the ecosystem? The more interesting questions is: What does this language add? For example, I need to deal with some unix process management tasks. I'm not crazy enough to do them in straight C, so I'm using python for it's strong C bindings, so I can use system calls in C with python level control flow. However, there's a lot to be desired, because dealing with all this concurrency and inherent raciness of the system in a language like python is a royal pain in the rearside. Similar things with java and performant programming. Some of my major programs at work don't use the java ecosystem outside of log4j, junit, apache commons lang and apache commons collections. They use java, because java is darned fast without being as brittle as C or as nuts as C++. So java adds a lot of really cheap development security. Designing yet another language that puts the existing features together with just a tiny difference doesn't add anything. I've enjoyed looking at experimental languages, and I've enjoyed tinkering around with ideas myself some time ago. But by now, I've accepted I'm not smart or talented enough to make something worthwile, and it will be some time until an interesting new breakthrough occurs.
  - seanmcdirmid 12y ago
    My languages tend to be very innovative or even inventive; e.g. I recently posted this to HN: http://research.microsoft.com/en-us/people/smcdirm/managedtime.aspx http://research.microsoft.com/en-us/people/smcdirm/managedti... My problem is quite the opposite: when the languages are so different from what already exists, people (and even myself) have trouble thinking about how adoption would even occur, at least in the short term.
    - tetha 12y ago
      A lot of languages have made one major mistake: It's not easy to get a development environment going, and some even wanted me to not use vim. The first step to some kind of adoption is that I can get it running with an "aptitude install crazy-foo-language-dk", maybe after adding a repository. After that, crazy-foo or crazy-fooc should be interpreter and/or compiler, and that's that.
      - 12y ago
bainsfather 12y ago
Hi eudox, I would love to use CL for my projects - I've read (most of) Norvig's and Graham's books, written small programs etc. I really like most of the language, especially the 'break into debugger on error'. But, when I came to write CL for a project that needed gui, web scraping, using APIs etc (i.e. I wanted to connect it to the outside world) I was stumped. Unlike in e.g. python, there was no nice gui library that I could just use. Now maybe people will reply telling me that something exists, but compared to python, the difference in ecosystem is massive. (possibly some commercial CL environment has this, but I am unwilling to pay for something that is closed). I really want to use CL for more than just toy programs, but I don't/can't. So, when you are asking yourself "Is it worth it? Whose lives will it change?" - the answer is mine, but only if you think you are able to fix this. Good luck.
- crazypyro 12y ago
  I've had good luck using Haskell for similar things that you are describing. Hackage has a good variety of packages that do similar things to beautifulsoup, numpy, pandas, etc. Generally, I've found them to be less built up, but they still provide a decent level of abstraction. Cabal hell is real though.
- codygman 12y ago
  Regarding CL with some quick googling (searched "common lisp html parser" and "common lisp http client") I found: Html Parser: http://www.cliki.net/cl-html-parse http://www.cliki.net/cl-html-parse Http client: http://weitz.de/drakma/ http://weitz.de/drakma/ I'm not sure why you are/were stumped? I could search for qt bindings for you, but I'm sure they exist and are pretty mature.
  - bainsfather 12y ago
    Thanks for that. Maybe I will try again some time. I'm not saying it is impossible, rather that it is hard work compared to e.g. python. I recall looking through a few promising google results and finding e.g. that the library is no longer maintained, has bits missing etc. With python, when I use e.g. beautiful soup, it just works - I lose almost no time getting setup. When doing a project that needs many external libraries, that makes a big difference. e.g. the first item on http://www.cliki.net/GUI http://www.cliki.net/GUI has its last news item from 2007, and the mailing list gives 'not found'. But there is a new(er) github page with the comment "I cleaned up the library just in case I needed GUI in Lisp, but it turned out that I did not. Hence, the primary extent of my testing is running test-gtk:gtk-demo application. Bug reports and/or patches are welcome." Since the parent was talking about ecosystems etc, I thought my experiences were worth mentioning - "Python novice uses python to get stuff done" but "CL novice gets lost in old/unmaintained libraries, gives up and sticks with c++ & python". I mean no disrespect to the people who write these libraries etc., I am just describing what I found as a novice in CL.

nostrademons 12y ago

Dylan was my favorite language back in college. I remember following PG's essays to Lisp, and then Lisp to all the newer dialects like Dylan or Goo. Dylan had it all: a metaobject protocol, generic functions, optional static typing, infix macros. I even got started working on an Eclipse plugin for it, which I ended up shelving after like 3 weeks when I lost interest. Unfortunately, there are large network effects to programming languages, and the stuff that really makes you productive - libraries and tooling - Dylan just lacked. It wasn't practical to write anything larger than an ICFP contest entry in it. So I went from Dylan to Python, which lacks many of the really cool language features and is a lot slower, but at least comes with so many batteries included that you can whip up a prototype for anything really quickly.

wisty 12y ago
I don't really think it's purely tooling that's the issue. If it were so easy to use Dylan, then the tooling would be easy to write. I think it's performance. "Scripting languages" tear compiled languages apart when it comes to iterating. Write a "hello world" web app page in Python and Scala, then see which one takes 30 seconds and about a gig of RAM to actually display in the browser. On the other hand, if you needed performance scaled across millions of users, you needed C or C++. The big languages are C and C++ and Fortran (if you need performance), a long laundry list of "scripting" languages, and Java. Java is the only one that actually gets a spot because of its tooling, most other languages only got tooling because people loved the languages. There's a lot of compiled languages coming out (or re-emerging) these days (Scala, Haskell, Swift, Julia) with funky features that require compilation that don't get you closer to the metal. I suspect that's largely because computers now have the RAM and CPU power to actually make them fun to use.
- joshvm 12y ago
  The combo of C(++) and Python is very strong. A friend of mine works in a research HPC lab (think around 16000 cores); they use Python for the IO and interface glue and C for the heavily parallel science code. It's the best of both worlds.
- nostrademons 12y ago

enduser 12y ago

I appreciate this article by someone who is serious about contributing useful solutions to the world--not just the social aspect of programming--and appreciates a language that empowers him to develop those useful solutions as readily as possible. When I'm not programming I like to get some distance from my work and hang out with people who have diverse interests. When I'm serious about programming I use Common Lisp. When I'm serious about connecting with other people I use English. Many people seem to confound these pursuits and end up with languages that compromise weakly between talking to people and talking to computers. For me, programming is about solving business problems ASAP in a manner that is amenable to a long series of minor improvements over many years. Having a stable language standard with language improvements happening as add-on libraries is a huge win. My old code keeps working, so I can stay focused on improvements instead of bailing water. Also Lisp has the seemingly magical property of being one of the easiest languages to read, understand, and reason about by programmers who have the aptitude to learn it--and it scares the pants off of people who don't. With all of the "expert programmer" pretenders out there it's helpful as an employer to have something that separates the serious programmers from the pretenders.

tempodox 12y ago
I fully agree. Maybe it's just me, but that sounds much more hands-on to me than the typical HN comment.

loup-vaillant 12y ago

> Algebraic types? Dependent types? You'll never see them. They're too ... research-y. They stink of academe, which is: they stink of uselessness-to-industry. One may think that because closures are finally entering the mainstream (after what, 5 decades?), we have hope for those things to come as well. But then I saw Swift. Built-in support for an Option type, so one can avoid null pointer exceptions. At the same time, this languages manages to recognize the extreme usefulness of algebraic data types, without using them in their full generality. Like, why bother with a generic feature when we can settle for an ad-hoc one? I'd give much to know what went so deeply wrong in our industry that we keep making such basic mistakes.

cwzwarich 12y ago
> But then I saw Swift. Built-in support for an Option type, so one can avoid null pointer exceptions. At the same time, this languages manages to recognize the extreme usefulness of algebraic data types, without using them in their full generality. Like, why bother with a generic feature when we can settle for an ad-hoc one? Swift has the syntax to define arbitrary algebraic datatypes, even if it doesn't yet work in the beta versions of the compiler.
- tel 12y ago
  It seems to be likely that 1.0 will hit before this really holds true. In Beta 6 I'm told you can easily crash the compiler with recursive generic ADTs and most have to travel "through" some heap type to compile at all. Furthermore, Swift doesn't support enough laziness/deferral/coalgebraic formulation to have, say, an infinite stream type without breaking GCD. These will probably be fixed in time, but Swift's ADT support is still pretty experimental to say the least.
  - tempodox 12y ago
    Wow, where did you get Beta 6? I only have 5.
    - 12y ago

rayiner 12y ago

Dylan is also, unfortunately, an example of "worse is better" in action. The extant Dylan implementations were incredibly ambitious. CMU's d2c built on the experience with CMUCL. Harlequin Dylan (AKA OpenDylan) was "Dylan all the way down" with a sophisticated native code compiler and an IDE written in Dylan. Multithreading, generational GC, etc. And what filled the dynamic language niche? Interpreted languages like Ruby and Python that have yet to achieve 1970's levels of implementation sophistication. But simplicity made them agile and portable and allowed resources to be spent on libraries.

lispm 12y ago
Dylan wasn't really a 'dynamic language' in the classical sense. It was thought to bring the power of dynamic languages to application developers. Dylan was made obsolete by Objective-C at Apple, not Ruby or Python. Outside Apple Dylan was made obsolete by Java. Some of the early Dylan users complained that too much time was wasted on the IDEs, while the language implementation and libraries were lacking.

nickbauman 12y ago

If you want to gauge a language look at one thing and one thing only. Who are these people using it and what are they using it for? The answer to that question is more important than anything else on the list after that. Amen.

xplothar 12y ago
Good one. Used this heuristic to select Erlang and very happy since that time :-)

AndrewKemendo 12y ago

Something I never quite understood about languages was why they are restricted to one or another domain. For example the author writes: Algebraic types? Dependent types? You'll never see them. They're too ... research-y Why can't those features be baked into C++ or Java?

eudox 12y ago
Well, you can add ADT's to C/C++ with a macro system: https://github.com/eudoxia0/cmacro/blob/master/t/good-macros/adt.c https://github.com/eudoxia0/cmacro/blob/master/t/good-macros...
judk 12y ago
Java has a fixed assembly-language level syntax. Look to Scala or F#
peterkelly 12y ago
I think C++ is full now.
- Symmetry 12y ago
  People keep saying that yet the standards committee keeps nailing new tentacles on. http://c2.com/cgi/wiki?ExtraLegsOntoaDog http://c2.com/cgi/wiki?ExtraLegsOntoaDog
theseoafs 12y ago
Well as far as I know neither language is clever enough to allow arbitrary proofs to be expressed at compile-time. (I hear that C++'s template system is Turing-complete, which may or may not be true, but even if it is, that doesn't mean that dependent types could be expressed in a way that's easy to write/read.) So you'd have to essentially rewrite a big part of the compiler if you were to "bake dependent types" into C++ or Java, which is of course easier said than done. Not to mention that dependent types are extraordinarily complicated and have only been implemented in research systems.

BruceM 12y ago

I'm the Bruce that he mentioned in the post. For better or worse, I've been pushing Dylan forward heavily over the last few years and am effectively the primary maintainer. Over the last couple of years, we've made a lot of progress. We've completely revived the documentation from 1990s era FrameMaker files and have it published via a pretty modern system. We've converted from SVN to Git and moved to GitHub. We've done 4 actual releases. We've improved our platform portability. We've provided some basic debugging integration with LLDB. We've fixed some long standing issues in the compiler and tool chain. We've improved the GC integration on all platforms. But there's a lot to do. We need to fix our Unicode support. We need to improve the type system in at least minor ways if not major ways. We need to improve how parse failures are handled as the errors are not always friendly. We need more libraries. Some of this is really easy, some isn't. But for pretty much everything, there are bite-sized pieces of work that could be done in a couple of hours/week that would lead to significant gains. I've wanted to just flat out use Dylan for something and have built some small prototypes with it and while they've worked out well enough, the actual projects themselves didn't go anywhere (unrelated to the use of Dylan). I think this blog post was triggered by a comment that I'd made publicly yesterday that I'm feeling rather discouraged at this point. There was also a private email that I sent to 19 people who have been involved with Dylan recently, but the author of this post didn't get that email. I view Dylan, not as a language from the past, but as a stepping ladder towards building a better language for the future. We don't have to get bogged down in a lot of the minutiae involved in creating a new language as a lot of the work has been done. We get to focus on things at a different level and those things are just as important. People bring up Goo often when Dylan comes up. Goo is interesting, but the implementation is nothing close to being industrial enough to survive an encounter with the real world. I came to Dylan because I saw the mess that Scala and other languages were. I didn't like where they were going and following some people on Twitter like https://twitter.com/milessabin https://twitter.com/milessabin and others seems to show that I'm not alone. And that's why I'll probably keep at it with Dylan. I want a better future and I'm going to keep trying to build it.

justinpombrio 12y ago
It sounds like you care not only about Dylan but also about advancing programming language construction in general. Have you considered writing, e.g., blog posts about design decisions you make as you work on Dylan? Maybe it would help other people learn from your work.

Fice 12y ago

You could think of it as putting a low-pass filter on some of the good ideas from the ’60s and ’70s, as computing spread out much, much faster than educating unsophisticated people can happen. In the last 25 years or so, we actually got something like a pop culture ... So I think the lack of a real computer science today, and the lack of real software engineering today, is partly due to this pop culture. A Conversation with Alan Kay, ACM Queue, 2004, https://queue.acm.org/detail.cfm?id=1039523 https://queue.acm.org/detail.cfm?id=1039523

djur 12y ago
The history of mass computing involves numerous 'bottleneck' events where an increase in usage was bought with a curtailment of hardware and software quality. You had the first blast of cheap minis available outside military and academic environments. Then you had cheap home computers with limited BASIC implementations. That was followed by web apps scripted with half-assed '90s JavaScript and served from cramped mass hosting servers. Then you had smartphones -- iOS using a somewhat spruced-up but constrained version of the decades-old NeXTSTEP, and Android using an outdated and pared-down version of Java. This is pretty much what "Worse Is Better" is about. Cheap, readily-available software that runs on cheap, readily-available hardware is always going to have a huge head start.
- Joeri 12y ago
  Unix is another example of a worse reimplementation of an earlier system to fit the hardware constraints of the day. The thing is though, the security model of multics would be a much better fit for today's security needs, but we don't have it because the hardware that could run it was too expensive 40 years ago, which seems crazy when I think about it. Sometimes it feels like the industry as a whole is no longer ambitious, that building fundamentally better systems is no longer considered important. It's nice that you can run unix on your phone, but i would like to run something better than unix on my desktop. Where are the OS's that are ambitious enough to eventually turn into scarlett johansson in the movie 'Her'?

cschep 12y ago

just say fuck! we're all thinking it!

georgegeorges 12y ago

If you're going to rant, you might want to actually have a clear point to make. > college kids on comp.lang.lisp asking for the answers for problem-set 3 on last night's homework Surely not during the Naggum days. CLL was a hostile wasteland. > That is the lesson of perl and python and all these other languages. They're not good for anything. They suck. And they suck in libraries and syntax and semantics and weirdness-factor and everything. What? How have you not heard of CPAN? There is not a single language in the world that can touch Perl's libraries. I'm not sure why you feel the need to toss either Perl or Python under the bus to make some petty point about Dylan's lack of popularity. Python replaced Scheme at MIT. It's time to move on. I know I have. You have to have your head pretty far up your own ass to not see how much Common Lisp sucks. It's a language designed by committee, and it looks like it. I've used Erlang too. For everything Erlang does well, there are countless areas that make you want to bang your head against the desk. Languages don't matter. Platforms matter. APIs matter. Playing nicely with the rest of the world fucking matters. Common Lisp wouldn't.

eudox 12y ago
> It's a language designed by committee, and it looks like it. This means nothing. When a design succeeds, we say it was designed by the community; when it fails, by committee. You can produce examples for any language: In Common Lisp the committee that designed it and the community of previous Lisp users that backed it. For Python, the community of users and Guidos' mailing list pals. There is nothing inherently superior or inferior about design by community or committee, respectively.
- rtpg 12y ago
  I think a good reference for what people mean by "designed by committee" is to look at the reference spec for a language. I remember picking up the Common Lisp reference book in my college library. It was one of the thickest books in the entire CS section. Obviously much thicker than any books on Scheme, but even on the same level as "The C++ Programming Language". And the latter spends a lot more time explaining things than the Lisp book did. CL is a huge and complex language, and I'm still not sure why there are 80 different ways to loop. It's a confusing language, with not nearly as much orthogonality as it could have.

rdtsc 12y ago

> These languages are defined, right there in the dictionary. Erlang: see 'career wrecker.' Please. Someone, wreck my career some more. Unlike Dylan Erlang was created by a company for a purpose with very clear goals and it did and still excels at meeting those goals, and nothing out there gets close to the qualities it has. Not everyone needs those qualities, but sometimes nothing will do. Erlang is at the core of many solid industrial applications -- mobile to internet gateways, message queues, trading systems, large databases, handling millions of concurrent connection and billions of messages per day for WhatsApp. What does Dylan do? This is the second time I heard about Dylan. I've played with Mercury, Prolog, Nimrod, Curry (Haskell + Logic programming) and other rather obscure languages but haven't heard about Dylan much. Some languages just don't make it, sometimes it is just luck. However I don't like the disparaging and angry remarks thrown around at other languages and ecosystems. That does nothing to promote Dylan it only pushes people away.

Tomte 12y ago
> What does Dylan do? I'm totally unfair here, because I really don't know much about Dylan, but my view maybe explains a bit why Dylan does not even "win" with people like me, who really like those strange languages like Lisp or ML. To me, Dylan was never about creating something new and great. It was taking Common Lisp and "fixing" its syntax. Unfortunately, I can't stand Lisp without prefix notation and lots of parentheses. And people who hate prefix notation and lots of parentheses aren't interested too much in Common Lisp. The resulting set of people interested in Dylan was... well, I've heard of Bruce. Andreas Bogk was doing some heavy advocacy in Germany, to the point where people just didn't want to listen anymore. Let's say about ten people in the world cared. And I just don't see how that set could grow by orders of magnitude.
- nostrademons 12y ago
  Dylan started out with a prefix syntax; the infix syntax was added on late in its development at Apple, because it was believed the market wasn't willing to adopt a prefix-syntaxed language. Turns out it didn't matter very much. Dylan also standardized and simplified a lot of the dark, twisty corners of Common Lisp. Generic functions became a core part of the language, rather than something bolted on by ad-hoc macro packages. The collections API was unified to use generic functions (so no more mapc/mapcar/mapcan/mapl/maplist/mapcon mess), and the language syntax's infix operators were all defined in terms of generic functions as well, so you could eg. define a matrix type and have + and * work on it as expected. The superclass linearization was fixed to be monotonic. Macros are hygienic. It's a Lisp-1, and everything in the language is an object bound to a name. Honestly, I think its lack of success came down to a Worse-is-Better problem. Dylan is a really nice language, designed by a committee with a combined total of over 2 centuries of Common Lisp experience. But it doesn't excel in any one domain. It's a large language, fairly complicated, which tried to mainstream a number of features with complex interactions and limited applicability. To an outsider, who doesn't understand those features or how they might be useful, it's too hard to pick up in a weekend. And so simpler languages like Scheme, Go, Python, or Erlang steal its thunder - they lack the total power of Dylan, but you can point to Scheme and say "Small composable building blocks", or Go and "Fast concurrency with quick, simple deployment", or Python and "Prototype anything, with batteries included", or Erlang and "Reliable message-passing concurrency." If you point to Dylan and try to describe it in one sentence, you get "Generic functions all the way down", which means nothing to the average programmer. It's much the same problem that Scala faces, except that Scala features JVM interop and so Scala advocates can say "Tomorrow's language features with yesterday's Java code."

yarrel 12y ago

I love Dylan. We should all be using it instead of C++ and Java. Particularly C++. Each time I type "friend" in C++ or have to use the STL, I miss Dylan.

davidw 12y ago

There are a lot of things that strike me as being wrong about this essay. Most importantly, to understand the success or failure, you have to think about it, and its economics, like any other product on the market. Granted, programming languages have different characteristics as products than lighthouses or tv shows or telephones, but they do share things with them that we can learn from. This is a somewhat dated attempt of my own to share a bit of that thinking: http://www.welton.it/articles/programming_language_economics http://www.welton.it/articles/programming_language_economics A few other things: * Like rdtsc says, Erlang does not fit the mold in a lot of ways. It had a large corporate sponsor from the get-go, which was good for it in some ways (money for developers), and perhaps bad in others: lots of production code early on means it's not possible to change stuff that is less than optimal. * As per my article, new/small/unpopular languages need a niche, a beachhead if they are to gain traction. You can't create a new language and platform from scratch with an ecosystem as big as Java's (indeed, piggybacking on the JVM is a popular strategy because of this), so you'd better have one thing where you absolutely kick ass. Erlang has this in spades, for instance. Ruby had Rails. Tcl had Tk and a few other "killer apps". PHP was way easier to get started with than mod_perl, back in the day. I don't see this for Dylan, particularly, but then I don't know much about it, so maybe it's there somewhere, and BruceM will figure it out and the language will gain a following.

jbergens 12y ago

You must find the corporate developer who cares ;-) http://steve-yegge.blogspot.se/2010/12/haskell-researchers-announce-discovery.html http://steve-yegge.blogspot.se/2010/12/haskell-researchers-a... And then maybe try to get more like him/her.

pjmlp 12y ago

Very interesting read. I have a soft spot for Dylan from magazine articles back in the Newton days. Here we have a Lisp like language, with a more approachable syntax for the average Joe/Jane developers, AOT compilation and Apple kills it. I think it is also important to bring out the paper from Erik Meijer about using Visual Basic to hook the typical enterprise developer into FP (via LINQ). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.118.2025 http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.118....

ZenoArrow 12y ago

I don't know that much about Dylan, but isn't there some consensus that Julia is the "new Dylan"?

RichardFord 12y ago

I really like Dylan, but it has a huge problem that never gets discussed - verbosity. The syntax is entirely too verbose for a modern language. My suggestion would be to first complete the Intellij plugin and then actually change the syntax so it can compete with the scripting languages.