12 ms·

Programming Languages Are Simply Not Powerful Enough

147 points by dylangs1030 14y ago

moron4hire 14y ago
So this essentially sounds like "You gotta work at it to get it done."
treerex 14y ago
This is where Common Lisp comes in: through macros you can expand the language to include the abstractions your application needs. PG talks about this extensively. The issue is that doing this requires a deep understanding of the language, and many programmers are not able to grasp what the CL toolbox gives them.
cageface 14y ago
The claim is often made that languages like Forth and Scheme are better for building applications than less elegant and orthogonal languages. But the evidence on the ground is overwhelmingly to the contrary. The vast majority of commercially significant apps are built in relatively baroque and ugly languages like Java and C++. Personally I think that the ability to define your own language constructs is ultimately a lot less useful than things like rich libraries and robust tooling and I'm still waiting to see real-world proof of the advantages of programmable programming languages.
- BruceIV 14y ago
  Yes, this. You have to learn all the abstractions in a language, and the advantage of "batteries included" languages are that the abstractions are (largely) the same in everyone's code, aiding maintainability and re-use. On the other hand, if you build everything yourself, it might be better optimized to your use case, but you'll be the only person who's easily able to use the abstractions.
- tikhonj 14y ago
  This argument comes up every time somebody suggest an unpopular tool, and I think it's inherently flawed. The core problem is that popularity does not imply--and does not even necessarily correlate with--quality! Just because something is in some real sense better does not mean people will adopt it. Most people are not willing to learn something fundamentally new. They might be content going from something they know to something very similar, say Java to Python, but they will resist anything truly novel and different. And the others? They're mostly the ones already advocating Haskell or Lisp or Forth or what have you! Also, people have some really odd reservations when switching to a new technology. They are often not willing to take any visible steps back: the new might be better in a whole bunch of non-trivial ways, but if it's obviously worse on some easily identifiable property, people will avoid it. A new language might have a long list of advantages, but if it has an inefficient default string type, or slightly wonky syntax, or a sub-optimal packaging tool or any other superficial but obvious shortcoming, people aren't willing to make the switch. Another problem is that there are different kinds of productivity. There are "dense" productivity changes: if you have to write your own JSON library or deal with a broken build system, you'll be spending a contiguous amount of time on it. You'll have to devote maybe a whole day or even a week to getting around this problem. There is no way to miss this. On the other hand, if a language improves productivity in a "sparse" sort of way--say you spend 20% less time writing code and 30% less on debugging and maintenance--you won't notice quite as easily. And yet, over any reasonable time using the language, you'll come out far ahead even if you have to sink in days working around problems and libraries. A particular--and particularly important--example of this is in learning. As I said above, one of the main reasons people resist new technology is that they don't want to learn. They're too busy to spend a whole week or even month picking up something new. And yet, learning time is essentially a constant expense. Productivity gains, on the other hand, are at least linear in how much you program. If a language makes your code simpler, the gains can even be super-linear (that is, you get more benefits as you write bigger programs). These will dominate any learning time as soon as you actually start using the new technology widely. And yet, since the amount of time spent learning is obvious and the ambient productivity gains aren't, people put a larger than warranted cost on the former. Coincidentally, this does not only apply to programming languages. I've seen exactly the same sort of behavior in adopting any kind of new, non-trivial technology: Emacs, Vim, Git, Linux...etc. In short, don't trust popularity. This probably makes me sound elitist (and, to be fair, I probably am), but it's just like music or literature: the popular stuff is usually not particularly good and the good stuff is usually not particularly popular.
  - stuffihavemade 14y ago
    The difference between the popularity of Emacs, Vim, Git, Linux, and a programming language is that it doesn't matter if anyone else on the planet uses Vim Git, etc., they will work the same to me. But the difference in productivity from pre-made libraries for using Ruby (52,220 gems cut since July 2009) vs. scheme (http://planet.racket-lang.org/ http://planet.racket-lang.org/) is vast, especially when you consider that the power of the two languages is not that great.
    - irahul 14y ago
      > it doesn't matter if anyone else on the planet uses Vim Git, etc How does that work? Unless you are creating and supporting Git, git isn't going to work if you are the only person using it. Even then, there will be no github. And I don't know about you, but I won't use Vim if I lose my plugins(directly correlated to how many people are using vim)
      - stuffihavemade 14y ago
        Why wouldn't git work? The vast majority of projects I use it for are ones I work on by myself. As far as vim, the only plugins I use are syntax ones and command T, which I could live without.
  - stormbrew 14y ago
    While I'll agree that popularity is not everything, and that often people are stuck with overall subpar tools because those tools do one thing particularly well (php and application deployment comes immediately to mind), I don't think you can reasonably argue that programmers as a whole are so mired in institutional inertia that they can't move to things that are better for them within a relatively fast timeframe. The rapid adoptions of, for example: C, C++, Perl, Java, Python, and Ruby make it painfully obvious that that's not the case.
  - wvenable 14y ago
    Sorry but this argument comes up somebody suggests that unpopular tool is merely unpopular and not flawed. Lisp is one of the oldest programming languages in existence, it's still taught in Universities and has been for over 40 years. Age and exposure and still not popular. I'm not sure why you keep mentioning new technologies when the article is about old technologies and your examples (Emacs, Vim, Linux) are old technologies! The only example of a new technology, git, has seen an absolute rapid rise and completely changed the version control landscape in just a few years. A new language is not going to have a long list of advantages -- it's going to a have different list of trade-offs. Because any language feature that is objectively good with no downside or trade-off has already been implemented in some popular language somewhere. And if you can prove that you're doing something really different and really better but there's a strong learning curve people will learn it. Git is the perfect example.
- rayiner 14y ago
  The popularity of programming languages is entirely a matter of historical accident. C flourished while Pascal languished because UNIX was written in C. Was there really a deep technical reason for preferring C to Pascal? No, not really. Indeed, for most of early history, Pascal had far superior tooling to C. How else do you explain the popularity of languages like Javascript and Objective-C? Javascript is an objectively terrible language, hacked together by Brendan Eich in an afternoon. Yet, it's a tremendously popular language, and tremendous amounts of money and time have gone into building good tooling for it. Same thing with Objective-C. It's a shitty, ad-hoc, cobbled-together patchwork of C and Smalltalk, yet it's tremendously popular now because that's what the iPad is programmed in.
  - krichman 14y ago
    I upvoted you for talking about historical accident, but regretted it when you started in on Javascript and Objective-C. Yes they have some questionable decisions but they are by no means objectively awful.
    - rayiner 14y ago
      Given that Javascript was designed in 1994, not 1974, I think it rises past "questionable design decision" not to just use Scheme's lexical scoping rules (i.e. the only sensible lexical scoping rules). Although to be fair, scripting language designers keep getting this wrong for some reason.
      - krichman 14y ago
        Yeah that's a problem. I thought you could get proper scoping by writing "var" in front of everything?
        mercurial 14y ago
        That's function scope, not block scope. You can use the "let" keyword to get block scope since Javascript 1.7.
- chipsy 14y ago
  I think the problem is really that nobody is being taught in school to constantly redefine their language to create an ideal representation of the problem, when that's what "ideal" code tends to involve. The comfort zone is what you know, and if what you know is Java... However, I also think that we are making progress to go in the abstract direction, by dint of increasingly having polyglot systems and DSLs that are part of the standard libraries(e.g. regex and SQL access are expected everywhere now). If this trend accelerates we could have a hockey stick effect and before we know it, find ourselves programming almost entirely with tiny languages.
- nnq 14y ago
  And why do the rich libraries and tooling end up being built for languages like Java or C++? Because they require collaboration, and these languages, by handicapping the creation of DSLs actually make collaboration easier, and, more importantly, they make the collaboration between people of different IQs, backgrounds and expertise levels easier. ...sometimes handicapping the smartest person, more or less on purpose, makes him better at working in a team (even a "virtual" team like the one encompassing library creators and library users). It's ugly but it works! Of course, the ideal would be to have a team of very smart people - but when you can't have that , the next best alternative is to use a language that artificially "dumbs down" the smartest guys to a level similar enough to that of the others to allow collaboration... sad, I know :|
- npsimons 14y ago
  The vast majority of commercially significant apps are built in relatively baroque and ugly languages like Java and C++. I'm fairly certain there is vastly more code written in either FORTRAN or COBOL than Java and C++ combined. So I guess that makes FORTRAN and COBOL better than Java and C++ put together . . . ?
  - TheLarch 14y ago
    That's an interesting assertion. I can't readily find estimates about the number of lines of code in different languages. Nor could I find an estimate as to the number of programmers in the world over time, from which I hoped to make an extrapolation. Somebody could write and interesting report about this. Maybe my research wasn't deep enough. Anyway, how do you figure there's more FORTRAN and COBOL?
  - cageface 14y ago
    No. It means that language choice is less important than we like to think it is.
BruceIV 14y ago
Agreed, to a point. What kind of abstractions are sufficient, or useful? What happens when they leak? What happens when you build the wrong abstraction, and have to go back and build something slightly different from the ground up? What about when your abstraction works, but is so far removed from the actual execution environment that it's so slow as to be unusable? Yes, abstraction is a wonderful, useful thing that lets us think and do all sorts of wonderful, useful things - our high level programming languages and networking protocols are abstractions over lower level programming languages and networking protocols (repeat a few times) which are abstractions over the physical hardware they run on, and that's great. But abstractions are never perfect, and what you don't know about the lower levels of the stack will bite you, which is a good argument for using as few and as thin abstractions as you can to get your thing done (reasonably ... I'm not about to write this comment in binary ASCII, say).
- nsmartt 14y ago
  He mentions the point when the abstractions in the language are no longer sufficient. It seemed to me that this was more an argument in favor of using languages that are easily extended rather than languages full of abstraction with few options for extension. If you use a language like the latter, you'll still eventually reach a point where the abstractions aren't sufficient, but if you use a language like the former, though you'll reach that point sooner, you'll be better equipped to handle it. edit: word use
deleted 14y ago
[deleted]
Guthur 14y ago
For me a language is getting close to "powerful enough" when it does not get in the way of you as the programmer when you are developing abstractions of the form that seems most natural to you. This can cause problems in some cases; when the language is stepping that far back to allow you the freedom then it is up to your to enforce your own constraints to avoid your abstractions becoming...too abstract (maybe). Also in my opinion there is not one abstraction form that will appeal to every type of programmer. For me I like the freedom to manipulate and define symbolic forms with Common Lisp for others it might be the expressive type system of the Haskell. I don't think you can truly say one way is better than the other. I think those languages and a few others are giving the tools of abstraction that will not get in the way of you as the programmer.
- SomeCallMeTim 14y ago
  >enforce your own constraints to avoid your abstractions becoming...too abstract (maybe). Too abstract isn't necessarily the problem. When I've stepped too far away from standard OO structure, the problem typically becomes organization. In OO you know where just about everything SHOULD be, at least. If you're writing your own rules, it's harder to be certain where new variables get set, or what members you should find in what structures. It takes a certain level of discipline to not shoot yourself in the foot. Or at least to clean things up before they get too messy.
  - Guthur 14y ago
    Depending on the implementation of the OO system in question it could provide you with relatively limited options for abstraction. I can only really speak from a Common Lisp point of view and the problem there that I was alluding to is the fact that you are so free to produce your own forms of abstractions whether it be mini DSL macros, dynamic dispatch method combinations, data driven programming using first class functions etc. that if you are not careful you can easily obfuscate the real intention rather than making it clearer.
    - SomeCallMeTim 14y ago
DeepDuh 14y ago
Am I the only one being put off by the CSS? I had to crank the display brightness to the max in order to read it without strain. Edit: I agree a lot with the content though - For many complex problems, especially the kind where you can't just throw some standard software solution at it, it would be helpful to first develop a suitable DSL. From personal experience I'd say spend one third of your time budget into developing your domain specific toolset - even if your managers get nervous. If it works you will finish sooner than you expected and you have developed reusable tools and skills, if it doesn't you can still throw in a crunch and get it done. For me it has always worked though.
- aroman 14y ago
  Perhaps it is because I am in a dark room reading in bed with a MacBook Air set at 50% brightness, but I actually really liked the experience of reading that. Though I can absolutely understand where someone might not.
  - DeepDuh 14y ago
    I'm in a bright[1] office with an rMBP. I'm imagining everyone with displays that give less brightness / contrast is going to have a bad time. [1] It's a Japanese office. These people really like their cold bright neon. I think it has to with dark east asian eyes.
  - Kluny 14y ago
    I liked it too, but I almost went blind when I came back to HN.
  - cpeterso 14y ago
    You might like Flux. It adjusts your display's color temperature depending on the time of day. No more blinding screens when you open your MacBook at night. :) http://stereopsis.com/flux/ http://stereopsis.com/flux/
    - errnoh 14y ago
      Using flux as well but that site was one of the most enjoyable sites to read that I've seen in couple years. The colors aren't that beautiful but it's much less stressful for my eyes, and the highlight colors were spot on as well. Thanks to emidln for informing that it's Solarized Dark, I've used the default Solarized but I've skipped the Dark one because of somewhat ugly colors.
andolanra 14y ago
As a counterpoint to this article, I'd point to Yossi Kreinin's excellent "My History with Forth & Stack Machines" [1] which has been posted on HN before [2] and which discusses more extensively the particulars of Forth as a powerful toolbox of primitive operations, as well as the use of Forth in actual applications. [1]: http://www.yosefk.com/blog/my-history-with-forth-stack-machines.html http://www.yosefk.com/blog/my-history-with-forth-stack-machi... [2]: http://news.ycombinator.com/item?id=1680149 http://news.ycombinator.com/item?id=1680149
tikhonj 14y ago
I think this roughly reflects my philosophy in languages as well. You basically have two options: you can bend your problem to your language or your language to your problem. The former is epitomized by Java and Python--all Java and Python code looks the same, regardless of what it's actually doing. The latter is the domain of Lisp and Haskell and Forth and so on. I'm unabashedly in favor of the latter. Ultimately, it's easier to reason about some domain in that domain. I would love my code for any particular project to look very similar to what I would write on a whiteboard when solving the problem. If I'm solving something inherently mathematical, I want the code to resemble math notation. If I'm writing some parsing code, I want it to resemble a grammar. If I'm writing an interpreter, I want it to resemble the operational semantics of the language. This fits into more "practical" domains as well: if I'm writing a web app, I want the routing to look like, well, routing and the database code to look as close to the database structure as possible. Any details in the implementation rather than the domain should be separated from the domain logic. If you're trying to make the domain logic mirror the domain itself, this is essentially inevitable. This makes your code more general and easier to debug. You can redirect your domain logic to a different backend, and it's much easier to figure out whether you have a problem in the implementation or the logic. More generally, such a strict separation of concerns is extremely important. The difficulty of a problem increases exponentially with its complexity: all the parts affect each other. If you manage to neatly split a problem like this in two, each sub-problem will be less than half as difficult as the main problem. So splitting away the high-level logic from the IO and the memory management and the other implementation details is inherently valuable. In fact, this is valuable even if the sum of both sub-problems is a bit greater than the original problem. The malleable language approach gives you a very natural way to split your problems up: one part is implementing the DSL, modifying the host language or writing a library--creating abstractions. The other part is actually using these abstractions. I find this sort of split to not only be very useful but also very natural; it mirrors how I usually think about things. One very interesting recent development taking this idea to the extreme is OMeta. There your program is essentially a tower of DSLs, with the base language just being a simple way to specify the DSLs. This idea really is extreme: for example, the code for implementing TCP is actually the ASCII art diagrams describing it! [1] Of course, all the usual suspects like Lisp and Haskell are also worth looking at. Embedded DSLs in these langauges can go quite far, and they have some of their own advantages to consider. [1]: http://www.moserware.com/2008/04/towards-moores-law-software-part-3-of-3.html http://www.moserware.com/2008/04/towards-moores-law-software...
army 14y ago
I agree with the general approach, but isn't this just bottom-up design?
pjbrow 14y ago
It's hard to argue with this article - it's self evident that languages lock us into abstractions that aren't powerful enough to comfortably support the increasingly ambitious applications of code. Object oriented programming was a big step because it's so heavily modelled on hierarchies - since found to be one of the most important principles in human reasoning. I'm no expert in the area, but I'd love to hear ideas on other human reasoning principles or models that a language could be influenced by (particularly if you know about neuroscience or AI).
coffeemug 14y ago
One simple example from everyday life: Imagine that you don’t have in your vocabulary notion of a sum. Therefore, if you would like to tell someone to do a sum of some things you would always have to express the process of summation in details instead of just telling and then you sum these numbers. I hear this a lot to explain the idea of abstraction to people, but the trouble with applying it to software is that picking really good abstractions is very, very very hard. Abstractions evolve over a long time under pressure of peer review and evolution of software systems. It took us ~2000 years to get a solid foundation for mathematical concepts and notation (we understood sum way before that, but you get the idea). We don't have 2000 years to pick really good abstractions for software systems. I'm convinced that a programming language must be really good at allowing you to change and evolve your programs as quickly as possible as your understanding of abstractions improves. This is far more important than more powerful ways to describe abstractions IMO. Functions are fine. Just let me change things drastically without breaking too much or having to rewrite half the darned thing, and I'll be happy.
- deleted 14y ago
  [deleted]
berlinbrown 14y ago
And yea Java is popular and doesn't really abstract much. I guess with interfaces, it is kind of a rough abstraction.
kamaal 14y ago
>>One exceptionally good resource is the Guy Steel’s keynote from OOPSLA’98 conference about Growing a Language. If you want to know one practical language today that embodies this philosophy and is yet main stream today, its Perl. Perl 6 takes it further. Infact Perl 6 by design is supposed to be such. Here is comment Larry Wall himself(http://www.perlmonks.org/?node_id=836573 http://www.perlmonks.org/?node_id=836573). Thought the whole thread was a troll thread. The first time I encountered this was like 6 months back while searching something related to Perl 6 on the internet. And it stuck me that how amazing this concept was. I then went back and watched some of his keynotes. I will recommend everybody to have a look at it. Perl 6 is definitely going to be a big game changer in this place. And unlike Lisp its a very practical language for your everyday programmer.
- cmwelsh 14y ago
  Perl 7 is going to be another Perl 5 stable release, just wait and see. No one is going to use Perl 6. See http://news.ycombinator.com/item?id=5179513 http://news.ycombinator.com/item?id=5179513
  - groovy2shoes 14y ago
    Then Perl and Scheme are more similar than I thought! See http://scheme-reports.org/2009/working-group-1-charter.html http://scheme-reports.org/2009/working-group-1-charter.html
- aerique 14y ago
  You are comparing a specific language (Perl 6) with a familiy of languages (Lisp). Some members of the Lisp family are actually very practical, take for example Common Lisp.
- sergiosgc 14y ago
  I had high hopes for perl 6. When the hoopla about versioning perl 5 came about I "remembered" perl 6 again and went on to read about its current state of development. My conclusion: it's a pharaonic project and these IT projects die more often than not :-( I'm afraid perl 6 is doomed...
keefe 14y ago
overall great article, particularly this exceptionally elegant turn of phrase - 'It is known since the early ages of programming, most elegant programs are built as layers of "languages" or "dictionaries". These are represented by abstractions in a program in order to facilitate expressing the ideas in a way that is more related to the domain.' Overall, I like to draw a line between canonical/ontological representations of models and representations of models designed for calculation. The ontological representation often times exists in a relational database or in an explicit OWL/RDF ontology, but may also exist only ideally, where the storage and calculation representations of the model are each customized for their particular use, especially at large scale. the following concept, I almost agree with... 'it is our job to express the abstractions of the domain for the best economy of the expression' As programmers (vs ontologists) I would say that our job is first to express the abstractions such that they enable efficient, maintainable computation that hopefully is expressing those abstractions economically. I feel like programming languages don't lack power, they just make it quite complicated to express complex abstractions.
michaelwww 14y ago
Isn't this really describing what it means to write a program? Assembler is really just mov, add, branch, etc.. but some really complex and useful programs have been written in assembler. Or am I missing something? I had to squint to read the text.
hdra 14y ago
Maybe its just that I don't have enough experience, but I've never met and can't think of any problem that may made think that I need to write a programming language. Most languages I know allows me to form some type of class/data structures for my need. What kind of problems are OP referring to? some kind of extensive math/scientific problem?
edem 14y ago
Have you tried Lisp?
marpalmin 14y ago
Check Jetbrains MPS
capkutay 14y ago
"One simple example from everyday life: Imagine that you don’t have in your vocabulary notion of a sum. Therefore, if you would like to tell someone to do a sum of some things you would always have to express the process of summation in details instead of just telling and then you sum these numbers." If you can encapsulate an undefined idea into one stream of commands, can't you just create a function to do it? What does that say about programming languages? Maybe I missed the point.
Joeri 14y ago
This is true from an academic perspective, but not from a practical perspective. Typical crud apps don't have complex problem domains. They're just electronic filers with a search field. You do need some dsl's, for formulating views, tying those views to models, and tying those models to persistent storage. That's why most popular frameworks implement pseudo-dsl's for MVC and ORM. MVC and ORM can be solved in a generic way at the framework or even language level, and it could be argued that you shouldn't try to solve them in a project-specific way to ease maintenance. There is little value for many (or even most) projects in having a more powerfully abstractable language.
- babarock 14y ago
  And maybe, just maybe, programming is more than "typical crud apps". A recurring bias here on HN is to believe that all programmers are web devs, stacking simple business logic on top of a database. What you call "many (or even most) projects" is alien to a lot of programmers.
- mattgreenrocks 14y ago
  > Typical crud apps don't have complex problem domains I beg your pardon, but I think you're forgetting that there are plenty of business apps that have very complex requirements due to legislation, a myriad of cross-cutting concerns, and audit requirements. Domain complexity is a very real thing, and it often necessitates architecture beyond cramming everything into this week's CRUD framework.
  - mamcx 14y ago
    Lets ride with that idea. The problem is that the info about functional programming show how maginificient it is for math-alike problems but barely never for crud-alike problems. I can understand some of the genius of haskell, but can't see how it make a better django or ruby on rails. CRUD's apps have the stigma of be "patetic simple projects". Are not! Building erp-like software is dam damm hard. Making a search engine? Where clever math solve it? Where well understood ideas of scalability are clear? NOTHING against the complexity of fly at the speed of a business, deadlines measured not in months, but HOURS. Underfunded, not enough staff, working on legacy code, and that is only the surface. Building a language to truly be a MASTER of CRUDs apps is something I truly wish for (the most close thing was Foxpro, IMHO). Despite my love for python, and what I know for my use of .net, ruby, obj-c, I don't think is something well solve yet
adamsaleh 14y ago
Could we say, DSLs are the new frameworks? For example for a web-dev in Java you'd use a MVC framework. In Clojure you get a query DSL to do your model, templating DSL to create your view and routing DSL to specify your controler.
Confusion 14y ago
Obligatory reference to Ometa [1], a simple almost-PEG parser embedded in a Scheme that allows the VPRI [2] (headed by Alan Kay of Smalltalk fame) to quickly build layers upon layers of DSL's. Their goal: to create a model of an entire personal computer system that is extremely compact (under 20,000 lines of code) [3] [1] http://tinlizzie.org/ometa/ http://tinlizzie.org/ometa/ [2] http://www.vpri.org/index.html http://www.vpri.org/index.html [3] http://www.vpri.org/html/work/ifnct.htm http://www.vpri.org/html/work/ifnct.htm
nollidge 14y ago
Low-contrast typography is not powerful enough.
protomyth 14y ago
I have worked at places that used the same language but had totally different custom business logic class libraries and practices. Learning a new DSL probably would have been easier since the <Biz Speak> => <Code> step would have been easier.

147 points by dylangs1030 14y ago

So this essentially sounds like "You gotta work at it to get it done."

This is where Common Lisp comes in: through macros you can expand the language to include the abstractions your application needs. PG talks about this extensively. The issue is that doing this requires a deep understanding of the language, and many programmers are not able to grasp what the CL toolbox gives them.

cageface 14y ago

The claim is often made that languages like Forth and Scheme are better for building applications than less elegant and orthogonal languages. But the evidence on the ground is overwhelmingly to the contrary. The vast majority of commercially significant apps are built in relatively baroque and ugly languages like Java and C++. Personally I think that the ability to define your own language constructs is ultimately a lot less useful than things like rich libraries and robust tooling and I'm still waiting to see real-world proof of the advantages of programmable programming languages.

BruceIV 14y ago
Yes, this. You have to learn all the abstractions in a language, and the advantage of "batteries included" languages are that the abstractions are (largely) the same in everyone's code, aiding maintainability and re-use. On the other hand, if you build everything yourself, it might be better optimized to your use case, but you'll be the only person who's easily able to use the abstractions.
tikhonj 14y ago
This argument comes up every time somebody suggest an unpopular tool, and I think it's inherently flawed. The core problem is that popularity does not imply--and does not even necessarily correlate with--quality! Just because something is in some real sense better does not mean people will adopt it. Most people are not willing to learn something fundamentally new. They might be content going from something they know to something very similar, say Java to Python, but they will resist anything truly novel and different. And the others? They're mostly the ones already advocating Haskell or Lisp or Forth or what have you! Also, people have some really odd reservations when switching to a new technology. They are often not willing to take any visible steps back: the new might be better in a whole bunch of non-trivial ways, but if it's obviously worse on some easily identifiable property, people will avoid it. A new language might have a long list of advantages, but if it has an inefficient default string type, or slightly wonky syntax, or a sub-optimal packaging tool or any other superficial but obvious shortcoming, people aren't willing to make the switch. Another problem is that there are different kinds of productivity. There are "dense" productivity changes: if you have to write your own JSON library or deal with a broken build system, you'll be spending a contiguous amount of time on it. You'll have to devote maybe a whole day or even a week to getting around this problem. There is no way to miss this. On the other hand, if a language improves productivity in a "sparse" sort of way--say you spend 20% less time writing code and 30% less on debugging and maintenance--you won't notice quite as easily. And yet, over any reasonable time using the language, you'll come out far ahead even if you have to sink in days working around problems and libraries. A particular--and particularly important--example of this is in learning. As I said above, one of the main reasons people resist new technology is that they don't want to learn. They're too busy to spend a whole week or even month picking up something new. And yet, learning time is essentially a constant expense. Productivity gains, on the other hand, are at least linear in how much you program. If a language makes your code simpler, the gains can even be super-linear (that is, you get more benefits as you write bigger programs). These will dominate any learning time as soon as you actually start using the new technology widely. And yet, since the amount of time spent learning is obvious and the ambient productivity gains aren't, people put a larger than warranted cost on the former. Coincidentally, this does not only apply to programming languages. I've seen exactly the same sort of behavior in adopting any kind of new, non-trivial technology: Emacs, Vim, Git, Linux...etc. In short, don't trust popularity. This probably makes me sound elitist (and, to be fair, I probably am), but it's just like music or literature: the popular stuff is usually not particularly good and the good stuff is usually not particularly popular.
- stuffihavemade 14y ago
  The difference between the popularity of Emacs, Vim, Git, Linux, and a programming language is that it doesn't matter if anyone else on the planet uses Vim Git, etc., they will work the same to me. But the difference in productivity from pre-made libraries for using Ruby (52,220 gems cut since July 2009) vs. scheme (http://planet.racket-lang.org/ http://planet.racket-lang.org/) is vast, especially when you consider that the power of the two languages is not that great.
  - irahul 14y ago
    > it doesn't matter if anyone else on the planet uses Vim Git, etc How does that work? Unless you are creating and supporting Git, git isn't going to work if you are the only person using it. Even then, there will be no github. And I don't know about you, but I won't use Vim if I lose my plugins(directly correlated to how many people are using vim)
    - stuffihavemade 14y ago
      Why wouldn't git work? The vast majority of projects I use it for are ones I work on by myself. As far as vim, the only plugins I use are syntax ones and command T, which I could live without.
- stormbrew 14y ago
  While I'll agree that popularity is not everything, and that often people are stuck with overall subpar tools because those tools do one thing particularly well (php and application deployment comes immediately to mind), I don't think you can reasonably argue that programmers as a whole are so mired in institutional inertia that they can't move to things that are better for them within a relatively fast timeframe. The rapid adoptions of, for example: C, C++, Perl, Java, Python, and Ruby make it painfully obvious that that's not the case.
- wvenable 14y ago
  Sorry but this argument comes up somebody suggests that unpopular tool is merely unpopular and not flawed. Lisp is one of the oldest programming languages in existence, it's still taught in Universities and has been for over 40 years. Age and exposure and still not popular. I'm not sure why you keep mentioning new technologies when the article is about old technologies and your examples (Emacs, Vim, Linux) are old technologies! The only example of a new technology, git, has seen an absolute rapid rise and completely changed the version control landscape in just a few years. A new language is not going to have a long list of advantages -- it's going to a have different list of trade-offs. Because any language feature that is objectively good with no downside or trade-off has already been implemented in some popular language somewhere. And if you can prove that you're doing something really different and really better but there's a strong learning curve people will learn it. Git is the perfect example.
rayiner 14y ago
The popularity of programming languages is entirely a matter of historical accident. C flourished while Pascal languished because UNIX was written in C. Was there really a deep technical reason for preferring C to Pascal? No, not really. Indeed, for most of early history, Pascal had far superior tooling to C. How else do you explain the popularity of languages like Javascript and Objective-C? Javascript is an objectively terrible language, hacked together by Brendan Eich in an afternoon. Yet, it's a tremendously popular language, and tremendous amounts of money and time have gone into building good tooling for it. Same thing with Objective-C. It's a shitty, ad-hoc, cobbled-together patchwork of C and Smalltalk, yet it's tremendously popular now because that's what the iPad is programmed in.
- krichman 14y ago
  I upvoted you for talking about historical accident, but regretted it when you started in on Javascript and Objective-C. Yes they have some questionable decisions but they are by no means objectively awful.
  - rayiner 14y ago
    Given that Javascript was designed in 1994, not 1974, I think it rises past "questionable design decision" not to just use Scheme's lexical scoping rules (i.e. the only sensible lexical scoping rules). Although to be fair, scripting language designers keep getting this wrong for some reason.
    - krichman 14y ago
      Yeah that's a problem. I thought you could get proper scoping by writing "var" in front of everything?
      - mercurial 14y ago
        That's function scope, not block scope. You can use the "let" keyword to get block scope since Javascript 1.7.
chipsy 14y ago
I think the problem is really that nobody is being taught in school to constantly redefine their language to create an ideal representation of the problem, when that's what "ideal" code tends to involve. The comfort zone is what you know, and if what you know is Java... However, I also think that we are making progress to go in the abstract direction, by dint of increasingly having polyglot systems and DSLs that are part of the standard libraries(e.g. regex and SQL access are expected everywhere now). If this trend accelerates we could have a hockey stick effect and before we know it, find ourselves programming almost entirely with tiny languages.
nnq 14y ago
And why do the rich libraries and tooling end up being built for languages like Java or C++? Because they require collaboration, and these languages, by handicapping the creation of DSLs actually make collaboration easier, and, more importantly, they make the collaboration between people of different IQs, backgrounds and expertise levels easier. ...sometimes handicapping the smartest person, more or less on purpose, makes him better at working in a team (even a "virtual" team like the one encompassing library creators and library users). It's ugly but it works! Of course, the ideal would be to have a team of very smart people - but when you can't have that , the next best alternative is to use a language that artificially "dumbs down" the smartest guys to a level similar enough to that of the others to allow collaboration... sad, I know :|
npsimons 14y ago
The vast majority of commercially significant apps are built in relatively baroque and ugly languages like Java and C++. I'm fairly certain there is vastly more code written in either FORTRAN or COBOL than Java and C++ combined. So I guess that makes FORTRAN and COBOL better than Java and C++ put together . . . ?
- TheLarch 14y ago
  That's an interesting assertion. I can't readily find estimates about the number of lines of code in different languages. Nor could I find an estimate as to the number of programmers in the world over time, from which I hoped to make an extrapolation. Somebody could write and interesting report about this. Maybe my research wasn't deep enough. Anyway, how do you figure there's more FORTRAN and COBOL?
- cageface 14y ago
  No. It means that language choice is less important than we like to think it is.

BruceIV 14y ago

Agreed, to a point. What kind of abstractions are sufficient, or useful? What happens when they leak? What happens when you build the wrong abstraction, and have to go back and build something slightly different from the ground up? What about when your abstraction works, but is so far removed from the actual execution environment that it's so slow as to be unusable? Yes, abstraction is a wonderful, useful thing that lets us think and do all sorts of wonderful, useful things - our high level programming languages and networking protocols are abstractions over lower level programming languages and networking protocols (repeat a few times) which are abstractions over the physical hardware they run on, and that's great. But abstractions are never perfect, and what you don't know about the lower levels of the stack will bite you, which is a good argument for using as few and as thin abstractions as you can to get your thing done (reasonably ... I'm not about to write this comment in binary ASCII, say).

nsmartt 14y ago
He mentions the point when the abstractions in the language are no longer sufficient. It seemed to me that this was more an argument in favor of using languages that are easily extended rather than languages full of abstraction with few options for extension. If you use a language like the latter, you'll still eventually reach a point where the abstractions aren't sufficient, but if you use a language like the former, though you'll reach that point sooner, you'll be better equipped to handle it. edit: word use

deleted 14y ago

[deleted]

Guthur 14y ago

For me a language is getting close to "powerful enough" when it does not get in the way of you as the programmer when you are developing abstractions of the form that seems most natural to you. This can cause problems in some cases; when the language is stepping that far back to allow you the freedom then it is up to your to enforce your own constraints to avoid your abstractions becoming...too abstract (maybe). Also in my opinion there is not one abstraction form that will appeal to every type of programmer. For me I like the freedom to manipulate and define symbolic forms with Common Lisp for others it might be the expressive type system of the Haskell. I don't think you can truly say one way is better than the other. I think those languages and a few others are giving the tools of abstraction that will not get in the way of you as the programmer.

SomeCallMeTim 14y ago
>enforce your own constraints to avoid your abstractions becoming...too abstract (maybe). Too abstract isn't necessarily the problem. When I've stepped too far away from standard OO structure, the problem typically becomes organization. In OO you know where just about everything SHOULD be, at least. If you're writing your own rules, it's harder to be certain where new variables get set, or what members you should find in what structures. It takes a certain level of discipline to not shoot yourself in the foot. Or at least to clean things up before they get too messy.
- Guthur 14y ago
  Depending on the implementation of the OO system in question it could provide you with relatively limited options for abstraction. I can only really speak from a Common Lisp point of view and the problem there that I was alluding to is the fact that you are so free to produce your own forms of abstractions whether it be mini DSL macros, dynamic dispatch method combinations, data driven programming using first class functions etc. that if you are not careful you can easily obfuscate the real intention rather than making it clearer.
  - SomeCallMeTim 14y ago

DeepDuh 14y ago

Am I the only one being put off by the CSS? I had to crank the display brightness to the max in order to read it without strain. Edit: I agree a lot with the content though - For many complex problems, especially the kind where you can't just throw some standard software solution at it, it would be helpful to first develop a suitable DSL. From personal experience I'd say spend one third of your time budget into developing your domain specific toolset - even if your managers get nervous. If it works you will finish sooner than you expected and you have developed reusable tools and skills, if it doesn't you can still throw in a crunch and get it done. For me it has always worked though.

aroman 14y ago
Perhaps it is because I am in a dark room reading in bed with a MacBook Air set at 50% brightness, but I actually really liked the experience of reading that. Though I can absolutely understand where someone might not.
- DeepDuh 14y ago
  I'm in a bright[1] office with an rMBP. I'm imagining everyone with displays that give less brightness / contrast is going to have a bad time. [1] It's a Japanese office. These people really like their cold bright neon. I think it has to with dark east asian eyes.
- Kluny 14y ago
  I liked it too, but I almost went blind when I came back to HN.
- cpeterso 14y ago
  You might like Flux. It adjusts your display's color temperature depending on the time of day. No more blinding screens when you open your MacBook at night. :) http://stereopsis.com/flux/ http://stereopsis.com/flux/
  - errnoh 14y ago
    Using flux as well but that site was one of the most enjoyable sites to read that I've seen in couple years. The colors aren't that beautiful but it's much less stressful for my eyes, and the highlight colors were spot on as well. Thanks to emidln for informing that it's Solarized Dark, I've used the default Solarized but I've skipped the Dark one because of somewhat ugly colors.

andolanra 14y ago

As a counterpoint to this article, I'd point to Yossi Kreinin's excellent "My History with Forth & Stack Machines" [1] which has been posted on HN before [2] and which discusses more extensively the particulars of Forth as a powerful toolbox of primitive operations, as well as the use of Forth in actual applications. [1]: http://www.yosefk.com/blog/my-history-with-forth-stack-machines.html http://www.yosefk.com/blog/my-history-with-forth-stack-machi... [2]: http://news.ycombinator.com/item?id=1680149 http://news.ycombinator.com/item?id=1680149

tikhonj 14y ago

I think this roughly reflects my philosophy in languages as well. You basically have two options: you can bend your problem to your language or your language to your problem. The former is epitomized by Java and Python--all Java and Python code looks the same, regardless of what it's actually doing. The latter is the domain of Lisp and Haskell and Forth and so on. I'm unabashedly in favor of the latter. Ultimately, it's easier to reason about some domain in that domain. I would love my code for any particular project to look very similar to what I would write on a whiteboard when solving the problem. If I'm solving something inherently mathematical, I want the code to resemble math notation. If I'm writing some parsing code, I want it to resemble a grammar. If I'm writing an interpreter, I want it to resemble the operational semantics of the language. This fits into more "practical" domains as well: if I'm writing a web app, I want the routing to look like, well, routing and the database code to look as close to the database structure as possible. Any details in the implementation rather than the domain should be separated from the domain logic. If you're trying to make the domain logic mirror the domain itself, this is essentially inevitable. This makes your code more general and easier to debug. You can redirect your domain logic to a different backend, and it's much easier to figure out whether you have a problem in the implementation or the logic. More generally, such a strict separation of concerns is extremely important. The difficulty of a problem increases exponentially with its complexity: all the parts affect each other. If you manage to neatly split a problem like this in two, each sub-problem will be less than half as difficult as the main problem. So splitting away the high-level logic from the IO and the memory management and the other implementation details is inherently valuable. In fact, this is valuable even if the sum of both sub-problems is a bit greater than the original problem. The malleable language approach gives you a very natural way to split your problems up: one part is implementing the DSL, modifying the host language or writing a library--creating abstractions. The other part is actually using these abstractions. I find this sort of split to not only be very useful but also very natural; it mirrors how I usually think about things. One very interesting recent development taking this idea to the extreme is OMeta. There your program is essentially a tower of DSLs, with the base language just being a simple way to specify the DSLs. This idea really is extreme: for example, the code for implementing TCP is actually the ASCII art diagrams describing it! [1] Of course, all the usual suspects like Lisp and Haskell are also worth looking at. Embedded DSLs in these langauges can go quite far, and they have some of their own advantages to consider. [1]: http://www.moserware.com/2008/04/towards-moores-law-software-part-3-of-3.html http://www.moserware.com/2008/04/towards-moores-law-software...

army 14y ago

I agree with the general approach, but isn't this just bottom-up design?

pjbrow 14y ago

It's hard to argue with this article - it's self evident that languages lock us into abstractions that aren't powerful enough to comfortably support the increasingly ambitious applications of code. Object oriented programming was a big step because it's so heavily modelled on hierarchies - since found to be one of the most important principles in human reasoning. I'm no expert in the area, but I'd love to hear ideas on other human reasoning principles or models that a language could be influenced by (particularly if you know about neuroscience or AI).

coffeemug 14y ago

One simple example from everyday life: Imagine that you don’t have in your vocabulary notion of a sum. Therefore, if you would like to tell someone to do a sum of some things you would always have to express the process of summation in details instead of just telling and then you sum these numbers. I hear this a lot to explain the idea of abstraction to people, but the trouble with applying it to software is that picking really good abstractions is very, very very hard. Abstractions evolve over a long time under pressure of peer review and evolution of software systems. It took us ~2000 years to get a solid foundation for mathematical concepts and notation (we understood sum way before that, but you get the idea). We don't have 2000 years to pick really good abstractions for software systems. I'm convinced that a programming language must be really good at allowing you to change and evolve your programs as quickly as possible as your understanding of abstractions improves. This is far more important than more powerful ways to describe abstractions IMO. Functions are fine. Just let me change things drastically without breaking too much or having to rewrite half the darned thing, and I'll be happy.

deleted 14y ago
[deleted]

berlinbrown 14y ago

And yea Java is popular and doesn't really abstract much. I guess with interfaces, it is kind of a rough abstraction.

kamaal 14y ago

>>One exceptionally good resource is the Guy Steel’s keynote from OOPSLA’98 conference about Growing a Language. If you want to know one practical language today that embodies this philosophy and is yet main stream today, its Perl. Perl 6 takes it further. Infact Perl 6 by design is supposed to be such. Here is comment Larry Wall himself(http://www.perlmonks.org/?node_id=836573 http://www.perlmonks.org/?node_id=836573). Thought the whole thread was a troll thread. The first time I encountered this was like 6 months back while searching something related to Perl 6 on the internet. And it stuck me that how amazing this concept was. I then went back and watched some of his keynotes. I will recommend everybody to have a look at it. Perl 6 is definitely going to be a big game changer in this place. And unlike Lisp its a very practical language for your everyday programmer.

cmwelsh 14y ago
Perl 7 is going to be another Perl 5 stable release, just wait and see. No one is going to use Perl 6. See http://news.ycombinator.com/item?id=5179513 http://news.ycombinator.com/item?id=5179513
- groovy2shoes 14y ago
  Then Perl and Scheme are more similar than I thought! See http://scheme-reports.org/2009/working-group-1-charter.html http://scheme-reports.org/2009/working-group-1-charter.html
aerique 14y ago
You are comparing a specific language (Perl 6) with a familiy of languages (Lisp). Some members of the Lisp family are actually very practical, take for example Common Lisp.
sergiosgc 14y ago
I had high hopes for perl 6. When the hoopla about versioning perl 5 came about I "remembered" perl 6 again and went on to read about its current state of development. My conclusion: it's a pharaonic project and these IT projects die more often than not :-( I'm afraid perl 6 is doomed...

keefe 14y ago

overall great article, particularly this exceptionally elegant turn of phrase - 'It is known since the early ages of programming, most elegant programs are built as layers of "languages" or "dictionaries". These are represented by abstractions in a program in order to facilitate expressing the ideas in a way that is more related to the domain.' Overall, I like to draw a line between canonical/ontological representations of models and representations of models designed for calculation. The ontological representation often times exists in a relational database or in an explicit OWL/RDF ontology, but may also exist only ideally, where the storage and calculation representations of the model are each customized for their particular use, especially at large scale. the following concept, I almost agree with... 'it is our job to express the abstractions of the domain for the best economy of the expression' As programmers (vs ontologists) I would say that our job is first to express the abstractions such that they enable efficient, maintainable computation that hopefully is expressing those abstractions economically. I feel like programming languages don't lack power, they just make it quite complicated to express complex abstractions.

michaelwww 14y ago

Isn't this really describing what it means to write a program? Assembler is really just mov, add, branch, etc.. but some really complex and useful programs have been written in assembler. Or am I missing something? I had to squint to read the text.

hdra 14y ago

Maybe its just that I don't have enough experience, but I've never met and can't think of any problem that may made think that I need to write a programming language. Most languages I know allows me to form some type of class/data structures for my need. What kind of problems are OP referring to? some kind of extensive math/scientific problem?

edem 14y ago

Have you tried Lisp?

marpalmin 14y ago

Check Jetbrains MPS

capkutay 14y ago

"One simple example from everyday life: Imagine that you don’t have in your vocabulary notion of a sum. Therefore, if you would like to tell someone to do a sum of some things you would always have to express the process of summation in details instead of just telling and then you sum these numbers." If you can encapsulate an undefined idea into one stream of commands, can't you just create a function to do it? What does that say about programming languages? Maybe I missed the point.

Joeri 14y ago

This is true from an academic perspective, but not from a practical perspective. Typical crud apps don't have complex problem domains. They're just electronic filers with a search field. You do need some dsl's, for formulating views, tying those views to models, and tying those models to persistent storage. That's why most popular frameworks implement pseudo-dsl's for MVC and ORM. MVC and ORM can be solved in a generic way at the framework or even language level, and it could be argued that you shouldn't try to solve them in a project-specific way to ease maintenance. There is little value for many (or even most) projects in having a more powerfully abstractable language.

babarock 14y ago
And maybe, just maybe, programming is more than "typical crud apps". A recurring bias here on HN is to believe that all programmers are web devs, stacking simple business logic on top of a database. What you call "many (or even most) projects" is alien to a lot of programmers.
mattgreenrocks 14y ago
> Typical crud apps don't have complex problem domains I beg your pardon, but I think you're forgetting that there are plenty of business apps that have very complex requirements due to legislation, a myriad of cross-cutting concerns, and audit requirements. Domain complexity is a very real thing, and it often necessitates architecture beyond cramming everything into this week's CRUD framework.
- mamcx 14y ago
  Lets ride with that idea. The problem is that the info about functional programming show how maginificient it is for math-alike problems but barely never for crud-alike problems. I can understand some of the genius of haskell, but can't see how it make a better django or ruby on rails. CRUD's apps have the stigma of be "patetic simple projects". Are not! Building erp-like software is dam damm hard. Making a search engine? Where clever math solve it? Where well understood ideas of scalability are clear? NOTHING against the complexity of fly at the speed of a business, deadlines measured not in months, but HOURS. Underfunded, not enough staff, working on legacy code, and that is only the surface. Building a language to truly be a MASTER of CRUDs apps is something I truly wish for (the most close thing was Foxpro, IMHO). Despite my love for python, and what I know for my use of .net, ruby, obj-c, I don't think is something well solve yet

adamsaleh 14y ago

Could we say, DSLs are the new frameworks? For example for a web-dev in Java you'd use a MVC framework. In Clojure you get a query DSL to do your model, templating DSL to create your view and routing DSL to specify your controler.

Confusion 14y ago

Obligatory reference to Ometa [1], a simple almost-PEG parser embedded in a Scheme that allows the VPRI [2] (headed by Alan Kay of Smalltalk fame) to quickly build layers upon layers of DSL's. Their goal: to create a model of an entire personal computer system that is extremely compact (under 20,000 lines of code) [3] [1] http://tinlizzie.org/ometa/ http://tinlizzie.org/ometa/ [2] http://www.vpri.org/index.html http://www.vpri.org/index.html [3] http://www.vpri.org/html/work/ifnct.htm http://www.vpri.org/html/work/ifnct.htm

nollidge 14y ago

Low-contrast typography is not powerful enough.

protomyth 14y ago

I have worked at places that used the same language but had totally different custom business logic class libraries and practices. Learning a new DSL probably would have been easier since the <Biz Speak> => <Code> step would have been easier.