3 ms·
Yes I totally agree -- C is way more productive than assembly, and dynamic languages are way more productive than C. I think those are incontrovertible facts t
by chubot 7y ago
Yes I totally agree -- C is way more productive than assembly, and dynamic languages are way more productive than C. I think those are incontrovertible facts that have been proved over and over again in the market, according to the test in my post.
I'm arguing against the OP who is essentially claiming that the homoiconity and metaprogramming/DSLs of Lisp are a huge advantage over say Python.
I would argue the opposite: all the essential innovations of Lisp made its way into other languages like Python and Ruby (and Julia, which has a Lisp-like macro system by way of its femtolisp front end [1]). It was a hugely influential language but it's not ahead of the state of the art right now.
Another way of saying it: PARSING is the only difference between Lisp metaprogramming and metaprogramming in any other language, and parsing isn't an essential difference. It's a bit of friction. (Though I want to improve parsing, long story ... [2] )
-----
In fact I've done what you could consider a software engineering experiment along these lines in the last few years.
I wrote a bash-compatible shell, running thousands of lines of unmodified bash scripts, in Python. It's 5-7x fewer lines of code than bash, due to metaprogramming with Python, ASDL, and re2c. I claim it's significantly faster to develop and has fewer bugs because of this. It's very very different than how bash is written.
Line counts: http://www.oilshell.org/blog/2019/06/17.html#why-is-it-written-in-python http://www.oilshell.org/blog/2019/06/17.html#why-is-it-writt...
Well it took over 3 years, but bash has been around for 30, and I also enhanced the language in many ways. I won't argue if someone says the jury's still out, but here is a significant achievement on performance:
http://www.oilshell.org/blog/2020/01/parser-benchmarks.html http://www.oilshell.org/blog/2020/01/parser-benchmarks.html
Despite being written in Python, and parsing more thoroughly by design, the parser (which is a ~10K line program in Python, and more in bash) is faster than bash's once it's automatically translated to C++. So basically the high level "DSL" of Python or "OPy" retains enough information to make it better than the state of the art.
Here's a comment on this non-Lisp heavily DSL-based architecture from a few months ago:
https://news.ycombinator.com/item?id=20799926 https://news.ycombinator.com/item?id=20799926
To summarize it, if you look at how bash is implemented, it's 140K lines of "groveling through backslashes and braces one at a time in C", as I like to call it. And it has about as bugs / mistakes as you'd expect with that style, including some serious security problems.
So you can use DSLs to "compress" that code -- and I claim make it more correct and faster (jury is still out on memory usage). And you don't need Lisp to do it. Python was a fine choice for metaprogramming and DSLs; Lisp doesn't have any advantage.
In fact in January 2016 I hacked on femtolisp specifically because I was thinking about borrowing Julia's implementation approach for Oil. But yeah it's not enough of a win to justify the downsides. If someone wants to prove me wrong, write a bash-compatible shell in Lisp in under 3 years ;)
It's not going to be any easier, just like writing an x86 backend isn't much easier in Lisp. The difficulty is from the problem domain and not the language. And I noticed that all compilers use metaprogramming and DSLs anyway -- LLVM has hundreds of thousands of lines of TableGen. The Go compiler uses a Lisp-like DSL to express SSA transformations.
So basically Lisp is great but it's been thoroughly absorbed, and PG basically extended a falsifiable argument that was proven false. All of YC's biggest successes use dynamic languages, not Lisp. Stripe uses a ton of Rails, AirBNB is Rails, Dropbox is Python, etc.
https://www.ycombinator.com/topcompanies/ https://www.ycombinator.com/topcompanies/
Instagram and YouTube are Python, etc. Wikipedia and Facebook are PHP, etc. I think that constitutes enough evidence.
-----
edit: I should also add/clarify that I've met programmers who I'm sure can bang out the equivalent of bash's 140K lines of C in (much better) C in a relatively short period. I'm not saying the way I did it is the only way to do it. Rather, I'm saying Lisp doesn't have any fundamental advantage in metaprogramming. The only difference is parsing, which is a small bit of friction.
The other point which people have brought up many times is that most people are bad DSL designers. It may be better to rely on "known" DSLs than to invent your own for every program. That can be a drag on productivity negating any of the potential benefits of smaller code.
[1] https://docs.julialang.org/en/v1/manual/metaprogramming/ https://docs.julialang.org/en/v1/manual/metaprogramming/
[2] This was supposed to be the start of that argument http://www.oilshell.org/blog/tags.html?tag=parsing-is-difficult#parsing-is-difficult http://www.oilshell.org/blog/tags.html?tag=parsing-is-diffic...
- kragen 7y ago> dynamic languages are way more productive than C. I meant to say that high-level languages were more productive than C; I think there are a lot of people who will be happy to tell you that they are more productive in Haskell than in C, as much so as in Python, even though Haskell is a lot more static than C. (I don't know Haskell well enough to say; the only Hindley–Milner language I know is OCaml, and I feel less productive in OCaml than in C.) > I'm arguing against the OP who is essentially claiming that the homoiconity and metaprogramming/DSLs of Lisp are a huge advantage over say Python. ¶ I would argue the opposite: all the essential innovations of Lisp made its way into other languages like Python and Ruby I pretty much agree. Lisp was a unique language in, say, 1990, when it had garbage collection, dynamic typing, dynamically expanding data structures, higher-order functions, eval, easy serialization and deserialization of data structures, an object-graph memory model that made it easy to represent things like parse trees, and easy metaprogramming through homoiconicity. Meanwhile, the other popular programming languages were things like C, Pascal, BASIC, FORTRAN, sh, Lotus 1-2-3 formulas, dBASE, and COBOL, which had none of those; a lot of them didn't even have recursion. Today, though, Python has all of them except for homoiconicity, and we can do quite a bit of metaprogramming in Python, using its pervasive polymorphism and higher-order programming rather than macros. Most other popular modern languages like Perl 5, Ruby, JS, Lua, and PHP 5+ are basically dialects of Lisp. Even languages that differ dramatically from Lisp, like OCaml, Kotlin, C#, Swift, and recent versions of Java are pretty close. > I wrote a bash-compatible shell, running thousands of lines of unmodified bash scripts, in Python. It's 5-7x fewer lines of code than bash, due to metaprogramming with Python, ASDL, and re2c. I claim it's significantly faster to develop and has fewer bugs because of this. It's very very different than how bash is written. That's a very impressive achievement! Does it use more RAM than bash does? I'd think that one of the potential benefits of C over Python is that, because C requires you to specify your memory layout, C programs tend to use significantly less memory than otherwise equivalent Python, usually by about an order of magnitude. That is, some of the order-of-magnitude complexity reduction you got in your experiment is surely due to Python, but perhaps some of it is due to RAM costing about US$100 per megabyte when Brian Fox started writing bash in 1988. > Python was a fine choice for metaprogramming and DSLs; Lisp doesn't have any advantage. I think that may be overstating the case; no doubt there are some things that are easier in Python and some things that are easier in Lisp. SBCL or Racket can usually beat CPython or Jython by about an order of magnitude on speed, for example, and having first-class lambdas really is an improvement over Python's botched mixture of context managers, function decorators, and generator abuse. In the other direction, Python's syntax is significantly more readable — especially for arithmetic — and SBCL and Racket don't have anything like Numpy, and their own compilers are not an adequate replacement. But the difference between Lisp and Python, whichever one it favors, certainly isn't close to the enormous advantage Lisp or Python has over Pascal. (And I think Clojure is really the most interesting Lisp, actually.) > I should also add/clarify that I've met programmers who I'm sure can bang out the equivalent of bash's 140K lines of C in (much better) C in a relatively short period I would be very surprised about that; but Andrew Appel did come close to winning the ICFP programming contest one year using that well-known functional programming language, C. And a few years later someone won using C++. On the other hand, you did end up writing OSH in OPy rather than, strictly speaking, in Python. The initial Python versions sort of ended up being a prototype, in a way. And you could presumably have written OPy in some other language. Maybe it would even have been easier to write it in OCaml or F# or Haskell. (But if the language it compiled were less compatible with Python, I guess you would have had to rewrite a lot of the Python standard library.) So maybe that's what "much better C" would look like in the context of bash: a smallish core compiler or interpreter for the DSL in which the rest of the shell is written?