6 ms·
I've been observing something akin to Zawinski's Law ("Every program attempts to expand until it can read mail. Those programs which cannot so expand are replac
by munificent 6y ago
I've been observing something akin to Zawinski's Law ("Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.") but for programming languages:
Every dynamically-typed programming language attempts to expand until it has static types and compiles to machine code.
* Common Lisp has long had type annotations that can be use to generate specialized code.
* JS started out as a simple bytecode VM, then got a JIT from V8 and others, then TypeScript came and gave it static types.
* Python has had several JITs over the years—Unladen Swallow, PyPy, etc. It got static types with mypy (and others) and is now adding type annotations directly to the core language.
* Ruby's 3x3 plan involves adding a type-specializing JIT. Sorbet adds static types and I believe Matz wants the core language to go in that direction.
* Facebook created Hack to statically type PHP and PHP core added "scalar type dependencies". Facebook's HipHop VM brings a JIT to PHP.
* LuaJIT brought a high performance JIT to Lua and there's been a number of projects that layer static type annotations onto the language.
* Dart started with an optional type system and moved to a fully sound static type system.
So what I see is a language that starts out simple and dynamically typed with a set of core libraries and idioms designed around dynamism. Then later people add static types on the front end to help people maintain larger programs. And they add type specializing JITs on the back end to generate faster code.
But right in the middle you're still stuck with a mountain of existing code designed around the assumption that code and data don't need to be statically shaped. So even though you end up doing all the work (and adding all the complexity) to design a static type system and native code generator, you don't get the full benefits.
The static type systems are almost always unsound in order to play nice with existing dynamic idioms, so the back end can't use the static types for optimization purposes. You end up with these fantastically complex type systems like TypeScript's and these incredibly complex JITs, but you still don't get the performance you get from a simple fully-statically typed language like Go or, hell, Pascal.
I think the reasonable take-away is that if you ever intend your language to be used for large programs, just take the hit and start off with static types. Your future self will thank you.
- chubot 6y agoI think the reasonable take-away is that if you ever intend your language to be used for large programs, just take the hit and start off with static types. Your future self will thank you. I think this is missing something: the creators of those languages wanted languages that were fun/productive for writing small programs. That is, extremely flexible languages (and yes maybe they were unaware back in the 90's how this would limit their optimization potential) And all big programs were once small programs. Facebook is probably one of the easiest to see, since it was "just" a bunch of PHP scripts (although people tend to underestimate/dismiss it for that reason). Just like nobody ever says: "Well I estimate that in 10 years Facebook will consist of 10M lines of PHP with 5,000 programers working on it -- I should probably write it in another language". Nobody ever says: "Well I think my language is going to be used by millions of people and will have billions of lines of code". Well, there probably were people who thought that, but those were exactly the people who didn't make languages as useful as Python and JS :) ---- That said I think the phrase "irrational exuberance languages" referring to Python/JS is kinda funny, and in a way accurate ... https://blog.sigplan.org/2020/10/12/from-heavy-metal-to-irrational-exuberance/ https://blog.sigplan.org/2020/10/12/from-heavy-metal-to-irra... Although again I would say the unexpected part was not that they thought single core scaling would continue forever and make their languages fast, it's that those "slow" languages turned out to be the "best" ones for writing some of the most important systems of the last couple decades (not just commercial ones, but also Wikipedia, BitTorrent, etc.) ---- I'd make another analogy, to ISAs. If you talk to anyone who knows about CPU design, they'll say that x86 is shitty with a big pile of hacks. "It would be better" if someone designed it from the beginning with current applications in mind. But if anyone actually did that back in 1980, they wouldn't have been successful. And from my perspective, I mostly don't care, because the C compiler makes it all work for me (although I know the people who make it work care very much). So I guess the point is that technology adoption proceeds by evolution, and trying to plan 10 or 20 years ahead of time never works. ---- Also, there is a pretty hard tradeoff between static types and metaprogramming. Recent languages have come closer to reconciling these features (Zig, Nim, D), but dynamic languages chose reflection/metaprogramming, and that's a primary reason why they became successful. Ruby on Rails is a great example of that. It uses Ruby metaprogramming/reflection to a hilt, and lots of people who have no idea what that is love it, and they built tons of things with it (which now makes it an interesting optimization target)
- munificent 6y agoYeah, there's a longer essay you or I could write about path dependence here. The language that enables you to get to the point where your codebase is that big in the first place may not be the language best suited to that codebase once it has. That's the main argument in favor of optional type systems like TypeScript. Start small in a dynamically typed language and then add the types later when you need them. But my personal experience is that I've never found types to cause much friction when programming in the small. What I have found painful is not having GC and not having type inference. If I was doing a startup and needed to be able to prototype and change quickly, then C++ or Java would be pretty painful. But C# or any modern typed, managed language with a decent modern IDE? I would be surprised if you were any less productive than someone using Python or Ruby.
- chubot 6y agoYeah, "path dependence" pretty much sums up the comment. I should probably write something about that, because a huge factor in the design of https://oilshell.org https://oilshell.org. But I still get questions about it (like "Why can't we just get rid of shell and start over from scratch?") A few points I would like to add on: - I don't view the existence of TypeScript, MyPy, and Sorbet as evidence in favor of static typing. It's evidence in favor of gradual typing! - The Oil project gave me a lot of experience with the relation between metaprogramming/reflection, dynamic types vs. static types, performance, and code length. In particular the code moved from dynamic to static typing, and got a lot faster. But dynamic typing wasn't a mistake. Short recap: Oil's code is 5-7x shorter than bash [1], and a lot of that is due to starting out as dynamically typed, with a lot of metaprogramming. (I still believe "size is code's worst enemy" -- big code is understood more poorly, which makes it harder to modify, regardless of static types.) It is also something like 30x-50x slower than bash in Python! So, unusably slow. However the surprise is that I statically typed this code, and semi-automatically translated it to C++, and the result is now faster than bash. [2] ----- So the high level, short code has enough semantic information to be fast (after you add explicit types). The static typing process mainly evolved expanding metaprogramming to textual code generation! I estimate that this was at least 9 months of rewriting. That's what your arguments are missing IMO. If you're writing Java in Python, then sure Python is going to seem like it offers no advantages, and it might as well be statically typed. But that's not how people write programs in dynamic languages (and honestly I thought you would appreciate that more, having written so much about dynamic languages!) The porting process taught me exactly how much dynamism I was using, and it was a lot! It was pulling a lot of weight. I should show all the code generators and generated code in an essay... it's a very concrete demonstration. So there is the fallacy of "type inference" solves the problem -- it's not that we're too lazy to write down types; it's that we're using techniques that static type systems can't handle. Good thread about that: https://twitter.com/sliminality/status/1317331149354463232 https://twitter.com/sliminality/status/1317331149354463232 I'm not saying the Oil experience generalizes, since it's an unusual project, but it's definitely not as simple with "go with static types so you don't get trapped". That said, the conundrum you're talking about is very real. ----- But despite writing all that, I'm actually leaning in your direction, and I started a statically typed language :) https://old.reddit.com/r/ProgrammingLanguages/comments/jb5i5m/help_i_keep_stealing_features_from_elixir_because/g8urxou/ https://old.reddit.com/r/ProgrammingLanguages/comments/jb5i5... I would start with an interpreter so it can have metaprogramming (e.g. like a constexpr interpreter, or what Zig does). I would like to add a gradual type system, but I don't really know how to write one, so the first cut will be a traditional static type system. (In this world, getting the 30-50x speedup relies on the program being 100% statically typed, yet gradual typing is still important IMO. It's very simple, no infinite treadmill of JIT work as you see in the "professional" projects.) ----- This is probably something for an essay, but I would say dynamic types are demonstrably better than static types for at least 3 domains: UI, data science, and security/reverse engineering (and I have a bunch of experience to back this up). Basically anything that involves "learning about the world", or "schema discovery". [1] http://www.oilshell.org/blog/2019/06/17.html#why-is-it-written-in-python http://www.oilshell.org/blog/2019/06/17.html#why-is-it-writt... [2] https://www.oilshell.org/blog/2020/01/parser-benchmarks.html https://www.oilshell.org/blog/2020/01/parser-benchmarks.html Good thread about how successful Ruby and Python have been in YC companies: https://news.ycombinator.com/item?id=24279611 https://news.ycombinator.com/item?id=24279611
- deleted 6y ago[deleted]
- jononor 6y agoStatically typed languages are going the other way to. Like focusing on ergonomics for dealing with key-value data with heterogeneous types (like JSON objects), to generics/traits not needing a specific baseclass, and structural typing more generally, and type inference to avoid having to specify types. So I think we are more seeing an ongoing convergence on combined static/dynamic typing approaches.
- morei 6y agoI think the convergence here is toward 'automatically discoverable types'. For python et al this means type annotation (because it's extremely difficult for discover types otherwise). For C++ et al this is things like the 'auto' keyword that allow skipping the type specification where the compiler can work it out. The end-game is specifying just enough type information that it's still understandable to both humans and compilers.
- throwaway189262 6y agoTypescript is extremely good at this. Rust is pretty good. Even Java is okay now. I firmly believe type inference is the future. Most objects in dynamic languages are statically typed anyways, the type just isn't exposed at compile time.
- hhas01 6y agoType inference is good, but it’s not a full answer. For instance, how often do you declare a variable as `int`, when what you really want to say is “an integer in the range 0...100”? A few languages (e.g. Eiffel) provide a formal mechanism for declaring these sorts of constraints, but most don’t, and you end up putting what should be declarative type-level information into the body of your code instead. And then there’s “cutting-edge” stuff like dependent types, where you really want to express one argument’s type in terms of another argument’s, a classic example being an array indexing method, where you really want to declare the index at compile time as an integer in the range `0..<array.length`, and let the type system propagate that rule and its implications throughout the code that uses it. Whereas most “modern” languages chuck a run-time error if you’re lucky; or just ralph and dump stack like some antiquated 1970s throwback (yeah, looking at you, Apple’s Swift). 3/10 Could do much better.
- kaliszad 6y agoSo where does Clojure and GraalVM fit in?
- chubot 6y agoNot to beat this into the ground, but I realized I could have written a much shorter response. This idea to "start with static types" presumes that there are no benefits to dynamically typed languages! But the simplest argument is also the one I think that's the most true: dynamically typed languages let you explore the problem domain with faster feedback, leading to more successful software. ----- Quote from John Ousterhout: The greatest performance improvement of all is when a system goes from not-working to working [1] And we should never be cavalier about how hard that is. Most software projects fail; the successful ones are miracles! I also take an expansive definition of "working" -- i.e. "useful to its users". [1] https://wiki.tcl-lang.org/page/John+Ousterhout https://wiki.tcl-lang.org/page/John+Ousterhout
- hhas01 6y agohttps://en.wikipedia.org/wiki/Gradual_typing https://en.wikipedia.org/wiki/Gradual_typing Also, TBH, types are kind of the wrong answer to the question. The question should be: How do we make the language efficiently expressive? For my money, that means defining constraints primarily on the code’s main interfaces, which a given constraint may be a combination of traditional generics, dependent types, and/or declarative run-time checks on input/output values. (e.g. Eiffel comes to mind.) For instance, in my kiwi language I can define an argument as: list (whole number (0, 100), 4, 4) Which is to say a 4-item list, where each item is an integer between 0 and 100. And since I don’t want to type all that every time I can give that constraint a descriptive name: define type (CMYK color, list (whole number (0, 100), 4, 4)) I can even include user documentation in that definition so the whole lot’s self-describing. Kiwi’s very late-bound and interpreted, so its constraints are implemented solely as run-time coercions with optional bounds checks; nothing fancy. But the semantics are quite well formalized so a linter could be implemented as an assistant authoring tool for users, and if you can implement a linter then you can implement a type checker; and so on. What matters is that the language has a formal mechanism by which it can guarantee that a given value will always satisfy one or more user-defined requirements; and whether those requirements are checked at compilation, execution, or some combination is secondary to that. How rigorous the user makes these declarations, and if/where she makes them, is entirely up to her. Remember, the goal of any language is to please its users. Not the machines it runs on, nor the designers who created it. And users’ needs are not constant, not even across the development cycle of a single program, so a language that cannot adapt to those users’ changing requirements as it goes has already failed its first hurdle. .. Perhaps if authors of existing languages like Python and C put less effort into chasing the constantly-diminishing returns of post-hoc micro-optimizations, and more into thinking how to design the next generation of languages so as to carry forward the good characteristics of their predecessors minus their original already-painted-themselves-into-a-corner limitations, we might actually have languages that tick all the boxes by now.
- teddyh 6y ago> I've been observing something akin to Zawinski's Law ("Every program attempts to expand until it can read mail. Those programs which cannot so expand are replaced by ones which can.") but for programming languages: Even more applicable, in this case, would be Greenspun's tenth rule: “Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.” https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule https://en.wikipedia.org/wiki/Greenspun%27s_tenth_rule
- kazinator 6y agoIt occurs to me that whereas plenty of such C programs have been seen, where can we find such a Fortran program?
- kazinator 6y agoI totally forgot that Lisp's List processing came from Fortran: a system called FLPL. This had the CAR and CDR naming (in an ugly form, with additional prefix and suffix characters: XCARF and XCDRF). https://www.informatimago.com/articles/flpl/flpl.html https://www.informatimago.com/articles/flpl/flpl.html
- kazinator 6y ago> Then later people add static types on the front end to help people maintain larger programs. More like, people whose hobby it is to create static front ends for dynamic languages will invariably invade the ecosystem of any sufficiently popular dynamic language.
- brokencode 6y agoMost of the projects listed there are made by large companies, not hobbyists. I know there are a lot of dynamic language fans out there, but it really does seem that there is business value in using static languages.
- girvo 6y agoCore PHP itself has a JIT and is even today expanding its type hint system to be more expressive and powerful/strict
- BerislavLopac 6y agoOne important point being that, while the languages mentioned have expanded to include static typing and native compiling, they haven't stopped being what they were originally -- they have expanded, rather than transformed. In other words, the meaning of "dynamic languages" can be taken to mean "everything a language might need to be efficient and useful".
- machiaweliczny 6y agoI think we could've 2 backends if possibile. Interpreter for prototyping and compiler for production. I usually don't want interpreter in PRD and don't want compiler when prototyping. That's why TS is sucessful IMO. You can just test stuff with using emit only while still having pieces of code highlighted that don't make sense yet.