4 ms·

Maybe we should be investing more time into creating meta-platforms so that we don't need millions of dollars to bootstrap production-ready languages.

shadowgovt 4y ago
I don't think it's a platform problem... It's a money problem. I've got a toy language I've built at home for my own reasons. Lots of developers do. The difference (especially in these modern times) between my toy and Rob Pike's toy is that Rob is paid by Google to develop it, hammer out the details, work out the bugs, and support it. As are the whole team Google dedicated to it. Some languages get supported by volunteer armies (Lua comes to mind, last I checked), but TypeScript, Go, and Rust are supported / maintained by corporate money and it's hard to compete with that because a corporation can throw money at a language that isn't done yet while it's harder to get volunteers excited about a broken toolset.
- nandalism 4y ago
  I constantly hear that "we can't do it without corporations" and yet there are so many counter examples. Most of the corporations were built on free languages and ecosystems which existed before they did.
  - deleted 4y ago
    [deleted]
  - mattgreenrocks 4y ago
    There's been a shift away from vague distrust of corporations (think Slashdot) to full-on embrace. The fallout from that is we can't imagine platforms that don't require tens of millions of dollars to bootstrap, and we cling to the ones we have because they're so expensive. Too big to fail is not the hacker ideology I grew up with. Maybe that's gone now.
    - shadowgovt 4y ago
      There's very little in the open source space that competes with, for example, npm + TypeScript + VSCode for usability (where "usability" here runs the gamut from package / dependency management to code comprehension / documentation / refactoring to debugging). That kind of polished integration often costs money. In contrast, in the open source world we limped along with gcc + gdb for decades until Clang was able to force the issue (which was funded by Apple, Google, Microsoft, ARM, Sony, Intel, and AMD).
      - OkayPhysicist 4y ago
        That strongly depends on your definition of "usability". If it means appealing to the lowest common denominator, that's probably fair. Open source software tends to lean technical, because it's made by technical people for themselves. If it means being useful to the people who use it, then I don't think your argument holds water. NeoVim and Emacs at least fairly compete with VSCode, npm is not at all special in the world of package management (Julia's Pkg and Elixir's Hex come to mind as OSS superior offerings), and the world of programming languages is chock full of better languages. To directly counter your "npm + TypeScript + VSCode", I counter with "Hex + Elixir + NeoVim".
        shadowgovt 4y ago
        > If it means being useful to the people who use it, then I don't think your argument holds water I recently worked on a project involving a couple Docker containers, one of which wraps a Postgres DB and the other extends it. Some of the features of the extension are broken because the Postgres maintainers changed something key in their Docker image (they disabled the DB listening on localhost) and the scripts the extension generates assume localhost is accessible. In the (volunteer, free) open source ecosystem, nobody is paid to have that be their problem to solve and so problems like that, to my observation, rarely get solved because these two groups are unrelated to each other and it's nobody's job to make sure "the whole offering" is easy to use. (And as an ardent and frequent user of Emacs, I simply must see the config file someone claims to have put together that competes with VSCode for managing TypeScript projects. That's a hard claim to swallow without evidence; I've tried, and it's extremely non-trivial. Emacs just pulled in eglot for LSP support... Not that one couldn't self-install it earlier, but every time you have to mutate a system to one-off your own solution you increase the maintenance burden of that system.) ... I'll have to carve out some time to look at Elixir. In my world, of course, "Doesn't have a tool to compile to JavaScript that has made it to v1.0 yet" is a major strike against it, but my problem domain is specific. ;)
  - pwinnski 4y ago
    I'm not sure what you're hearing is "we can't do it without corporations" so much as it is "the ones sponsored by corporations are better." I mean, GIMP exists, but any one of a half-dozen commercial products built by fewer people in less time are easier to use and have better feature lists. It's not that it's not possible for GIMP to be best-in-class, it's that it isn't best-in-class. (And sure, dear reader, GIMP works perfectly well for you. Great!) Given more time, I'd love to explore the idea that "most of the corporations were built on free languages and ecosystems" given how many of the companies we're talking about today seem to have some roots in C, developed under corporate control. But definitions are everything in a statement like that, so it probably would take a lot of time to sift through and ultimately convince nobody either way.
- mattgreenrocks 4y ago
  Google's Go team is not that large AFAIK. It's probably my love affair with Lisp speaking, but I really think there is space for PLs to implement 80% of production-level features with 20% effort and improve slowly as they turn out to be blockers. There's also a branding thing: users who are put off by lack of corporate funding won't be working on the ground floor of building a PL anyway. That's fine. LLVM/JVM/WASM give leverage to both indie and corporate langs. More of those types of tools are great.
- enneff 4y ago
  The difference between your toy and Robert, Rob, and Ken’s toy is that yours was built for your own reasons and theirs was built for the purposes of software engineering at scale. Whatever your thoughts about Go suiting that purpose, that was the intent, and why they were supported by Google in doing it.
brabel 4y ago
Exactly. This is something I believe will happen in a few decades (progress in this are is very slow). The JVM showed that a runtime can host a multitude of languages, compile to multiplatform bytecode that require only a "small" runtime (the core of the JVM is pretty simple, the complexity is on the stuff on the edges, like the stdlib) to run on anything and can be ported easily to any architecture. WASM is the new incarnation of the idea and if the people behind its specification stop worrying about politics and start getting stuff done, I am almost sure it will become the new "universal" runtime. Usually, new programming languages have a huge mountain to climb in terms of libraries that are required for everything these days (encryption, multiple data formas like YAML, JSON, XML, TOML and even the old ones that are still widely used in niches, like ASN.1... HTTP, Database Drivers, logging, monitoring, graphics, mathematics, the list goes on and on... we don't really want to rewrite all the stuff that exists... but interop between languages is very hard and sometimes very inefficient unless it's based on a common set of primitives (like Java bytecode... WASM still needs a lot of features like GC and interface types to be a real alternative). Zig and other languages that have very good and easy interop with C work around this problem. But I think that the future will be a universal runtime, maybe WASM, maybe something else that doesn't exist yet (or that's not yet known to most of us).
- Tozen 4y ago
  > ...easy interop with C work around this problem... Think this is a good point. Languages like Vlang, Dlang, Nim, etc... Use their interop with C to their advantage. They are not starting out in the wilderness, but can build off of the already strong C ecosystem, then add unique advantages that their userbases want or prefer.
- mike_hearn 4y ago
  JVM has moved beyond bytecode now. See my sibling comment. Truffle takes the JVM to the next level - you don't have to compile your language to bytecode anymore, you write an interpreter for the source code (or a binary form if you have one already) and then the JVM turns that interpreter into a JIT compiler, giving it many other features and tools along the way. You aren't constrained by what Java bytecode can express.
- 4y ago
hgsgm 4y ago
Like RPC and IPC and SWIG? We have that.
justinjlynn 4y ago
I've had very good luck with Racket ( https://racket-lang.org/ https://racket-lang.org/ ), in that regard. It's a Lisp which is geared towards the implementation of languages and their interoperation. There's even a strongly typed Haskell-like language - Turnstyle - implemented in it as part of ( https://www.ccs.neu.edu/home/stchang/popl2017/ https://www.ccs.neu.edu/home/stchang/popl2017/ - the paper: https://www.ccs.neu.edu/home/stchang/pubs/ckg-popl2017.pdf https://www.ccs.neu.edu/home/stchang/pubs/ckg-popl2017.pdf ) and the even more powerful ( http://lambda-the-ultimate.org/node/5587 http://lambda-the-ultimate.org/node/5587 ( the paper: https://www.ccs.neu.edu/home/stchang/pubs/cbtb-popl2020.pdf https://www.ccs.neu.edu/home/stchang/pubs/cbtb-popl2020.pdf ) ) Turnstyle+ which implements dependent types on top of that language as well. Which is just plain cool...
mike_hearn 4y ago
That's what GraalVM is. Created by a corporation of course, but it's open source. GraalVM has Truffle, which is an API for creating languages. If you use it, you get a lot of features for "free". All you have to do is write an AST interpreter. I wrote about it a long time ago here: https://blog.plan99.net/graal-truffle-134d8f28fb69 https://blog.plan99.net/graal-truffle-134d8f28fb69 Misc things you get: • Advanced JIT compiler with support for static and dynamic typing, multiple CPU archs. • Advanced multi-threaded portable runtime. • Several cutting edge garbage collectors (you don't have to use them). • Native standalone single binary for your language's interpreter/jit. • Interop with other languages! (call into js, python, ruby, java, kotlin etc). • Can also call into WebAssembly and native code compiled with LLVM, your JIT will optimize across the boundary. • Can implement your language stdlib in any other language (see interop) meaning programs written in your language run on every major OS out of the box. • Profiling support. • Debugger support (chrome devtools). • Sandboxing. • Heap snapshotting. • Code coverage support. • Some Language Server Protocol support. Probably more I forgot about already. It's really a pretty amazing generalization of the JVM and drops the cost of creating competitive new languages through the floor. IDE integration is the weakest part, but as you're writing your AST interp in Java, making a converter to the IntelliJ PSI AST API isn't much more work so an IntelliJ plugin would be easy. And IntelliJ is itself open source so you can fork it to make a dedicated language IDE if you want.

shadowgovt 4y ago

I don't think it's a platform problem... It's a money problem. I've got a toy language I've built at home for my own reasons. Lots of developers do. The difference (especially in these modern times) between my toy and Rob Pike's toy is that Rob is paid by Google to develop it, hammer out the details, work out the bugs, and support it. As are the whole team Google dedicated to it. Some languages get supported by volunteer armies (Lua comes to mind, last I checked), but TypeScript, Go, and Rust are supported / maintained by corporate money and it's hard to compete with that because a corporation can throw money at a language that isn't done yet while it's harder to get volunteers excited about a broken toolset.

nandalism 4y ago
I constantly hear that "we can't do it without corporations" and yet there are so many counter examples. Most of the corporations were built on free languages and ecosystems which existed before they did.
- deleted 4y ago
  [deleted]
- mattgreenrocks 4y ago
  There's been a shift away from vague distrust of corporations (think Slashdot) to full-on embrace. The fallout from that is we can't imagine platforms that don't require tens of millions of dollars to bootstrap, and we cling to the ones we have because they're so expensive. Too big to fail is not the hacker ideology I grew up with. Maybe that's gone now.
  - shadowgovt 4y ago
    There's very little in the open source space that competes with, for example, npm + TypeScript + VSCode for usability (where "usability" here runs the gamut from package / dependency management to code comprehension / documentation / refactoring to debugging). That kind of polished integration often costs money. In contrast, in the open source world we limped along with gcc + gdb for decades until Clang was able to force the issue (which was funded by Apple, Google, Microsoft, ARM, Sony, Intel, and AMD).
    - OkayPhysicist 4y ago
      That strongly depends on your definition of "usability". If it means appealing to the lowest common denominator, that's probably fair. Open source software tends to lean technical, because it's made by technical people for themselves. If it means being useful to the people who use it, then I don't think your argument holds water. NeoVim and Emacs at least fairly compete with VSCode, npm is not at all special in the world of package management (Julia's Pkg and Elixir's Hex come to mind as OSS superior offerings), and the world of programming languages is chock full of better languages. To directly counter your "npm + TypeScript + VSCode", I counter with "Hex + Elixir + NeoVim".
      - shadowgovt 4y ago
        > If it means being useful to the people who use it, then I don't think your argument holds water I recently worked on a project involving a couple Docker containers, one of which wraps a Postgres DB and the other extends it. Some of the features of the extension are broken because the Postgres maintainers changed something key in their Docker image (they disabled the DB listening on localhost) and the scripts the extension generates assume localhost is accessible. In the (volunteer, free) open source ecosystem, nobody is paid to have that be their problem to solve and so problems like that, to my observation, rarely get solved because these two groups are unrelated to each other and it's nobody's job to make sure "the whole offering" is easy to use. (And as an ardent and frequent user of Emacs, I simply must see the config file someone claims to have put together that competes with VSCode for managing TypeScript projects. That's a hard claim to swallow without evidence; I've tried, and it's extremely non-trivial. Emacs just pulled in eglot for LSP support... Not that one couldn't self-install it earlier, but every time you have to mutate a system to one-off your own solution you increase the maintenance burden of that system.) ... I'll have to carve out some time to look at Elixir. In my world, of course, "Doesn't have a tool to compile to JavaScript that has made it to v1.0 yet" is a major strike against it, but my problem domain is specific. ;)
- pwinnski 4y ago
  I'm not sure what you're hearing is "we can't do it without corporations" so much as it is "the ones sponsored by corporations are better." I mean, GIMP exists, but any one of a half-dozen commercial products built by fewer people in less time are easier to use and have better feature lists. It's not that it's not possible for GIMP to be best-in-class, it's that it isn't best-in-class. (And sure, dear reader, GIMP works perfectly well for you. Great!) Given more time, I'd love to explore the idea that "most of the corporations were built on free languages and ecosystems" given how many of the companies we're talking about today seem to have some roots in C, developed under corporate control. But definitions are everything in a statement like that, so it probably would take a lot of time to sift through and ultimately convince nobody either way.
mattgreenrocks 4y ago
Google's Go team is not that large AFAIK. It's probably my love affair with Lisp speaking, but I really think there is space for PLs to implement 80% of production-level features with 20% effort and improve slowly as they turn out to be blockers. There's also a branding thing: users who are put off by lack of corporate funding won't be working on the ground floor of building a PL anyway. That's fine. LLVM/JVM/WASM give leverage to both indie and corporate langs. More of those types of tools are great.
enneff 4y ago
The difference between your toy and Robert, Rob, and Ken’s toy is that yours was built for your own reasons and theirs was built for the purposes of software engineering at scale. Whatever your thoughts about Go suiting that purpose, that was the intent, and why they were supported by Google in doing it.

brabel 4y ago

Exactly. This is something I believe will happen in a few decades (progress in this are is very slow). The JVM showed that a runtime can host a multitude of languages, compile to multiplatform bytecode that require only a "small" runtime (the core of the JVM is pretty simple, the complexity is on the stuff on the edges, like the stdlib) to run on anything and can be ported easily to any architecture. WASM is the new incarnation of the idea and if the people behind its specification stop worrying about politics and start getting stuff done, I am almost sure it will become the new "universal" runtime. Usually, new programming languages have a huge mountain to climb in terms of libraries that are required for everything these days (encryption, multiple data formas like YAML, JSON, XML, TOML and even the old ones that are still widely used in niches, like ASN.1... HTTP, Database Drivers, logging, monitoring, graphics, mathematics, the list goes on and on... we don't really want to rewrite all the stuff that exists... but interop between languages is very hard and sometimes very inefficient unless it's based on a common set of primitives (like Java bytecode... WASM still needs a lot of features like GC and interface types to be a real alternative). Zig and other languages that have very good and easy interop with C work around this problem. But I think that the future will be a universal runtime, maybe WASM, maybe something else that doesn't exist yet (or that's not yet known to most of us).

Tozen 4y ago
> ...easy interop with C work around this problem... Think this is a good point. Languages like Vlang, Dlang, Nim, etc... Use their interop with C to their advantage. They are not starting out in the wilderness, but can build off of the already strong C ecosystem, then add unique advantages that their userbases want or prefer.
mike_hearn 4y ago
JVM has moved beyond bytecode now. See my sibling comment. Truffle takes the JVM to the next level - you don't have to compile your language to bytecode anymore, you write an interpreter for the source code (or a binary form if you have one already) and then the JVM turns that interpreter into a JIT compiler, giving it many other features and tools along the way. You aren't constrained by what Java bytecode can express.
4y ago

hgsgm 4y ago

Like RPC and IPC and SWIG? We have that.

justinjlynn 4y ago

I've had very good luck with Racket ( https://racket-lang.org/ https://racket-lang.org/ ), in that regard. It's a Lisp which is geared towards the implementation of languages and their interoperation. There's even a strongly typed Haskell-like language - Turnstyle - implemented in it as part of ( https://www.ccs.neu.edu/home/stchang/popl2017/ https://www.ccs.neu.edu/home/stchang/popl2017/ - the paper: https://www.ccs.neu.edu/home/stchang/pubs/ckg-popl2017.pdf https://www.ccs.neu.edu/home/stchang/pubs/ckg-popl2017.pdf ) and the even more powerful ( http://lambda-the-ultimate.org/node/5587 http://lambda-the-ultimate.org/node/5587 ( the paper: https://www.ccs.neu.edu/home/stchang/pubs/cbtb-popl2020.pdf https://www.ccs.neu.edu/home/stchang/pubs/cbtb-popl2020.pdf ) ) Turnstyle+ which implements dependent types on top of that language as well. Which is just plain cool...

mike_hearn 4y ago

That's what GraalVM is. Created by a corporation of course, but it's open source. GraalVM has Truffle, which is an API for creating languages. If you use it, you get a lot of features for "free". All you have to do is write an AST interpreter. I wrote about it a long time ago here: https://blog.plan99.net/graal-truffle-134d8f28fb69 https://blog.plan99.net/graal-truffle-134d8f28fb69 Misc things you get: • Advanced JIT compiler with support for static and dynamic typing, multiple CPU archs. • Advanced multi-threaded portable runtime. • Several cutting edge garbage collectors (you don't have to use them). • Native standalone single binary for your language's interpreter/jit. • Interop with other languages! (call into js, python, ruby, java, kotlin etc). • Can also call into WebAssembly and native code compiled with LLVM, your JIT will optimize across the boundary. • Can implement your language stdlib in any other language (see interop) meaning programs written in your language run on every major OS out of the box. • Profiling support. • Debugger support (chrome devtools). • Sandboxing. • Heap snapshotting. • Code coverage support. • Some Language Server Protocol support. Probably more I forgot about already. It's really a pretty amazing generalization of the JVM and drops the cost of creating competitive new languages through the floor. IDE integration is the weakest part, but as you're writing your AST interp in Java, making a converter to the IntelliJ PSI AST API isn't much more work so an IntelliJ plugin would be easy. And IntelliJ is itself open source so you can fork it to make a dedicated language IDE if you want.