5 ms·

Implementing your own language on the Parrot VM

44 points by Xichekolas 18y ago

icey 18y ago
In my opinion, if someone were to want to port Clojure to something other than the JVM, Parrot would be an amazing place to go. The idea of having Clojure + CPAN makes my pants tighter. I know that there is someone working on Clojure on .Net, but I don't think that really adds much vs the JVM; especially since it seems like some of the best libraries for the .Net stack have come from Java. I'm sure I'm at risk of sounding like yet another Clojure fanboy, but to me this really seems like one of the few times where I've seen an article and that's the first thing that popped into my head.
- old-gregg 18y ago
  In my opinion, anything on JVM isn't going to become a mainstream language because VMs refuse to share code between OS processes. .NET and Mono solve this with AOT-precompiled modules. Ruby and Python kind of solve this by heavily reusing C runtime/libraries, so at least there aren't 25 copies of printf() and qsort() wasting space in RAM. JVM doesn't solve this at all. JVM-based softwares act like pigs showing zero respect to the enrivonment they're in. They're good at "one program = one computer" tasks, i.e. only for server-side or perhaps developer's workstation scenarios. Flash/Air is even worse: not only it won't share anything, but it also brings in its own graphics, font rasterization, hot key bindings, scroll bar behavior, etc, making every software built on top of it look and feel like a poorly designed console game running on your PC under software emulator. This is why I never touch anything like JVM/AIR: I don't believe the cloud is "the answer" and I want my code to run on servers, desktops, netbooks, routers and cell phones.
  - icey 18y ago
    Help me understand your point.. You're saying Java will never become mainstream because it runs on a VM? Edit: The more I read your response the less it makes sense to me. You say that you hate VMs because they don't respect the OS, but then you go on to say you want to write code that will run everywhere... That's the point of running in a VM, no?
    - old-gregg 18y ago
      Thank you for proof-reading, I edited my post above: I guess I am complaining about implementation of JVM/AIR. Yes, I want my code to run everywhere, but I want to use more lightweight VMs for that, those that blend into native OS more easily, and code sharing between processes is a big part of it, i.e. if I want to be able to write my application as a module which can be shared between 20 processes without being copied around 20 times. Basically the only parts of VMs I like is the bytecode and garbage collection. JVM/AIR do a lot more than that.
      - deleted 18y ago
        [deleted]
      - andreyf 18y ago
        I want to use more lightweight VMs for that, those that blend into native OS more easily Well, it's a tradeoff, right? With the JVM, because of the separation from the OS, you have that write-once-run-anywhere interoperability?
    - Xichekolas 18y ago
      Not that I totally understand it either, but I think he was railing against the JVM in particular, not VMs in general.
  - nradov 18y ago
    I find it hard to believe that sharing libraries in memory makes a significant difference. Most libraries are small relative to RAM sizes. The only exception is on cell phones, which only run a few processes anyway.
    - old-gregg 18y ago
      You don't need to "believe", you just have to know it. CS is a discipline, not a church. As an exercise, get a full list of all your processes and do the math by replacing them by hypothetical JVM instances [one per process] and I suggest paying attention to "shared pool" size for each process too. I guarantee you'll be shocked by total RAM consumed by an "empty" OSX or Vista. Just imagine: every little tiny process, even simple background services, will have a full copy of your entire userspace layer in its address space. Moreover, with 12-core CPUs becoming a reality very soon, the issue becomes even more important. With proper code sharing on OS level 12 instances of a process should (theoretically) take not much more RAM than just one. But because of VM overhead you'll be essentially duplicating it 12 times. More code duplication also means more I/O hits on disk and system bus and CPU L1/L2 cache drain. This is why Java is for enterprise/web software only. It is simply retarded to build on it in a true multi-process environment. OS-level code sharing is hugely important. Your hard drive is full of code, gigabytes of it, and you can't just clone big chunks of it into every process who needs it. Without code sharing you'd have to wait a few more years for an iPhone. He-he, actually because of lack of it, you aren't reading this in a JVM-powered browser and probably never will.
      - nradov 18y ago
        Have you actually tested this on a complex system of multiple applications running or JVMs to measure the effects, or are you just guessing that it would be a problem? I don't dispute that sharing libraries across multiple processes would be a nice optimization, but for typical use cases it's way down on the priority list. Most Java applications are written to use multiple threads (rather than multiple processes), which all execute on the same JVM and share all memory. Web apps typically run in some sort of J2EE container and those can be configured to share libraries across multiple applications; the OS doesn't come into play. Even if you're talking about running multiple separate desktop applications those will mostly only have the J2SE runtime library in common; for Java 1.6 the rt.jar file is 42MB, and only a fraction of it even needs to be loaded into memory.
- MrRage 18y ago
  > I know that there is someone working on Clojure on .Net, but I don't think that really adds much vs the JVM The .NET CLR supports tail recursion, whereas the JVM doesn't. Don't think it's worth porting simply for that, though.
- andreyf 18y ago
  A lot of parts of Clojure seemed to be created specifically with the JVM in mind. Not sure if porting it is a good idea.
  - chancho 18y ago
    Can you give some examples? (Not doubting. I don't know much about Clojure and I'm curious.)
    - andreyf 18y ago
      My only experience with Clojure is reveiwing Stuart Halloway's books on it, so this is certainly not the most qualified opinion you'll get, but as others have mentioned, tail-call optimization (lack thereof in the JVM, that is) significantly affected the language. It's not possible on the JVM, so Clojure relies heavily on the 'recur' keyword, so that recursive code can be translated into non-recursive bytecode. Another language feature that could be ignored is the trampoline, which is made easy with the #(foo 'bar %1) notation, but can be ignored if you're not worrying about the JVM. Aside from that, I would be very interested in seeing a Parrot Scheme based on ideas from Arc and Clojure both.
  - celoyd 18y ago
    Yep. Clojure is designed around the JVM and libraries written in Java-the-language. If you ported it, by the time you took out the Java-y stuff and added things like TCO and call/cc, it would be a new lisp. And I think new mid-sized lisps are what we need. If Parrot brings a wave of lisps bigger than Scheme but smaller than CL, designed for Unix and the internet, competing with Clojure and Arc, everyone wins. If they can call each other's libraries, it'll be wonderful. We should be looking way beyond Clojure.
    - icey 18y ago
      Are you sure it is designed specifically around the JVM? Rich used to maintain side-by-side versions that ran on .Net and the JVM.
maximilian 18y ago
I'm planning a summer, "write my own language" project for fun, and I wonder how well the parrot compiler tools would be for this. I imagine its a nice way to get my language into an AST, even if I don't target the Parrot VM.
- Xichekolas 18y ago
  Yeah I'm doing the same 'summer project'. I was writing my own parser/lexer by hand, but vaguely remembered hearing that Parrot had hit 1.0, so figured it'd be a good platform to experiment on. No idea if it'll support everything I want to try, but it at least seems easy to get started.
  - maximilian 18y ago
    I tried doing that too, but writing a parser was a pain in the ass in C. I was going a little crazy trying.. I do a lot of numerical things for my masters, and I want to write a simple numerics language that is heavily JIT compiled to speed up the code. Most numerics is done in pretty tight loops (at least everything I see), so I'm hoping to get pretty good performance. Its also just fun to read about VMs and armed with a parser, it won't be as hard to target different ones and compare performance.
    - chancho 18y ago
      You should check out LLVM. It generates really fast code and has a JIT compiler. I don't know much about Parrot, but it seems heavily skewed toward making dynamic languages fast whereas I believe LLVM aims more toward static languages. Not that you couldn't implement any language on top of either, but take for example the fact that Apple's support of LLVM was (I think) in part so they could use it as a JIT compiler for OpenGL shaders in their software-fallback drivers. It has good support for cross-platform SIMD instructions and such, which will benefit numerical computation greatly. ------------- Also, why don't you both use something like flex and bison? Is writing a lexer/parser by hand still that much of a nerd rite-of-passage? That's the most banal part of implementing a language. You could spend your summer writing an optimizer instead.

44 points by Xichekolas 18y ago

icey 18y ago

In my opinion, if someone were to want to port Clojure to something other than the JVM, Parrot would be an amazing place to go. The idea of having Clojure + CPAN makes my pants tighter. I know that there is someone working on Clojure on .Net, but I don't think that really adds much vs the JVM; especially since it seems like some of the best libraries for the .Net stack have come from Java. I'm sure I'm at risk of sounding like yet another Clojure fanboy, but to me this really seems like one of the few times where I've seen an article and that's the first thing that popped into my head.

old-gregg 18y ago
In my opinion, anything on JVM isn't going to become a mainstream language because VMs refuse to share code between OS processes. .NET and Mono solve this with AOT-precompiled modules. Ruby and Python kind of solve this by heavily reusing C runtime/libraries, so at least there aren't 25 copies of printf() and qsort() wasting space in RAM. JVM doesn't solve this at all. JVM-based softwares act like pigs showing zero respect to the enrivonment they're in. They're good at "one program = one computer" tasks, i.e. only for server-side or perhaps developer's workstation scenarios. Flash/Air is even worse: not only it won't share anything, but it also brings in its own graphics, font rasterization, hot key bindings, scroll bar behavior, etc, making every software built on top of it look and feel like a poorly designed console game running on your PC under software emulator. This is why I never touch anything like JVM/AIR: I don't believe the cloud is "the answer" and I want my code to run on servers, desktops, netbooks, routers and cell phones.
- icey 18y ago
  Help me understand your point.. You're saying Java will never become mainstream because it runs on a VM? Edit: The more I read your response the less it makes sense to me. You say that you hate VMs because they don't respect the OS, but then you go on to say you want to write code that will run everywhere... That's the point of running in a VM, no?
  - old-gregg 18y ago
    Thank you for proof-reading, I edited my post above: I guess I am complaining about implementation of JVM/AIR. Yes, I want my code to run everywhere, but I want to use more lightweight VMs for that, those that blend into native OS more easily, and code sharing between processes is a big part of it, i.e. if I want to be able to write my application as a module which can be shared between 20 processes without being copied around 20 times. Basically the only parts of VMs I like is the bytecode and garbage collection. JVM/AIR do a lot more than that.
    - deleted 18y ago
      [deleted]
    - andreyf 18y ago
      I want to use more lightweight VMs for that, those that blend into native OS more easily Well, it's a tradeoff, right? With the JVM, because of the separation from the OS, you have that write-once-run-anywhere interoperability?
  - Xichekolas 18y ago
    Not that I totally understand it either, but I think he was railing against the JVM in particular, not VMs in general.
- nradov 18y ago
  I find it hard to believe that sharing libraries in memory makes a significant difference. Most libraries are small relative to RAM sizes. The only exception is on cell phones, which only run a few processes anyway.
  - old-gregg 18y ago
    You don't need to "believe", you just have to know it. CS is a discipline, not a church. As an exercise, get a full list of all your processes and do the math by replacing them by hypothetical JVM instances [one per process] and I suggest paying attention to "shared pool" size for each process too. I guarantee you'll be shocked by total RAM consumed by an "empty" OSX or Vista. Just imagine: every little tiny process, even simple background services, will have a full copy of your entire userspace layer in its address space. Moreover, with 12-core CPUs becoming a reality very soon, the issue becomes even more important. With proper code sharing on OS level 12 instances of a process should (theoretically) take not much more RAM than just one. But because of VM overhead you'll be essentially duplicating it 12 times. More code duplication also means more I/O hits on disk and system bus and CPU L1/L2 cache drain. This is why Java is for enterprise/web software only. It is simply retarded to build on it in a true multi-process environment. OS-level code sharing is hugely important. Your hard drive is full of code, gigabytes of it, and you can't just clone big chunks of it into every process who needs it. Without code sharing you'd have to wait a few more years for an iPhone. He-he, actually because of lack of it, you aren't reading this in a JVM-powered browser and probably never will.
    - nradov 18y ago
      Have you actually tested this on a complex system of multiple applications running or JVMs to measure the effects, or are you just guessing that it would be a problem? I don't dispute that sharing libraries across multiple processes would be a nice optimization, but for typical use cases it's way down on the priority list. Most Java applications are written to use multiple threads (rather than multiple processes), which all execute on the same JVM and share all memory. Web apps typically run in some sort of J2EE container and those can be configured to share libraries across multiple applications; the OS doesn't come into play. Even if you're talking about running multiple separate desktop applications those will mostly only have the J2SE runtime library in common; for Java 1.6 the rt.jar file is 42MB, and only a fraction of it even needs to be loaded into memory.
MrRage 18y ago
> I know that there is someone working on Clojure on .Net, but I don't think that really adds much vs the JVM The .NET CLR supports tail recursion, whereas the JVM doesn't. Don't think it's worth porting simply for that, though.
andreyf 18y ago
A lot of parts of Clojure seemed to be created specifically with the JVM in mind. Not sure if porting it is a good idea.
- chancho 18y ago
  Can you give some examples? (Not doubting. I don't know much about Clojure and I'm curious.)
  - andreyf 18y ago
    My only experience with Clojure is reveiwing Stuart Halloway's books on it, so this is certainly not the most qualified opinion you'll get, but as others have mentioned, tail-call optimization (lack thereof in the JVM, that is) significantly affected the language. It's not possible on the JVM, so Clojure relies heavily on the 'recur' keyword, so that recursive code can be translated into non-recursive bytecode. Another language feature that could be ignored is the trampoline, which is made easy with the #(foo 'bar %1) notation, but can be ignored if you're not worrying about the JVM. Aside from that, I would be very interested in seeing a Parrot Scheme based on ideas from Arc and Clojure both.
- celoyd 18y ago
  Yep. Clojure is designed around the JVM and libraries written in Java-the-language. If you ported it, by the time you took out the Java-y stuff and added things like TCO and call/cc, it would be a new lisp. And I think new mid-sized lisps are what we need. If Parrot brings a wave of lisps bigger than Scheme but smaller than CL, designed for Unix and the internet, competing with Clojure and Arc, everyone wins. If they can call each other's libraries, it'll be wonderful. We should be looking way beyond Clojure.
  - icey 18y ago
    Are you sure it is designed specifically around the JVM? Rich used to maintain side-by-side versions that ran on .Net and the JVM.

maximilian 18y ago

I'm planning a summer, "write my own language" project for fun, and I wonder how well the parrot compiler tools would be for this. I imagine its a nice way to get my language into an AST, even if I don't target the Parrot VM.

Xichekolas 18y ago
Yeah I'm doing the same 'summer project'. I was writing my own parser/lexer by hand, but vaguely remembered hearing that Parrot had hit 1.0, so figured it'd be a good platform to experiment on. No idea if it'll support everything I want to try, but it at least seems easy to get started.
- maximilian 18y ago
  I tried doing that too, but writing a parser was a pain in the ass in C. I was going a little crazy trying.. I do a lot of numerical things for my masters, and I want to write a simple numerics language that is heavily JIT compiled to speed up the code. Most numerics is done in pretty tight loops (at least everything I see), so I'm hoping to get pretty good performance. Its also just fun to read about VMs and armed with a parser, it won't be as hard to target different ones and compare performance.
  - chancho 18y ago
    You should check out LLVM. It generates really fast code and has a JIT compiler. I don't know much about Parrot, but it seems heavily skewed toward making dynamic languages fast whereas I believe LLVM aims more toward static languages. Not that you couldn't implement any language on top of either, but take for example the fact that Apple's support of LLVM was (I think) in part so they could use it as a JIT compiler for OpenGL shaders in their software-fallback drivers. It has good support for cross-platform SIMD instructions and such, which will benefit numerical computation greatly. ------------- Also, why don't you both use something like flex and bison? Is writing a lexer/parser by hand still that much of a nerd rite-of-passage? That's the most banal part of implementing a language. You could spend your summer writing an optimizer instead.