4 ms·

Are there any limitations on what code can run? (discounting e.g. memory limitations and OS interaction) I'd love to read about the design process. I think the

by Y_Y 2y ago

Are there any limitations on what code can run? (discounting e.g. memory limitations and OS interaction) I'd love to read about the design process. I think the idea of taking bytecode aimed at the runtime of dynamic languages like Python or Ruby or even Lisp or Java and making custom processors for that is awesome and (recently) under-explored. I'd be very interested to know why you chose to stay this, why it was a good idea, and how you went about the implementation (in broad strokes if necessary).

hwpythonner 2y ago
Thanks — really appreciate the interest! There are definitely some limitations beyond just memory or OS interaction. Right now, PyXL supports a subset of real Python. Many features from CPython are not implemented yet — this early version is mainly to show that it's possible to run Python efficiently in hardware. I'd prefer to move forward based on clear use cases, rather than trying to reimplement everything blindly. Also, some features (like heavy runtime reflection, dynamic loading, etc.) would probably never be supported, at least not in the traditional way, because the focus is on embedded and real-time applications. As for the design process — I’d love to share more! I'm a bit overwhelmed at the moment preparing for PyCon, but I plan to post a more detailed blog post about the design and philosophy on my website after the conference.
- mikepurvis 2y ago
  In terms of a feature-set to target, would it make sense to be going after RPython instead of "real" Python? Doing that would let you leverage all the work that PyPy has done on separating what are the essential primitives required to make a Python vs what are the sugar and abstractions that make it familiar: https://doc.pypy.org/en/latest/faq.html#what-is-pypy https://doc.pypy.org/en/latest/faq.html#what-is-pypy
- ammar2 2y ago
  > I'd prefer to move forward based on clear use cases Taking the concrete example of the `struct` module as a use-case, I'm curious if you have a plan for it and similar modules. The tricky part of course is that it is implemented in C. Would you have to rewrite those stdlib modules in pure python?
  - mikepurvis 2y ago
    As in my sibling comment, pypy has already done all this work. CPython's struct module is just a shim importing the C implementations: https://github.com/python/cpython/blob/main/Lib/struct.py https://github.com/python/cpython/blob/main/Lib/struct.py Pypy's is a Python(-ish) implementation, leveraging primitives from its own rlib and pypy.interpreter spaces: https://github.com/pypy/pypy/blob/main/pypy/module/struct/interp_struct.py https://github.com/pypy/pypy/blob/main/pypy/module/struct/in... The Python stdlib has enormous surface area, and of course it's also a moving target.
    - ammar2 2y ago
      Aah, neat! Yeah, piggy-backing off pypy's work here would probably make the most sense. It'll also be interesting to see how OP deals with things like dictionaries and lists.
hermitShell 2y ago
JVM I think I can understand, but do you happen to know more about LISP machines and whether they use an ISA specifically optimized for the language, or if the compilers for x86 end up just doing the same thing? In general I think the practical result is that x86 is like democracy. It’s not always efficient but there are other factors that make it the best choice.
- kragen 2y ago
  They used an ISA specifically optimized for the language. At the time it was not known how to make compilers for Lisp that did an adequate job on normal hardware. The vast majority of computers in the world are not x86.
  - f1shy 2y ago
    Wait. It was pretty well known how to make compilers for Lisp, and they were not bad. There were some little parts of some lisps (number tower, overflow to bignum, rationals) which was problematic (and still is today, if you do not have custom HW). But those pieces were and are not that important for general purpose. The era of LISP isa was not so long after all.
    - kragen 2y ago
      The stock-hardware compilers for Lisp that were available in 01979 when Knight designed the CADR, like MACLISP, were pretty poor on anything but numerical code. When Gabriel's book https://archive.org/details/PerformanceAndEvaluationOfLispSystems https://archive.org/details/PerformanceAndEvaluationOfLispSy... came out in 01985, the year after he founded Lucid to fix that problem, InterLisp on the PDP-10 was 8× slower on Tak (2") than his handcoded assembly PDP-10 reference version (¼") (pp. 83, 86, 88, "On 2060 in INTERLISP (bc)"), while MacLisp on SAIL (another PDP-10, a KL-10) was only 2× slower (.564"), and the Symbolics 3600 he benchmarked it on was slightly faster (.43") than MacLisp but still 50% slower than the PDP-10 assembly code. No Lucid Common Lisp benchmarks were included. Unfortunately, most of Gabriel's Lisp benchmarks don't have hand-tuned assembly versions to compare them to. Generational garbage collection was first published (by Lieberman and Hewitt) in 01983, but wouldn't become widely used for several more years. This was a crucial breakthrough that enabled garbage collection to become performance-competitive with explicit malloc/free allocation, sometimes even faster. Arena-based or region-based allocation was always faster, and was sometimes used (it was a crucial part of GCC from the beginning in the form of "obstacks"), but Lisp doesn't really have a reasonable way to use custom allocators for part of a program. So I would claim that, until generational garbage collection, it was impossible for stock-hardware Lisp compilers to be performance-competitive on many tasks. Tak, however, doesn't cons, so that wasn't the slowness Gabriel observed in it. So I will make two slightly independent assertions here: 1. Stock-hardware Lisp compilers available in the late 01970s, when LispMs were built, were, in absolute terms, pretty poorly performing. The above evidence doesn't prove this, but I think it's at least substantial evidence for it. 2. Whether my assertion #1 above is actually true or not, certainly it was widely believed at the time, even by the hardest core of the Lisp community; and this provided much of the impetus for building Lisp machines. Current Lisp compilers like SBCL and Chez Scheme are enormous improvements on what was available at the time, and they are generally quite competitive with C, without any custom hardware. Specializing JIT compilers (whether Franz-style trace compilers like LuaJIT or not) could plausibly offer still better performance, but neither SBCL neither Chez uses that approach. SBCL does open-code fixnum arithmetic, and I think Chez does too, but they have to precede those operations with bailout checks unless declarations entitle them to be unsafe. Stalin does better still by using whole-program type inference. Some links: https://dl.acm.org/doi/pdf/10.1145/152739.152747 https://dl.acm.org/doi/pdf/10.1145/152739.152747 "'Infant Mortality' and Generational Garbage Collection", Baker (from 01993 I think) https://dspace.mit.edu/handle/1721.1/5718 https://dspace.mit.edu/handle/1721.1/5718 "CADR", AIM-528, by Knight, 01979-05-01 https://www.researchgate.net/publication/221213025_A_LISP_machine https://www.researchgate.net/publication/221213025_A_LISP_ma... "A LISP machine", supposedly 01980-04, ACM SIGIR Forum 15(2):137-138, doi 10.1145/647003.711869, The Papers of the Fifth Workshop on Computer Architecture for Non-Numeric Processing, Greenblatt, Knight, Holloway, and Moon, but it looks like what Knight uploaded to ResearchGate was actually a 14-page AI memo by Greenblatt https://news.ycombinator.com/item?id=27715043 https://news.ycombinator.com/item?id=27715043 previous discussion of a slide deck entitled "Architecture of Lisp Machines", the slides being of little interest themselves but the discussion including gumby, Mark Watson, et al.
checker659 2y ago
Forth CPU (in SystemVerilog): https://www.youtube.com/watch?v=DRtSSI_4dvk https://www.youtube.com/watch?v=DRtSSI_4dvk
bokchoi 2y ago
There were a few chips that supported directly executing JVM bytecodes. I'm not sure why it didn't take off, but I think it is generally more performant to JIT compile hotspots to native code. https://en.wikipedia.org/wiki/Java_processor https://en.wikipedia.org/wiki/Java_processor
- deleted 2y ago
  [deleted]
- teruakohatu 2y ago
  It did take off just in a different direction: https://en.m.wikipedia.org/wiki/Java_Card https://en.m.wikipedia.org/wiki/Java_Card To the point where most adult humans in the world probably own a Java-supported processor on a SIM card. Or at least an emulator (for eSIMs). On example of a CPU arch used on JavaCard devices is the ARM926EJ-S that I believe can execute Java byte code.
tsukikage 2y ago
Running bytecode directly on hardware has certainly been tried (e.g. ARM's Jazelle). In today's world this is generally not great. Interpreted languages often include bytecode instructions that actually do very complex things and so do not nicely map to operations that can be sanely implemented in hardware. So you end up with all the usual boring alu, branch etc operations implemented in hardware, and anything else traps and runs a software handler. Separately, interpreted language bytecode is often a poor fit for hardware execution; e.g. for dotnet (and python) bytecode many otherwise trivial operations do not explicitly encode information about types, and therefore the hardware must track type information in order to do the right thing (floating point addition looks very very different from integer addition!) A lot of effort has been spent on compiler optimisation for x86 and ARM code. JIT compilers benefit massively from this. Meanwhile, interpreted language bytecode is often very lightly optimised, where it is optimised at all (until relatively recently, explicit Python policy as set by Guido van Rossum was to never optimise!) Optimisation has the side effect of throwing away potentially valuable high level / semantic information; optimising at the bytecode level hinders debuggability for interpreted code (which is a primary goal in Python) and can also be detrimental to JIT output; and the results are underwhelming compared to JIT since your small team of plucky bytecode optimisers isn't really going to compete with decades of x86 compiler development; and so the incentive is to not do much of that. So if you're running bytecode in hardware, on top of all the obvious costs, you are /running unoptimised code/. This is actually the thing that kills these projects - everything else can ultimately be solved by throwing more silicon at it, but this can only really be solved by JITting, and the existing JIT+x86 / JIT+ARM solution is cheap and battle tested.
- f1shy 2y ago
  I understand that is the reason Lisp Machines were dropped (even in the time where Lisp was still a very good seen language). At least I understand so in the SICP videos, like in 1986 it was already clear it was much better to compile to ASM.

by Y_Y 2y ago

hwpythonner 2y ago

Thanks — really appreciate the interest! There are definitely some limitations beyond just memory or OS interaction. Right now, PyXL supports a subset of real Python. Many features from CPython are not implemented yet — this early version is mainly to show that it's possible to run Python efficiently in hardware. I'd prefer to move forward based on clear use cases, rather than trying to reimplement everything blindly. Also, some features (like heavy runtime reflection, dynamic loading, etc.) would probably never be supported, at least not in the traditional way, because the focus is on embedded and real-time applications. As for the design process — I’d love to share more! I'm a bit overwhelmed at the moment preparing for PyCon, but I plan to post a more detailed blog post about the design and philosophy on my website after the conference.

mikepurvis 2y ago
In terms of a feature-set to target, would it make sense to be going after RPython instead of "real" Python? Doing that would let you leverage all the work that PyPy has done on separating what are the essential primitives required to make a Python vs what are the sugar and abstractions that make it familiar: https://doc.pypy.org/en/latest/faq.html#what-is-pypy https://doc.pypy.org/en/latest/faq.html#what-is-pypy
ammar2 2y ago
> I'd prefer to move forward based on clear use cases Taking the concrete example of the `struct` module as a use-case, I'm curious if you have a plan for it and similar modules. The tricky part of course is that it is implemented in C. Would you have to rewrite those stdlib modules in pure python?
- mikepurvis 2y ago
  As in my sibling comment, pypy has already done all this work. CPython's struct module is just a shim importing the C implementations: https://github.com/python/cpython/blob/main/Lib/struct.py https://github.com/python/cpython/blob/main/Lib/struct.py Pypy's is a Python(-ish) implementation, leveraging primitives from its own rlib and pypy.interpreter spaces: https://github.com/pypy/pypy/blob/main/pypy/module/struct/interp_struct.py https://github.com/pypy/pypy/blob/main/pypy/module/struct/in... The Python stdlib has enormous surface area, and of course it's also a moving target.
  - ammar2 2y ago
    Aah, neat! Yeah, piggy-backing off pypy's work here would probably make the most sense. It'll also be interesting to see how OP deals with things like dictionaries and lists.

hermitShell 2y ago

JVM I think I can understand, but do you happen to know more about LISP machines and whether they use an ISA specifically optimized for the language, or if the compilers for x86 end up just doing the same thing? In general I think the practical result is that x86 is like democracy. It’s not always efficient but there are other factors that make it the best choice.

kragen 2y ago
They used an ISA specifically optimized for the language. At the time it was not known how to make compilers for Lisp that did an adequate job on normal hardware. The vast majority of computers in the world are not x86.
- f1shy 2y ago
  Wait. It was pretty well known how to make compilers for Lisp, and they were not bad. There were some little parts of some lisps (number tower, overflow to bignum, rationals) which was problematic (and still is today, if you do not have custom HW). But those pieces were and are not that important for general purpose. The era of LISP isa was not so long after all.
  - kragen 2y ago
    The stock-hardware compilers for Lisp that were available in 01979 when Knight designed the CADR, like MACLISP, were pretty poor on anything but numerical code. When Gabriel's book https://archive.org/details/PerformanceAndEvaluationOfLispSystems https://archive.org/details/PerformanceAndEvaluationOfLispSy... came out in 01985, the year after he founded Lucid to fix that problem, InterLisp on the PDP-10 was 8× slower on Tak (2") than his handcoded assembly PDP-10 reference version (¼") (pp. 83, 86, 88, "On 2060 in INTERLISP (bc)"), while MacLisp on SAIL (another PDP-10, a KL-10) was only 2× slower (.564"), and the Symbolics 3600 he benchmarked it on was slightly faster (.43") than MacLisp but still 50% slower than the PDP-10 assembly code. No Lucid Common Lisp benchmarks were included. Unfortunately, most of Gabriel's Lisp benchmarks don't have hand-tuned assembly versions to compare them to. Generational garbage collection was first published (by Lieberman and Hewitt) in 01983, but wouldn't become widely used for several more years. This was a crucial breakthrough that enabled garbage collection to become performance-competitive with explicit malloc/free allocation, sometimes even faster. Arena-based or region-based allocation was always faster, and was sometimes used (it was a crucial part of GCC from the beginning in the form of "obstacks"), but Lisp doesn't really have a reasonable way to use custom allocators for part of a program. So I would claim that, until generational garbage collection, it was impossible for stock-hardware Lisp compilers to be performance-competitive on many tasks. Tak, however, doesn't cons, so that wasn't the slowness Gabriel observed in it. So I will make two slightly independent assertions here: 1. Stock-hardware Lisp compilers available in the late 01970s, when LispMs were built, were, in absolute terms, pretty poorly performing. The above evidence doesn't prove this, but I think it's at least substantial evidence for it. 2. Whether my assertion #1 above is actually true or not, certainly it was widely believed at the time, even by the hardest core of the Lisp community; and this provided much of the impetus for building Lisp machines. Current Lisp compilers like SBCL and Chez Scheme are enormous improvements on what was available at the time, and they are generally quite competitive with C, without any custom hardware. Specializing JIT compilers (whether Franz-style trace compilers like LuaJIT or not) could plausibly offer still better performance, but neither SBCL neither Chez uses that approach. SBCL does open-code fixnum arithmetic, and I think Chez does too, but they have to precede those operations with bailout checks unless declarations entitle them to be unsafe. Stalin does better still by using whole-program type inference. Some links: https://dl.acm.org/doi/pdf/10.1145/152739.152747 https://dl.acm.org/doi/pdf/10.1145/152739.152747 "'Infant Mortality' and Generational Garbage Collection", Baker (from 01993 I think) https://dspace.mit.edu/handle/1721.1/5718 https://dspace.mit.edu/handle/1721.1/5718 "CADR", AIM-528, by Knight, 01979-05-01 https://www.researchgate.net/publication/221213025_A_LISP_machine https://www.researchgate.net/publication/221213025_A_LISP_ma... "A LISP machine", supposedly 01980-04, ACM SIGIR Forum 15(2):137-138, doi 10.1145/647003.711869, The Papers of the Fifth Workshop on Computer Architecture for Non-Numeric Processing, Greenblatt, Knight, Holloway, and Moon, but it looks like what Knight uploaded to ResearchGate was actually a 14-page AI memo by Greenblatt https://news.ycombinator.com/item?id=27715043 https://news.ycombinator.com/item?id=27715043 previous discussion of a slide deck entitled "Architecture of Lisp Machines", the slides being of little interest themselves but the discussion including gumby, Mark Watson, et al.

checker659 2y ago

Forth CPU (in SystemVerilog): https://www.youtube.com/watch?v=DRtSSI_4dvk https://www.youtube.com/watch?v=DRtSSI_4dvk

bokchoi 2y ago

There were a few chips that supported directly executing JVM bytecodes. I'm not sure why it didn't take off, but I think it is generally more performant to JIT compile hotspots to native code. https://en.wikipedia.org/wiki/Java_processor https://en.wikipedia.org/wiki/Java_processor

deleted 2y ago
[deleted]
teruakohatu 2y ago
It did take off just in a different direction: https://en.m.wikipedia.org/wiki/Java_Card https://en.m.wikipedia.org/wiki/Java_Card To the point where most adult humans in the world probably own a Java-supported processor on a SIM card. Or at least an emulator (for eSIMs). On example of a CPU arch used on JavaCard devices is the ARM926EJ-S that I believe can execute Java byte code.

tsukikage 2y ago

Running bytecode directly on hardware has certainly been tried (e.g. ARM's Jazelle). In today's world this is generally not great. Interpreted languages often include bytecode instructions that actually do very complex things and so do not nicely map to operations that can be sanely implemented in hardware. So you end up with all the usual boring alu, branch etc operations implemented in hardware, and anything else traps and runs a software handler. Separately, interpreted language bytecode is often a poor fit for hardware execution; e.g. for dotnet (and python) bytecode many otherwise trivial operations do not explicitly encode information about types, and therefore the hardware must track type information in order to do the right thing (floating point addition looks very very different from integer addition!) A lot of effort has been spent on compiler optimisation for x86 and ARM code. JIT compilers benefit massively from this. Meanwhile, interpreted language bytecode is often very lightly optimised, where it is optimised at all (until relatively recently, explicit Python policy as set by Guido van Rossum was to never optimise!) Optimisation has the side effect of throwing away potentially valuable high level / semantic information; optimising at the bytecode level hinders debuggability for interpreted code (which is a primary goal in Python) and can also be detrimental to JIT output; and the results are underwhelming compared to JIT since your small team of plucky bytecode optimisers isn't really going to compete with decades of x86 compiler development; and so the incentive is to not do much of that. So if you're running bytecode in hardware, on top of all the obvious costs, you are /running unoptimised code/. This is actually the thing that kills these projects - everything else can ultimately be solved by throwing more silicon at it, but this can only really be solved by JITting, and the existing JIT+x86 / JIT+ARM solution is cheap and battle tested.

f1shy 2y ago
I understand that is the reason Lisp Machines were dropped (even in the time where Lisp was still a very good seen language). At least I understand so in the SICP videos, like in 1986 it was already clear it was much better to compile to ASM.