10 ms·

VOC: A transpiler that converts Python bytecode into Java bytecode

121 points by Lofkin 11y ago

hristov 11y ago
That is called a compiler.
- monochromatic 11y ago
  Eh, it isn't starting with source code.
  - poizan42 11y ago
    In the broadest sense a compiler is any program that translates between different representations af a program. In that sense one can also consider a decompiler as a compiler. Also we usually use the term compilation (or more specifically Ahead Of Time and Just In Time compilation) for when we translate bytecode into machinecode.
    - monochromatic 11y ago
      Well, sure. I'm just saying that calling this something more specific than "compiler" isn't wrong.
- poizan42 11y ago
  Transpilers are compilers... The term is usually used for a compiler which compiles between languages at about the same level of abstraction.
  - pjmlp 11y ago
    A term that I only started seeing after the crazyness to compile to JavaScript started and a part of my major was compiler design.
  - cwp 11y ago
    The term is usually used by people that don't know what compilers are. Snark aside, I think this is actually the case. I bet there are a LOT of CoffeeScript/Javascript developers who have never worked in a compiled language.
    - deleted 11y ago
      [deleted]
drxzcl 11y ago
Did they really have to drag the Dutch east India company into this by using that logo? It's going to make things... complicated around here.
- TillE 11y ago
  That is a bit odd, but kudos to them for finding a new way to play with the word "Java". I thought they'd all been used up by the late 90s.
  - sspiff 11y ago
    Yeah, exactly. I was looking at the logo, and I thought "huh, that looks exactly like the Dutch East Indies company logo!" After ctrl-f'ing to the first mention of "Dutch", I had to chuckle at their explanation though. Besides, it's not like most technical product/project names are so great. I enjoyed this reference & their pun.
- DasIch 11y ago
  Don't really see why you got downvoted. It looks like they copied the logo exactly[1]. I'm rather surprised anyone would want to be associated with such an organization. [1]: https://en.wikipedia.org/wiki/Dutch_East_India_Company https://en.wikipedia.org/wiki/Dutch_East_India_Company
- raverbashing 11y ago
  Why would it make things complicated? I think the VOC has ceased operations a long time ago (as opposed to the Hudson's Bay company)
  - fijal 11y ago
    Because for a lot of people in a lot of countries VOC was an oppressive regime. Sure, times were different, but these days you would put them into some sort of crimes against humanity tribunal. They caused a lot of suffering, so in a lot of places using that logo is not politically correct at all.
pacala 11y ago
This can be very useful. I wonder what the plans are for supporting C apis, given that a lot of Python core libraries depend on C, in my neck of the woods numpy and scipy being the first contenders. There are long running projects http://www.jython.org http://www.jython.org and http://www.jyni.org http://www.jyni.org, which, to the best of my knowledge, have not yet managed to fully support Python C extensions on the JVM.
- masklinn 11y ago
  I'm guessing none given it doesn't look to support Python's stdlib in the first place. The project seems to be quite literally what it says on the tin: it compiles Python bytecode to Java bytecode.
- the8472 11y ago
  They could run the C extensions on the JVM too, like the experimental jruby truffle backend does. http://chrisseaton.com/rubytruffle/cext/ http://chrisseaton.com/rubytruffle/cext/
  - glibgil 11y ago
    A Python and Truffle implementation already exists https://bitbucket.org/ssllab/zippy https://bitbucket.org/ssllab/zippy
shuri 11y ago
performance?
- grhmc 11y ago
  question?
alexanderdaw 11y ago
Can someone give a valid reason for wanting this? I'm kind of confused as to why mixing and matching python and java source with some central compiler would be useful.
- DasIch 11y ago
  Just for fun? To take advantage of the Java ecosystem similarly to Jython?
fijal 11y ago
So well, one interesting question as to why this is at the top of hackernews is - do people not know about jython? Jython does exactly that - it compiles python into java (I think it emits bytecode and not Java source, but don't call me on that) with a lot of runtime so it can support proper semantics. It has been around for over a decade now too and it supports full python (with usual caveats)
- brudgers 11y ago
  One reason might be a development environment preference since developing directly in Jython would typically imply the overhead of the JVM. Another possible use case may be porting where a transpiler could address differences between implementations automatically that would otherwise have to be addressed more manually at the source code level. To put it another way, Jython makes obvious sense when the primary target is the JVM and there's little legacy code. As target platforms proliferate and the code base becomes cruftier, it may become a less obvious choice. I'm not sure the Pythonic "only one way" extends quite so far as to make Jython worthy of monopoly status.
- jestar_jokin 11y ago
  Jython is quite outdated and has stagnated for a long time; support for Python 2.7 was only finalised in May this year. Forget about support for v3.5. Perhaps by compiling from the bytecode of the existing interpreter, instead writing a whole new source code interpreter for the JVM, it will have some benefits in supporting newer versions of the language. Or, maybe it will help to decouple development of the standard library and the interpreter, possibly letting more devs work on it. You can't really be sure what benefits/deficiencies a different approach will bring until you actually try it - see PyPy, a "silly" idea to implement Python in Python, which now yields faster runtime performance than CPython.
  - deleted 11y ago
    [deleted]
  - techdragon 11y ago
    This is exactly why I spent a few hours trying to work out if I could perform some advanced trickery to plumb the pypy tool chain into something that would end up producing a pypy.jar or other JVM equivalent. Turns out it's not that easy. C to JVM isn't a well trodden path. Jython is awesome and all but its fighting an uphill war against rapid progress in 3 with far too few resources.
stuaxo 11y ago
Cool does this work on android ?
- stuaxo 11y ago
  [EDIT] - I should have wrote, can this generate code that will work on Android ? [/EDIT]
skrebbel 11y ago
> The Vereenigde Oostindische Compagnie (VOC), or Dutch East India Company, is often considered the be the world's first multinational corporation. ... Ah, trivia time! The VOC also had the world's first entirely privatized army. The "monopoly" cited in the README basically included a license to kill. Think "Google Armed Forces", but then scarier. The army was mostly used to keep a grumpy unpaid workforce "motivated", to keep the spice coming, and thus to keep that lovely 18% annual dividend payout reality. Basically, the VOC made current evil multinational corps (e.g. the oil companies, monsanto, blackwater, etc) look like cute cuddly charities. That said, it's been centuries, not sure getting worked up about the name makes sense now. It's a compiler, not a guidebook about how to traffic humans. I just thought the README section made the VOC seem a "little" awesomer than they were.
- throwawayaway 11y ago
  Britain had an East India Company first: https://en.wikipedia.org/wiki/East_India_Company_%28disambiguation%29 https://en.wikipedia.org/wiki/East_India_Company_%28disambig... http://www.newstatesman.com/politics/politics/2014/04/worlds-first-multinational http://www.newstatesman.com/politics/politics/2014/04/worlds... It's up for debate who was the first multinational. No doubt both set the bar very high for those to follow, but I do think blackwater spinoff dyncorp with child sex slaves will take some beating, particularly with respect to the prevailing moral climate of the time: https://en.wikipedia.org/wiki/DynCorp#Bosnia_incidents https://en.wikipedia.org/wiki/DynCorp#Bosnia_incidents
  - pjmlp 11y ago
    > Britain had an East India Company first: Actually the Portuguese one was founded in 1549, but I will leave out the usual type of "products" that were traded in those days.
- PythonicAlpha 11y ago
  I absolutely agree. The VOC was a very bad company. Basically, they took the blood of the native people in the spice countries and made money from it. The enslaved the people and killed many, when they did not cooperate. When the VOC is seen as basic example of corporations, it is a real gruesome heritage!
empyrical 11y ago
They also have an in-browser bytecode runner: https://github.com/pybee/batavia https://github.com/pybee/batavia
Animats 11y ago
How are the differences in the execution model handled? The defining quality of Python execution is that anything can monkey-patch anything at any time, using "setattr()". This means the obvious implementation has dictionaries for everything, and optimizers, as in PyPi have to work very, very hard. So what did they do here? Is every object a dictionary to the Java runtime? That's easy enough to do, but no gain over CPython. Did they weaken the ability to modify objects so they could map to Java objects? That gets a performance win, as with Shed Skin, but a lot of Python code won't work. They're probably not doing all the stuff PyPi does; that's hard and took a decade of work by many people.

121 points by Lofkin 11y ago

hristov 11y ago

That is called a compiler.

monochromatic 11y ago
Eh, it isn't starting with source code.
- poizan42 11y ago
  In the broadest sense a compiler is any program that translates between different representations af a program. In that sense one can also consider a decompiler as a compiler. Also we usually use the term compilation (or more specifically Ahead Of Time and Just In Time compilation) for when we translate bytecode into machinecode.
  - monochromatic 11y ago
    Well, sure. I'm just saying that calling this something more specific than "compiler" isn't wrong.
poizan42 11y ago
Transpilers are compilers... The term is usually used for a compiler which compiles between languages at about the same level of abstraction.
- pjmlp 11y ago
  A term that I only started seeing after the crazyness to compile to JavaScript started and a part of my major was compiler design.
- cwp 11y ago
  The term is usually used by people that don't know what compilers are. Snark aside, I think this is actually the case. I bet there are a LOT of CoffeeScript/Javascript developers who have never worked in a compiled language.
  - deleted 11y ago
    [deleted]

drxzcl 11y ago

Did they really have to drag the Dutch east India company into this by using that logo? It's going to make things... complicated around here.

TillE 11y ago
That is a bit odd, but kudos to them for finding a new way to play with the word "Java". I thought they'd all been used up by the late 90s.
- sspiff 11y ago
  Yeah, exactly. I was looking at the logo, and I thought "huh, that looks exactly like the Dutch East Indies company logo!" After ctrl-f'ing to the first mention of "Dutch", I had to chuckle at their explanation though. Besides, it's not like most technical product/project names are so great. I enjoyed this reference & their pun.
DasIch 11y ago
Don't really see why you got downvoted. It looks like they copied the logo exactly[1]. I'm rather surprised anyone would want to be associated with such an organization. [1]: https://en.wikipedia.org/wiki/Dutch_East_India_Company https://en.wikipedia.org/wiki/Dutch_East_India_Company
raverbashing 11y ago
Why would it make things complicated? I think the VOC has ceased operations a long time ago (as opposed to the Hudson's Bay company)
- fijal 11y ago
  Because for a lot of people in a lot of countries VOC was an oppressive regime. Sure, times were different, but these days you would put them into some sort of crimes against humanity tribunal. They caused a lot of suffering, so in a lot of places using that logo is not politically correct at all.

pacala 11y ago

This can be very useful. I wonder what the plans are for supporting C apis, given that a lot of Python core libraries depend on C, in my neck of the woods numpy and scipy being the first contenders. There are long running projects http://www.jython.org http://www.jython.org and http://www.jyni.org http://www.jyni.org, which, to the best of my knowledge, have not yet managed to fully support Python C extensions on the JVM.

masklinn 11y ago
I'm guessing none given it doesn't look to support Python's stdlib in the first place. The project seems to be quite literally what it says on the tin: it compiles Python bytecode to Java bytecode.
the8472 11y ago
They could run the C extensions on the JVM too, like the experimental jruby truffle backend does. http://chrisseaton.com/rubytruffle/cext/ http://chrisseaton.com/rubytruffle/cext/
- glibgil 11y ago
  A Python and Truffle implementation already exists https://bitbucket.org/ssllab/zippy https://bitbucket.org/ssllab/zippy

shuri 11y ago

performance?

grhmc 11y ago
question?

alexanderdaw 11y ago

Can someone give a valid reason for wanting this? I'm kind of confused as to why mixing and matching python and java source with some central compiler would be useful.

DasIch 11y ago
Just for fun? To take advantage of the Java ecosystem similarly to Jython?

fijal 11y ago

So well, one interesting question as to why this is at the top of hackernews is - do people not know about jython? Jython does exactly that - it compiles python into java (I think it emits bytecode and not Java source, but don't call me on that) with a lot of runtime so it can support proper semantics. It has been around for over a decade now too and it supports full python (with usual caveats)

brudgers 11y ago
One reason might be a development environment preference since developing directly in Jython would typically imply the overhead of the JVM. Another possible use case may be porting where a transpiler could address differences between implementations automatically that would otherwise have to be addressed more manually at the source code level. To put it another way, Jython makes obvious sense when the primary target is the JVM and there's little legacy code. As target platforms proliferate and the code base becomes cruftier, it may become a less obvious choice. I'm not sure the Pythonic "only one way" extends quite so far as to make Jython worthy of monopoly status.
jestar_jokin 11y ago
Jython is quite outdated and has stagnated for a long time; support for Python 2.7 was only finalised in May this year. Forget about support for v3.5. Perhaps by compiling from the bytecode of the existing interpreter, instead writing a whole new source code interpreter for the JVM, it will have some benefits in supporting newer versions of the language. Or, maybe it will help to decouple development of the standard library and the interpreter, possibly letting more devs work on it. You can't really be sure what benefits/deficiencies a different approach will bring until you actually try it - see PyPy, a "silly" idea to implement Python in Python, which now yields faster runtime performance than CPython.
- deleted 11y ago
  [deleted]
- techdragon 11y ago
  This is exactly why I spent a few hours trying to work out if I could perform some advanced trickery to plumb the pypy tool chain into something that would end up producing a pypy.jar or other JVM equivalent. Turns out it's not that easy. C to JVM isn't a well trodden path. Jython is awesome and all but its fighting an uphill war against rapid progress in 3 with far too few resources.

stuaxo 11y ago

Cool does this work on android ?

stuaxo 11y ago
[EDIT] - I should have wrote, can this generate code that will work on Android ? [/EDIT]

skrebbel 11y ago

> The Vereenigde Oostindische Compagnie (VOC), or Dutch East India Company, is often considered the be the world's first multinational corporation. ... Ah, trivia time! The VOC also had the world's first entirely privatized army. The "monopoly" cited in the README basically included a license to kill. Think "Google Armed Forces", but then scarier. The army was mostly used to keep a grumpy unpaid workforce "motivated", to keep the spice coming, and thus to keep that lovely 18% annual dividend payout reality. Basically, the VOC made current evil multinational corps (e.g. the oil companies, monsanto, blackwater, etc) look like cute cuddly charities. That said, it's been centuries, not sure getting worked up about the name makes sense now. It's a compiler, not a guidebook about how to traffic humans. I just thought the README section made the VOC seem a "little" awesomer than they were.

throwawayaway 11y ago
Britain had an East India Company first: https://en.wikipedia.org/wiki/East_India_Company_%28disambiguation%29 https://en.wikipedia.org/wiki/East_India_Company_%28disambig... http://www.newstatesman.com/politics/politics/2014/04/worlds-first-multinational http://www.newstatesman.com/politics/politics/2014/04/worlds... It's up for debate who was the first multinational. No doubt both set the bar very high for those to follow, but I do think blackwater spinoff dyncorp with child sex slaves will take some beating, particularly with respect to the prevailing moral climate of the time: https://en.wikipedia.org/wiki/DynCorp#Bosnia_incidents https://en.wikipedia.org/wiki/DynCorp#Bosnia_incidents
- pjmlp 11y ago
  > Britain had an East India Company first: Actually the Portuguese one was founded in 1549, but I will leave out the usual type of "products" that were traded in those days.
PythonicAlpha 11y ago
I absolutely agree. The VOC was a very bad company. Basically, they took the blood of the native people in the spice countries and made money from it. The enslaved the people and killed many, when they did not cooperate. When the VOC is seen as basic example of corporations, it is a real gruesome heritage!

empyrical 11y ago

They also have an in-browser bytecode runner: https://github.com/pybee/batavia https://github.com/pybee/batavia

Animats 11y ago

How are the differences in the execution model handled? The defining quality of Python execution is that anything can monkey-patch anything at any time, using "setattr()". This means the obvious implementation has dictionaries for everything, and optimizers, as in PyPi have to work very, very hard. So what did they do here? Is every object a dictionary to the Java runtime? That's easy enough to do, but no gain over CPython. Did they weaken the ability to modify objects so they could map to Java objects? That gets a performance win, as with Shed Skin, but a lot of Python code won't work. They're probably not doing all the stuff PyPi does; that's hard and took a decade of work by many people.