10 ms·
Show HN: Python-to-Python compiler for some 3.6 features in older versions
- sly010 9y agoImho this is a much better solution than 2to3. If python3 adoption is the goal, python3 should be the input and not the output.
- lloeki 9y agoThat's why 3to2 is a thing. https://wiki.python.org/moin/3to2 https://wiki.python.org/moin/3to2 https://pypi.python.org/pypi/3to2 https://pypi.python.org/pypi/3to2
- masklinn 9y ago> Imho this is a much better solution than 2to3. That's nonsensical since it solves a completely unrelated problem. 2to3 was conceived of as a one-shot migration tool for Python 2 codebases. Not as a way to build cross-version Python, and not as something to use repeatedly (which it why it is relatively simplistic and fallible), just as a way to take an existing codebase and handle the first 90% of migrating to Python 3, leaving you with the second 90%.
- nas 9y agoI don't think it is nonsensical. There was a tool 3to2 that took Python 3 code and produced Python 2 code. You had to write in a subset of Python 3 but the idea was sound, I think. Python 3 is more strict in ways than 2 and so downgrading the code was easier (e.g. no implicity str/unicode coercions). It looks like 3to2 is not maintained but I liked the idea. If I were writing a library that needed both 2 and 3 compatibility, I would prefer to write Python 3 code and then run 3to2 on it, rather than going the other way. My own solution is to fork the library, provide only bug fixes for the 2.x version and do all new development in Python 3 only. Porting has taken some effort but not as much as I feared. In retrospect, I'm happy to have invested the time.
- masklinn 9y ago> I don't think it is nonsensical. It is completely nonsensical in the context it is made in, the purpose of TFA has nothing to do with the purpose of 2to3, it makes no sense to "rank" them. > There was a tool 3to2 that took Python 3 code and produced Python 2 code. You had to write in a subset of Python 3 but the idea was sound, I think. Python 3 is more strict in ways than 2 and so downgrading the code was easier (e.g. no implicity str/unicode coercions). It looks like 3to2 is not maintained but I liked the idea. That still has nothing to do with 2to3, whose intended purpose once again is as a helper for a one-shot conversion of an existing Python 2 project to Python 3. > If I were writing a library that needed both 2 and 3 compatibility, I would prefer to write Python 3 code and then run 3to2 on it, rather than going the other way. And guess what? If you're starting from a Python 2 project you can use 2to3 to convert it to Python3 then use some other converter to provide P2 releases.
- sitkack 9y agoYou are not being charitable. 2to3 was _conceived_ of as migration tool, as if evolutionary lineages just all stopped, changed their genome and restarted. Except it only refactored some easy stuff and let the rest fall on the floor. Six has had way more impact than 2to3. It was only because core Python said running a single 2/3 codebase was UnPythonic that people didn't embrace it sooner. Running a single codebases across both 2 and 3 is now the defacto technique. Writing in 3 and "compiling down" to 2 gets people writing 3 which is the point, even if they can't run CPython3.
- masklinn 9y ago> You are not being charitable. There's nothing to be charitable about, GP's comment is rather clear and stands for itself. > 2to3 was _conceived_ of as migration tool And it still is one, so far as I know, which is actually part of the issues with it. > Except it only refactored some easy stuff and let the rest fall on the floor. I'm rather well aware of that, which you'd have noted if you'd actually read my comment. > Six has had way more impact than 2to3. […] Running a single codebases across both 2 and 3 is now the defacto technique. Sure, but again that has nothing whatsoever to do with the inanity of ranking 2to3 and backwards. > Writing in 3 and "compiling down" to 2 gets people writing 3 which is the point, even if they can't run CPython3. You do realise that's complementary to 2to3 right? A big issue of 2to3 is that once you've converted your P2 codebase to pure-3 (not a common subset of 2 and 3) you either have to maintain two different branches making packaging and backporting difficult or you just leave your P2 users out in the cold. Once again, the purpose of 2to3 is not to regularly run 2to3 to generate a Python3 release from a Python2 source.
- chatmasta 9y agoForward adoption is one thing the JS ecosystem does well (probably due to the sheer necessity of it given the many varying platforms outside control of the developer). Transpiling supersets of the language to the lowest supportable denominator is such an elegant approach. Thanks to projects like babel, typescript, webpack, etc, you can write your code in the most modern language spec, but target platforms not yet supporting it. I suspect this may be easier to achieve with JS than other languages because its prototypical nature makes it easy to compose dependencies. However, there is certainly a lot that other communities can learn from the JS community's solutions to this problem.
- orf 9y ago> is such an elegant approach In theory more than practice. For example if you use array destructuring (`[x, y] = something`) and Babel cannot verify that `something` is always an array it inserts a 664 character function that handles generators and other things. That adds performance and size overhead (grep for _slicedToArray in your compiled js files), when all you really want is `x = something[0]...` https://babeljs.io/repl/#?babili=false&evaluate=true&lineWrap=false&presets=es2015%2Creact%2Cstage-2&targets=&browsers=&builtIns=false&debug=false&code=function%20a(something)%7B%0A%20const%20%5Bx%2Cy%5D%20%3D%20something%3B%0A%7D%0A https://babeljs.io/repl/#?babili=false&evaluate=true&lineWra...
- goatlover 9y agoIt's a solution to a problem that exists because browsers incrementally support different features of the language, instead of upgrading all at once. That and the older browsers in use when you want to maintain backwards compatibility. The only reason the Python 2/3 issue exists is because of just enough breaking backwards compatibility in a way that a lot of people didn't agree with in the community. This isn't a thing in Ruby.
- skierscott 9y ago> If python3 adoption is the goal So the goal should be to have codebases in Python 3. > python3 should be the input and not the output How would this accomplish that goal? All the codebases would be in Python 2 then. I bet that removing barriers to run Python 2 code would cause it to stay around longer.
- dec0dedab0de 9y agoAll the code bases are already in 2. This would allow someone to write new internal libraries in 3.6, or migrate existing code one at a time, but still have the project as a whole run on a python2.7 until it is ready.
- nas 9y agoInteresting but I wonder when you would need a tool like that. I have something related, a fork of Python 3.6 that adds a bunch of backwards compatible behaviour so 2.7 code runs more easily, https://github.com/nascheme/ppython https://github.com/nascheme/ppython .
- sigjuice 9y agoI am not a Python developer, but thanks for not using the unnecessary term "transpiler".
- castis 9y agoBut isn't that what this is? I was under the impression that a compiler turns source code into machine code whereas a transpiler turns source code into differnet source code?
- daeken 9y agoAll compilers turn one form of code into another. Whether that's C to assembly (to machine code), or JavaScript to bytecode to machine code, or Python to Python. Transpiler is just a term referring to a compiler taking one high-level language to another. It's unnecessary because a Coffeescript to JavaScript compiler is a transpiler already; saying "transpiler" there tells you nothing you don't already know.
- omaranto 9y agoThe phrase "an A to B compiler" might not gain anything from changing compiler to transpiler, but the phrase "an A compiler" does.
- sigjuice 9y agoI would suggest reading the first couple of pages of any compiler book. e.g. https://books.google.com/books?id=_tgh4bgQ6PAC&printsec=frontcover&dq=engineering+a+compiler&hl=en&sa=X&ved=0ahUKEwiM3_aVgMXTAhVD0WMKHd1YDtkQ6AEIJjAA#v=onepage&q=engineering%20a%20compiler&f=false https://books.google.com/books?id=_tgh4bgQ6PAC&printsec=fron... Compilers are computer programs that translate a program written in one language into a program written in another language. Example of a compiler in action. $ cat hello.c #include <stdio.h> int main(void) { printf("hello world\n"); return 42; } $ cc -O3 -S hello.c -o - .section __TEXT,__text,regular,pure_instructions .macosx_version_min 10, 12 .globl _main .align 4, 0x90 _main: ## @main .cfi_startproc ## BB#0: pushq %rbp Ltmp0: .cfi_def_cfa_offset 16 Ltmp1: .cfi_offset %rbp, -16 movq %rsp, %rbp Ltmp2: .cfi_def_cfa_register %rbp leaq L_str(%rip), %rdi callq _puts movl $42, %eax popq %rbp retq .cfi_endproc .section __TEXT,__cstring,cstring_literals L_str: ## @str .asciz "hello world" .subsections_via_symbols
- wbond 9y agoEver since mypy started gaining attention, I’ve been looking for a way to write using Python 3 annotations and strip the annotations for release so that the code can run on Python 2. I never was able to find anyone talking about that goal with mypy, but it looks like this tool may be a solution.
- tyingq 9y agoGreat idea. A writeup on how you handled mapping bytes, bytearrays, unicode, etc, to 2.7 constructs would be interesting. In code that supports both 2 and 3, I end up with ugly stuff testing sys.hexversion to deal with 3rd party libraries, like pyserial, that expect str in v2, bytearrays in v3.
- benhoyt 9y agoI've scanned the source code and it doesn't look like it handles the bytes/unicode thing. Which, while the project is cool, will mean it won't work for a ton of Python 3 code.
- nas 9y agoThe implicit coercion between bytes and unicode was one of the biggest hurtles I ran into when porting a relatively large (30k+ lines) code base from Python 2 to 3. Also challenging was the change in comparisons (None no longer smaller than all types, TypeError when trying to order disjoint types). While this looks like an interesting hack, I can't see anyone seriously using it to backport code to Python 2.7. As I mentioned elsewhere, I created "ppython" to handle these two porting problems. The source code is on github: https://github.com/nascheme/ppython https://github.com/nascheme/ppython . If someone needs a Windows binary, I could build it if you ask politely. ;-) For my project, it has been a useful tool to speed up porting. Porting code is not trivial and there is huge amount of Python 2 out there. I wish more effort had been spent building tools to help porting of code. Even simple things like disallowing the 'u' prefix on strings in Python 3 was a big mistake. Mostly those things have been corrected but it is still going to take a very long time to move the majority of the community to Python 3. There will be Python 2 running for the next 50 years easily, I'm sure, maybe 100 years. At least Python is open source and you will not be stranded like VB developers where when MS drastically changed Visual Basic.
- sitkack 9y agoYou probably fixed lots of bugs in the process. How long did it take?
- dbcurtis 9y agoSerious question: Why would anyone still be doing anything with Python 2.7 except porting the last remnants of their code base to 3.X?
- gizmo385 9y agoBecause sometimes the amount of work required to port a 2.7 project to 3.x isn't worth it.
- dbcurtis 9y agoSo, then it is a small project that is near end-of-life, and easily replaced?
- richardwhiuk 9y agoNo, a large project, which isn't easily replaced, whether or not it's near the end of it's life, or anything using a Py2 only dependency
- compuguy 9y agoAnsible currently only supports Python 2.7....
- run4yourlives2 9y agoSerious Answer: Because some of us have some rather large perfectly working systems written in 2.7 and using lots of different libraries that moving to 3 is a major project requiring a good amount of time and effort. Not all of us have hobby-sized projects on the go.
- awinter-py 9y agohuzzah -- now that guy who complained python 3 wasn't turing complete can sleep at night
- jwilk 9y agoI'm confused with the way the "Supported features" section is divided. Does it mean I can only use f''-strings if the target version is 3.5? Why such a limitation?
- linux2647 9y agoI think it means that you tell the compiler what version of Python the original source code is in, so it knows how to compile it. So, if your codebase has f''-strings then the compiler needs to know. That's my guess, anyway.
- deleted 9y ago[deleted]
- zem 9y agoit means if your target is 3.5 those are the only features that need to be compiled to another form; the lower ones already work
- deleted 9y ago[deleted]
- dangayle 9y agoQuick! Someone inform Zed Shaw!
- pekk 9y agoPigs fly - he already has a draft of LPTHW for Python 3
- sametmax 9y agoPlease don't. The transpiler route ended very badly for JS. Let's not open this pandora box for Python.
- forgottenacc57 9y agoIf you need python 3 features that badly then you have made the case for upgrading your entire system to use the real python 3. Python 2 is near end if life anyway.