11 ms·

Mypyc: Compile type-annotated Python to C

234 points by mvolfik 6y ago

jd115 6y ago
This sounds awesome. What does it do?
- lvass 6y ago
  It compiles type-annotated Python to C
  - optimalsolver 6y ago
    Does the resulting code run as fast as native C?
    - sitkack 6y ago
      > Mypyc is a compiler that compiles mypy-annotated, statically typed Python modules into CPython C extensions. Currently our primary focus is on making mypy faster through compilation -- the default mypy wheels are compiled with mypyc. Compiled mypy is about 4x faster than without compilation. My wager is that it does not. It may if you have math intensive code, but if you have an algorithm that touches lots of python built in datatypes, access to those types will be the bottleneck.
rtpg 6y ago
Actually spent the evening trying to compile black through mypyc. The tooling is there (blacks setup.py has a thing) but most recent revisions of mypyc with black aren’t quite working for me The biggest issue right now seems to be miscompiles and the resulting errors being a bit inscrutable. It leaves you in the “am I wrong or is the system what’s wrong?” stuff a bit still. But overall I think the techniques are really sound and I believe this is the most promising way forward for perf in Python.
zedr 6y ago
Cython has a similar feature: https://cython.readthedocs.io/en/latest/src/tutorial/pure.html#pep-484-type-annotations https://cython.readthedocs.io/en/latest/src/tutorial/pure.ht...
- tomthe 6y ago
  Yes and it works. What is the difference between cython and mypyc? I think they should answer the question why anyone would want this over cython on the readme.
  - makeworld 6y ago
    > Note the use of cython.int rather than int - Cython does not translate an int annotation to a C integer by default since the behaviour can be quite different with respect to overflow and division. This seems like an important difference to me. Your regular type annotations can be used.
  - syastrov 6y ago
    Not having worked with cython, the difference seems to be that cython requires using special types in its annotations as well as not supporting specializing the standard types like ‘list’. Mypy aims to be compatible with the standard Python type annotations and still be able to optimize them. So in theory, you don’t need to modify your existing type-annotated program. In practice I believe there are limitations currently.
    - JesseMeyer 6y ago
      Cython has first class treatment for Numpy arrays. Can Mypyc generate machine optimized code for chomping Numpy arrays element-wise?
      - throwaway894345 6y ago
        I don’t think I want my toolchain to have first class knowledge of specific libraries...
dvh 6y ago
Murus
- gligorot 6y ago
  The downvotes probably came from non-slavic readers. I read it as Murus too haha.
syastrov 6y ago
As a bit of background info, mypyc is “not really” ready for broader use yet. The devs are planning a soft-launch: https://github.com/mypyc/mypyc/issues/780 https://github.com/mypyc/mypyc/issues/780 It is quite promising though, if it becomes more robust and compatible. I also believe they have still only scratched the surface of possible optimizations.
- mvolfik 6y ago
  Yes, this. Actually I first shared it here, because I thought that's cool and could work quite cleanly since mypy works well, but when I actually tried compiling one of my Advent of Code solutions with it, what i got was goto stuffed mess. I know I can't expect nice C code, but i certainly didn't expect gotos. As for the performance gain - 13.5 s with Python, 9 s compiled. It was a naive implementation of AoC 2020/23, so a lot of array cutting, concatenation etc. So this isn't really math, rather lot of RAM I/O
  - czardoz 6y ago
    There's nothing wrong with gotos in compiled code. At the end of the day, machine code is really just a bunch of gotos with other instructions in between. The reason goto is considered bad is that it can make code hard to follow for humans. Since this is an intermediate step in compilation, that's not an issue here.
polyrand 6y ago
I am always keeping and eye on mypyc, typed_python (llvm Python compiler)[0] and nuitka[1] I guess that because Python is extremely dynamic, we may never have a full everything-works compiler, but I’m excited about the possibility of this becoming some kind of intermediate step where different parts of the program get compiled when possible. [0] https://github.com/APrioriInvestments/typed_python https://github.com/APrioriInvestments/typed_python [1] https://github.com/Nuitka/Nuitka https://github.com/Nuitka/Nuitka
- CyberRabbi 6y ago
  I feel like it should be the agenda of the typed python syntax to allow writing annotated python code that can be compiled into a form that is as fast as equivalent c code.
- intrepidhero 6y ago
  Cython is also mentioned downthread. typed_python is new to me. I'll check it out. I'm too am keeping an eye on this space. I think that compiling or transpiling python may be the solution to both the major problems I have with python: performance and distribution. Exciting times.
- crazypython 6y ago
  Nuitka only removes interpreter overhead. (just 30%) It's still quite slow. To get real performance improvements, we'd need memory optimizations such as a modern JIT's hidden classes and shapes, which store data directly on the object, instead of inside a dictionary. https://mathiasbynens.be/notes/shapes-ics https://mathiasbynens.be/notes/shapes-ics
- anaphor 6y ago
  We can have a compiler that does everything. It's just a matter of whether you have to stick the python interpreter in the compiled binary or not, or how much of it you have to use and whether you can only use the parts required. This is how a lot of Scheme compilers work, even though you still have `eval` and similar things.
- dehrmann 6y ago
jagtesh 6y ago
It seems really interesting that the mypy team went to such lengths to create a binary version of their linter. The big draw with mypyc has got to be direct integration with other source code in C. Can anyone answer if it’s possible to replace PyPy’s VM backend with LLVM for AOT compilation? I wonder if that will results in any performance improvements.
asah 6y ago
IMHO it makes little sense to compile complete Python programs vs just compiling the slow parts. Some of the best reasons to choose Python are precisely the ones that preclude compilation, including: - "batteries included" including a massive set of libraries (any one of which won't be supported by the compiler) - dynamism which makes it easy to wrangle syntax to your needs (including the creation of domain-specific languages), but which destroys the performance improvement of compilation, even if the compiler can handle all the crazy introspection.
- Twirrim 6y ago
  This isn't about compiling an entire program, this is about compiling the individual libraries that you may be consuming, if they already have type hint coverage. A "free" performance boost. If I have a pure python, fully type hinted library I'm consuming, hats off to them, and they choose to use this, awesome.
- mlthoughts2018 6y ago
  I think this extends outside of Python. Performance and safety are trade offs, not absolutes, and the balance of needing safety or performance vs extensibility vs ease of development may result in dozens or hundreds of different trade off needs in different parts of a single application. One consequence is that it never makes sense to use static typing or compilation as application-wide absolutes for any language or paradigm. You should virtually never be writing whole applications in Rust, C, C++, Java, Haskell, etc. It is a huge sign of bad premature optimization and dogmatism. Compiling subsets in these languages and then exposing them through extension modules in other languages that don’t force those constant trade offs is almost always superior, and it’s very telling about poor engineering culture when this results in debates or vitriolic dismissiveness from people with prior dogmatic commitments to static typing or AOT compilation.
  - c-cube 6y ago
    That's a weird take from someone named MLthoughts :p But you seem to think everyone agrees that dynamic languages are more productive and that using (say) haskell is a trade-off for performance. For people used to (good) static type systems — for me that'd be OCaml — this is just not the case. Types do not impede, they help. I guess it might be a question of taste or habit, but don't make it a universal truth and accuse others of being biased when they disagree.
cabalamat 6y ago
> Classes are compiled into extension classes without __dict__ (much, but not quite, like if they used __slots__) Is there any way to say "no, a really want a __dict__ class here, please"?
- dragonwriter 6y ago
  > Is there any way to say "no, a really want a __dict__ class here, please"? Write it in a module you aren't compiling, and import it, since this supports compiled modules using noncompiled ones.
- Znafon 6y ago
  I think defining __dict__ explicitly should work.
optimalsolver 6y ago
Does the resulting code run as fast as native C? Would love to see some benchmarks on this.
- dragonwriter 6y ago
  > Does the resulting code run as fast as native C? The motivating use case is mypy, so I guess if someone wants to hand code mypy in native C we can assess this. But not doing that is as much, I would expect, of the motivation as speeding up mypy is.
bratao 6y ago
Here is one recent benchmark. Looks very promising. https://github.com/mypyc/mypyc-benchmark-results/blob/master/reports/summary-microbenchmarks.md https://github.com/mypyc/mypyc-benchmark-results/blob/master...
mikepurvis 6y ago
Somewhat related, I had a devil of a time a little bit ago trying to ship a small Python app as a fully standalone environment runnable on "any Linux" (but for practical purposes, Ubuntu 16.04, 18.04, and 20.04). It turns out that if you don't want to use pip, and you don't want to build separate bundles for different OSes and Python versions, it can be surprisingly tricky to get this right. Just bundling the whole interpreter doesn't work either because it's tied to a particular stdlib which is then linked to specific versions of a bunch of system dependencies, so if you go that route, you basically end up taking an entire rootfs/container with you. After evaluating a number of different solutions, I ended up being quite happy with pex: https://github.com/pantsbuild/pex https://github.com/pantsbuild/pex It basically bundles up the wheels for whatever your workspace needs, and then ships them in an archive with a bootstrap script that can recreate that environment on your target. But critically, it natively supports the idea of targeting multiple OS and Python versions, you just explicitly tell it which ones to include, eg: --platform=manylinux2014_x86_64-cp-38-cp38 # 16.04 --platform=manylinux2014_x86_64-cp-36-cp36m # 18.04 --platform=manylinux2014_x86_64-cp-35-cp35m # 20.04 Docs on this: https://pex.readthedocs.io/en/latest/buildingpex.html#platform https://pex.readthedocs.io/en/latest/buildingpex.html#platfo... And you can see the tags in use for any package on PyPI which ships compiled parts, eg: https://pypi.org/project/numpy/#files https://pypi.org/project/numpy/#files I don't know that this would be suitable for something like a game, but in my case for a small utility supporting a commercial product, it was perfect.
- jtdev 6y ago
  > “if you don't want to use pip” Why wouldn’t you want to use pip?
  - mikepurvis 6y ago
    Pip is suitable for use by developers working in python, setting up python workspaces with python sources and python dependencies, but it's a UX fiasco for an end-user who just wants to run a black box application and not have to care. In my particular case the "application" was in fact interactive bootstrap/install scripts for a large, proprietary blob which wouldn't have been suitable for publishing on PyPI, anyway. Setting up a separate, possibly authenticated PyPI instance, and then training end users how to use it, vs just shipping everything together in a single package? Total non-starter.
deleted 6y ago
[deleted]
fwsgonzo 6y ago
It would be a dream-come-true to be able to compile Python (or some kind of very-close Python) down to a static binary. I want to run it like a Go binary.
- colejohnson66 6y ago
  You already could?[0] Or are you asking about something else? [0]: https://stackoverflow.com/questions/39913847/is-there-a-way-to-compile-a-python-application-into-static-binary https://stackoverflow.com/questions/39913847/is-there-a-way-...
  - fwsgonzo 6y ago
    I looked into this and it seems like no, these are not static binaries at all. They dynamically load stuff, and cython seems to be embedding very specific headers, including linux specific ones (asm/errno.h). I tried to build using musl-gcc but it was too different. When I say build static like a Go binary, I mean that the binary contains everything, and is not allowed to dynamically load anything at all. Also, preferrably it doesn't need a C standard library, and does system calls manually.
    - zbentley 6y ago
      > When I say build static like a Go binary, I mean that the binary contains everything, and is not allowed to dynamically load anything at all. Just code? Or are you talking about data (e.g. HTML files) too? A lot of python's ecosystem seems pretty built around the assumption that non-code assets won't be bundled into single files, so any magic single-file Python compiler would probably have to include the ability to distribute those assets separately with the executable. > preferrably it doesn't need a C standard library I don't believe that golang even meets this standard--not with default options, at least. Also, why? How often have dynamically linked libc version differences really been a challenge for you in production? > does system calls manually This is generally considered to be a bad idea. Golang has moved away from this on several platforms, and people are making noise about doing it on Linux, too. A minority of compiled languages opt to go to this route. Why do you want this?
buescher 6y ago
CUDA is basically C with Fortran semantics, right? Wouldn't something like that be possible with Python?
- pjmlp 6y ago
  Nope, CUDA is a polyglot runtime for GPGPU programming, using PTX bytecode, with support for C, C++ and Fortran from NVidia, and third party support for .NET, Java, Haskell, Julia. Nowadays, CUDA hardware uses C++11 memory semantics, which are based on Java's memory model. This is one of the reasons why Krhonos insistence in keeping OpenCL a "C99 dialect on GPUs" has not gained the love of most researchers, and now is too late to win them back, despite SPIR.
  - buescher 6y ago
    Thanks - I think you got exactly what I meant, how out of date I am, and answered the real question.
Bostonian 6y ago
There is also Pyccel https://github.com/pyccel/pyccel https://github.com/pyccel/pyccel. When I last tried it, it worked on most small codes, but there were some bugs. "The aim of Pyccel is to provide a simple way to generate automatically, parallel low level code. The main uses would be: Convert a Python code (or project) into a Fortran or C code. Accelerate Python functions by converting them to Fortran or C functions. Pyccel can be viewed as: Python-to-Fortran/C converter a compiler for a Domain Specific Language with Python syntax"
chrisjharris 6y ago
I am slightly surprised about the general take on this I see here. I am definitely no python-hater and would take it a hundred times out of a hundred over, say, javascript, but as a language I feel like it is showing its age in a number of ways, and is becoming a little bloated as it tries to be all things to all people. What keeps me from ditching it is the best-in-class datascience ecosystem. Personally, I would prefer that the direction of travel be that I write less python, but keep the ecosystem. Whereas this project is facilitating writing python in other ecosystems.
- agucova 6y ago
  Have you tried the Julia language? With relatively performant Python interoperability it seems to fit your needs.
  - chrisjharris 6y ago
    Thanks, I have been meaning to check out Julia for some time and I think your comment may have finally spurred me into action.
    - eigenspace 6y ago
      Come say hi in https://julialang.zulipchat.com https://julialang.zulipchat.com or https://discourse.julialang.org/ https://discourse.julialang.org/, especially if you have questions about anything!

234 points by mvolfik 6y ago

jd115 6y ago

This sounds awesome. What does it do?

lvass 6y ago
It compiles type-annotated Python to C
- optimalsolver 6y ago
  Does the resulting code run as fast as native C?
  - sitkack 6y ago
    > Mypyc is a compiler that compiles mypy-annotated, statically typed Python modules into CPython C extensions. Currently our primary focus is on making mypy faster through compilation -- the default mypy wheels are compiled with mypyc. Compiled mypy is about 4x faster than without compilation. My wager is that it does not. It may if you have math intensive code, but if you have an algorithm that touches lots of python built in datatypes, access to those types will be the bottleneck.

rtpg 6y ago

Actually spent the evening trying to compile black through mypyc. The tooling is there (blacks setup.py has a thing) but most recent revisions of mypyc with black aren’t quite working for me The biggest issue right now seems to be miscompiles and the resulting errors being a bit inscrutable. It leaves you in the “am I wrong or is the system what’s wrong?” stuff a bit still. But overall I think the techniques are really sound and I believe this is the most promising way forward for perf in Python.

zedr 6y ago

Cython has a similar feature: https://cython.readthedocs.io/en/latest/src/tutorial/pure.html#pep-484-type-annotations https://cython.readthedocs.io/en/latest/src/tutorial/pure.ht...

tomthe 6y ago
Yes and it works. What is the difference between cython and mypyc? I think they should answer the question why anyone would want this over cython on the readme.
- makeworld 6y ago
  > Note the use of cython.int rather than int - Cython does not translate an int annotation to a C integer by default since the behaviour can be quite different with respect to overflow and division. This seems like an important difference to me. Your regular type annotations can be used.
- syastrov 6y ago
  Not having worked with cython, the difference seems to be that cython requires using special types in its annotations as well as not supporting specializing the standard types like ‘list’. Mypy aims to be compatible with the standard Python type annotations and still be able to optimize them. So in theory, you don’t need to modify your existing type-annotated program. In practice I believe there are limitations currently.
  - JesseMeyer 6y ago
    Cython has first class treatment for Numpy arrays. Can Mypyc generate machine optimized code for chomping Numpy arrays element-wise?
    - throwaway894345 6y ago
      I don’t think I want my toolchain to have first class knowledge of specific libraries...

dvh 6y ago

Murus

gligorot 6y ago
The downvotes probably came from non-slavic readers. I read it as Murus too haha.

syastrov 6y ago

As a bit of background info, mypyc is “not really” ready for broader use yet. The devs are planning a soft-launch: https://github.com/mypyc/mypyc/issues/780 https://github.com/mypyc/mypyc/issues/780 It is quite promising though, if it becomes more robust and compatible. I also believe they have still only scratched the surface of possible optimizations.

mvolfik 6y ago
Yes, this. Actually I first shared it here, because I thought that's cool and could work quite cleanly since mypy works well, but when I actually tried compiling one of my Advent of Code solutions with it, what i got was goto stuffed mess. I know I can't expect nice C code, but i certainly didn't expect gotos. As for the performance gain - 13.5 s with Python, 9 s compiled. It was a naive implementation of AoC 2020/23, so a lot of array cutting, concatenation etc. So this isn't really math, rather lot of RAM I/O
- czardoz 6y ago
  There's nothing wrong with gotos in compiled code. At the end of the day, machine code is really just a bunch of gotos with other instructions in between. The reason goto is considered bad is that it can make code hard to follow for humans. Since this is an intermediate step in compilation, that's not an issue here.

polyrand 6y ago

I am always keeping and eye on mypyc, typed_python (llvm Python compiler)[0] and nuitka[1] I guess that because Python is extremely dynamic, we may never have a full everything-works compiler, but I’m excited about the possibility of this becoming some kind of intermediate step where different parts of the program get compiled when possible. [0] https://github.com/APrioriInvestments/typed_python https://github.com/APrioriInvestments/typed_python [1] https://github.com/Nuitka/Nuitka https://github.com/Nuitka/Nuitka

CyberRabbi 6y ago
I feel like it should be the agenda of the typed python syntax to allow writing annotated python code that can be compiled into a form that is as fast as equivalent c code.
intrepidhero 6y ago
Cython is also mentioned downthread. typed_python is new to me. I'll check it out. I'm too am keeping an eye on this space. I think that compiling or transpiling python may be the solution to both the major problems I have with python: performance and distribution. Exciting times.
crazypython 6y ago
Nuitka only removes interpreter overhead. (just 30%) It's still quite slow. To get real performance improvements, we'd need memory optimizations such as a modern JIT's hidden classes and shapes, which store data directly on the object, instead of inside a dictionary. https://mathiasbynens.be/notes/shapes-ics https://mathiasbynens.be/notes/shapes-ics
anaphor 6y ago
We can have a compiler that does everything. It's just a matter of whether you have to stick the python interpreter in the compiled binary or not, or how much of it you have to use and whether you can only use the parts required. This is how a lot of Scheme compilers work, even though you still have `eval` and similar things.
dehrmann 6y ago

jagtesh 6y ago

It seems really interesting that the mypy team went to such lengths to create a binary version of their linter. The big draw with mypyc has got to be direct integration with other source code in C. Can anyone answer if it’s possible to replace PyPy’s VM backend with LLVM for AOT compilation? I wonder if that will results in any performance improvements.

asah 6y ago

IMHO it makes little sense to compile complete Python programs vs just compiling the slow parts. Some of the best reasons to choose Python are precisely the ones that preclude compilation, including: - "batteries included" including a massive set of libraries (any one of which won't be supported by the compiler) - dynamism which makes it easy to wrangle syntax to your needs (including the creation of domain-specific languages), but which destroys the performance improvement of compilation, even if the compiler can handle all the crazy introspection.

Twirrim 6y ago
This isn't about compiling an entire program, this is about compiling the individual libraries that you may be consuming, if they already have type hint coverage. A "free" performance boost. If I have a pure python, fully type hinted library I'm consuming, hats off to them, and they choose to use this, awesome.
mlthoughts2018 6y ago
I think this extends outside of Python. Performance and safety are trade offs, not absolutes, and the balance of needing safety or performance vs extensibility vs ease of development may result in dozens or hundreds of different trade off needs in different parts of a single application. One consequence is that it never makes sense to use static typing or compilation as application-wide absolutes for any language or paradigm. You should virtually never be writing whole applications in Rust, C, C++, Java, Haskell, etc. It is a huge sign of bad premature optimization and dogmatism. Compiling subsets in these languages and then exposing them through extension modules in other languages that don’t force those constant trade offs is almost always superior, and it’s very telling about poor engineering culture when this results in debates or vitriolic dismissiveness from people with prior dogmatic commitments to static typing or AOT compilation.
- c-cube 6y ago
  That's a weird take from someone named MLthoughts :p But you seem to think everyone agrees that dynamic languages are more productive and that using (say) haskell is a trade-off for performance. For people used to (good) static type systems — for me that'd be OCaml — this is just not the case. Types do not impede, they help. I guess it might be a question of taste or habit, but don't make it a universal truth and accuse others of being biased when they disagree.

cabalamat 6y ago

> Classes are compiled into extension classes without __dict__ (much, but not quite, like if they used __slots__) Is there any way to say "no, a really want a __dict__ class here, please"?

dragonwriter 6y ago
> Is there any way to say "no, a really want a __dict__ class here, please"? Write it in a module you aren't compiling, and import it, since this supports compiled modules using noncompiled ones.
Znafon 6y ago
I think defining __dict__ explicitly should work.

optimalsolver 6y ago

Does the resulting code run as fast as native C? Would love to see some benchmarks on this.

dragonwriter 6y ago
> Does the resulting code run as fast as native C? The motivating use case is mypy, so I guess if someone wants to hand code mypy in native C we can assess this. But not doing that is as much, I would expect, of the motivation as speeding up mypy is.

bratao 6y ago

Here is one recent benchmark. Looks very promising. https://github.com/mypyc/mypyc-benchmark-results/blob/master/reports/summary-microbenchmarks.md https://github.com/mypyc/mypyc-benchmark-results/blob/master...

mikepurvis 6y ago

Somewhat related, I had a devil of a time a little bit ago trying to ship a small Python app as a fully standalone environment runnable on "any Linux" (but for practical purposes, Ubuntu 16.04, 18.04, and 20.04). It turns out that if you don't want to use pip, and you don't want to build separate bundles for different OSes and Python versions, it can be surprisingly tricky to get this right. Just bundling the whole interpreter doesn't work either because it's tied to a particular stdlib which is then linked to specific versions of a bunch of system dependencies, so if you go that route, you basically end up taking an entire rootfs/container with you. After evaluating a number of different solutions, I ended up being quite happy with pex: https://github.com/pantsbuild/pex https://github.com/pantsbuild/pex It basically bundles up the wheels for whatever your workspace needs, and then ships them in an archive with a bootstrap script that can recreate that environment on your target. But critically, it natively supports the idea of targeting multiple OS and Python versions, you just explicitly tell it which ones to include, eg: --platform=manylinux2014_x86_64-cp-38-cp38 # 16.04 --platform=manylinux2014_x86_64-cp-36-cp36m # 18.04 --platform=manylinux2014_x86_64-cp-35-cp35m # 20.04 Docs on this: https://pex.readthedocs.io/en/latest/buildingpex.html#platform https://pex.readthedocs.io/en/latest/buildingpex.html#platfo... And you can see the tags in use for any package on PyPI which ships compiled parts, eg: https://pypi.org/project/numpy/#files https://pypi.org/project/numpy/#files I don't know that this would be suitable for something like a game, but in my case for a small utility supporting a commercial product, it was perfect.

jtdev 6y ago
> “if you don't want to use pip” Why wouldn’t you want to use pip?
- mikepurvis 6y ago
  Pip is suitable for use by developers working in python, setting up python workspaces with python sources and python dependencies, but it's a UX fiasco for an end-user who just wants to run a black box application and not have to care. In my particular case the "application" was in fact interactive bootstrap/install scripts for a large, proprietary blob which wouldn't have been suitable for publishing on PyPI, anyway. Setting up a separate, possibly authenticated PyPI instance, and then training end users how to use it, vs just shipping everything together in a single package? Total non-starter.

deleted 6y ago

[deleted]

fwsgonzo 6y ago

It would be a dream-come-true to be able to compile Python (or some kind of very-close Python) down to a static binary. I want to run it like a Go binary.

colejohnson66 6y ago
You already could?[0] Or are you asking about something else? [0]: https://stackoverflow.com/questions/39913847/is-there-a-way-to-compile-a-python-application-into-static-binary https://stackoverflow.com/questions/39913847/is-there-a-way-...
- fwsgonzo 6y ago
  I looked into this and it seems like no, these are not static binaries at all. They dynamically load stuff, and cython seems to be embedding very specific headers, including linux specific ones (asm/errno.h). I tried to build using musl-gcc but it was too different. When I say build static like a Go binary, I mean that the binary contains everything, and is not allowed to dynamically load anything at all. Also, preferrably it doesn't need a C standard library, and does system calls manually.
  - zbentley 6y ago
    > When I say build static like a Go binary, I mean that the binary contains everything, and is not allowed to dynamically load anything at all. Just code? Or are you talking about data (e.g. HTML files) too? A lot of python's ecosystem seems pretty built around the assumption that non-code assets won't be bundled into single files, so any magic single-file Python compiler would probably have to include the ability to distribute those assets separately with the executable. > preferrably it doesn't need a C standard library I don't believe that golang even meets this standard--not with default options, at least. Also, why? How often have dynamically linked libc version differences really been a challenge for you in production? > does system calls manually This is generally considered to be a bad idea. Golang has moved away from this on several platforms, and people are making noise about doing it on Linux, too. A minority of compiled languages opt to go to this route. Why do you want this?

buescher 6y ago

CUDA is basically C with Fortran semantics, right? Wouldn't something like that be possible with Python?

pjmlp 6y ago
Nope, CUDA is a polyglot runtime for GPGPU programming, using PTX bytecode, with support for C, C++ and Fortran from NVidia, and third party support for .NET, Java, Haskell, Julia. Nowadays, CUDA hardware uses C++11 memory semantics, which are based on Java's memory model. This is one of the reasons why Krhonos insistence in keeping OpenCL a "C99 dialect on GPUs" has not gained the love of most researchers, and now is too late to win them back, despite SPIR.
- buescher 6y ago
  Thanks - I think you got exactly what I meant, how out of date I am, and answered the real question.

Bostonian 6y ago

There is also Pyccel https://github.com/pyccel/pyccel https://github.com/pyccel/pyccel. When I last tried it, it worked on most small codes, but there were some bugs. "The aim of Pyccel is to provide a simple way to generate automatically, parallel low level code. The main uses would be: Convert a Python code (or project) into a Fortran or C code. Accelerate Python functions by converting them to Fortran or C functions. Pyccel can be viewed as: Python-to-Fortran/C converter a compiler for a Domain Specific Language with Python syntax"

chrisjharris 6y ago

I am slightly surprised about the general take on this I see here. I am definitely no python-hater and would take it a hundred times out of a hundred over, say, javascript, but as a language I feel like it is showing its age in a number of ways, and is becoming a little bloated as it tries to be all things to all people. What keeps me from ditching it is the best-in-class datascience ecosystem. Personally, I would prefer that the direction of travel be that I write less python, but keep the ecosystem. Whereas this project is facilitating writing python in other ecosystems.

agucova 6y ago
Have you tried the Julia language? With relatively performant Python interoperability it seems to fit your needs.
- chrisjharris 6y ago
  Thanks, I have been meaning to check out Julia for some time and I think your comment may have finally spurred me into action.
  - eigenspace 6y ago
    Come say hi in https://julialang.zulipchat.com https://julialang.zulipchat.com or https://discourse.julialang.org/ https://discourse.julialang.org/, especially if you have questions about anything!