3 ms·

What I would prefer to see happen in the Python community is for Python packages to embed APE binaries and launch them as subprocesses (ideally using posix_spaw

by jart 3y ago

What I would prefer to see happen in the Python community is for Python packages to embed APE binaries and launch them as subprocesses (ideally using posix_spawn) because then you can mostly just ship pure Python with a few APE programs for your native needs, and nothing about the Python packaging ecosystem needs to change. That was my dream when I worked on the TensorFlow team. One of the questions I asked myself is, if a MatMul kernel takes milliseconds to perform an op, and it takes microseconds to launch a subprocess, then why do I need to dynamically link a 50mb TensorFlow DSO when I can just whip up a simple C program to perform the very small amount of native math, system calls, and data crunching that I need for TensorBoard? Well, now with Cosmopolitan, it's finally possible. Just whip up the ~100 or so lines of C/C++ code that's impossible to do with Python into a small fat static APE binary, and vendor that program like any other .txt or .dat file asset. Then launch it when you need to, while never caring or needing to worry about shipping multiple .whl files or setting up Jenkins build farms for BSDs, Windows, etc. Life stays easy.

modeless 3y ago
I'm not sure I understand fully. Are you saying that instead of e.g. writing a neural net in Python using PyTorch (which loads several native extensions into the Python interpreter) I should write the whole neural net in C using a C neural net library, and ship it as an executable that my Python code can launch as a subprocess whenever it wants to run a forward or backward pass? Or are you saying that I can use PyTorch in Python but instead of loading a native extension into the Python process it should ship an executable that it launches every time it wants to run a compute kernel? Or something else?
- jart 3y ago
  Here's what I'm saying. Let's say you are a Python package author. You ship your app or library on PyPi to a lot of users. You have a problem that can be solved in 100 lines of C++, but it can't be solved using Python code. Python would just go too slow. That's a problem, because in order to introduce a single line of native code into your Python package, it would require assuming numerous burdens relating to compilation and distribution. Native code is hard. Native code being hard is the whole reason we use high-level languages like Python. It's hard because you'd have to setup a devops process to build and test your C code on all these different operating systems. So what I'm proposing that you consider instead, is compile the ~100 lines of C/C++ code you want as a ~100kb fat APE binary, on your one dev system of choice, using Cosmo. Then embed that APE executable inside your pure Python package, launch it as a subprocess, and never have to worry about devops and distribution toil.
  - modeless 3y ago
    I like that idea for a lot of applications (I also like the python "multiprocessing" library, other languages could learn a thing or two from it, and it really takes the sting out of the GIL). But I don't see it working well for something like numpy or PyTorch which are sharing arbitrary memory between Python and native and calling back and forth at a very high frequency. PyTorch and the neural net implementations built on it are the whole reason I'm using Python at all. If the answer was "build the native extensions you need for PyTorch/etc into the Python binary" that would be fine too, I have no love for dynamic library loading. But I don't see the PyTorch team going out of their way to change anything about their build process or runtime to work with Cosmopolitan, as much as I wish they would, so my guess is I won't be able to use Cosmopolitan Python for the foreseeable future.
chrismorgan 3y ago
> and it takes microseconds to launch a subprocess On my laptop (ASUS GA503QM, 5800HS), running /usr/bin/true to completion takes around 400μs, demonstrated by these three different approaches to spawning a thousand processes: $ python -m timeit -n 1000 --setup 'import subprocess' 'subprocess.run(["/usr/bin/true"])' 1000 loops, best of 5: 441 usec per loop $ rustc -o x - <<<'fn main() { for _ in 0..1000 { let _ = std::process::Command::new("/usr/bin/true").status(); } }' && time ./x ./x 0.33s user 0.07s system 99% cpu 0.402 total $ time (i=0; while [ $i -lt 1000 ]; do i=$(($i + 1)); /usr/bin/true; done) [elided] 0.66s user 0.43s system 100% cpu 1.088 total Historically spawning processes was vastly slower on Windows; I’m not certain of the current state though I think it improved a few years ago, but I believe it’s still considerably more than one millisecond; 10–20ms is the vague number that’s lodged in my memory, but not supported, and I can’t test it now. (I invite anyone running Windows to provide figures. You may well already have a true.exe somewhere, or there’s https://github.com/mlocati/true-for-windows https://github.com/mlocati/true-for-windows, or just write your own stub executable.) If you’re going to be using such calls often, spawning just one persistent process and using IPC for I/O is likely to be much faster.
- dataflow 3y ago
  > Historically spawning processes was vastly slower on Windows Indeed. > I’m not certain of the current state though I think it improved a few years ago It did? I feel like it gets slower and slower every time. > I believe it’s still considerably more than one millisecond; 10–20ms is the vague number that’s lodged in my memory, but not supported, and I can’t test it now. (I invite anyone running Windows to provide figures.) Yep. On the Windows 10 machine I have it's around 4ms, though that's antivirus and other similar sources of slowness disabled. You could probably get it closer to 2-3ms on a faster machine, but that's the ballpark.

by jart 3y ago

modeless 3y ago

I'm not sure I understand fully. Are you saying that instead of e.g. writing a neural net in Python using PyTorch (which loads several native extensions into the Python interpreter) I should write the whole neural net in C using a C neural net library, and ship it as an executable that my Python code can launch as a subprocess whenever it wants to run a forward or backward pass? Or are you saying that I can use PyTorch in Python but instead of loading a native extension into the Python process it should ship an executable that it launches every time it wants to run a compute kernel? Or something else?

jart 3y ago
Here's what I'm saying. Let's say you are a Python package author. You ship your app or library on PyPi to a lot of users. You have a problem that can be solved in 100 lines of C++, but it can't be solved using Python code. Python would just go too slow. That's a problem, because in order to introduce a single line of native code into your Python package, it would require assuming numerous burdens relating to compilation and distribution. Native code is hard. Native code being hard is the whole reason we use high-level languages like Python. It's hard because you'd have to setup a devops process to build and test your C code on all these different operating systems. So what I'm proposing that you consider instead, is compile the ~100 lines of C/C++ code you want as a ~100kb fat APE binary, on your one dev system of choice, using Cosmo. Then embed that APE executable inside your pure Python package, launch it as a subprocess, and never have to worry about devops and distribution toil.
- modeless 3y ago
  I like that idea for a lot of applications (I also like the python "multiprocessing" library, other languages could learn a thing or two from it, and it really takes the sting out of the GIL). But I don't see it working well for something like numpy or PyTorch which are sharing arbitrary memory between Python and native and calling back and forth at a very high frequency. PyTorch and the neural net implementations built on it are the whole reason I'm using Python at all. If the answer was "build the native extensions you need for PyTorch/etc into the Python binary" that would be fine too, I have no love for dynamic library loading. But I don't see the PyTorch team going out of their way to change anything about their build process or runtime to work with Cosmopolitan, as much as I wish they would, so my guess is I won't be able to use Cosmopolitan Python for the foreseeable future.

chrismorgan 3y ago

> and it takes microseconds to launch a subprocess On my laptop (ASUS GA503QM, 5800HS), running /usr/bin/true to completion takes around 400μs, demonstrated by these three different approaches to spawning a thousand processes: $ python -m timeit -n 1000 --setup 'import subprocess' 'subprocess.run(["/usr/bin/true"])' 1000 loops, best of 5: 441 usec per loop $ rustc -o x - <<<'fn main() { for _ in 0..1000 { let _ = std::process::Command::new("/usr/bin/true").status(); } }' && time ./x ./x 0.33s user 0.07s system 99% cpu 0.402 total $ time (i=0; while [ $i -lt 1000 ]; do i=$(($i + 1)); /usr/bin/true; done) [elided] 0.66s user 0.43s system 100% cpu 1.088 total Historically spawning processes was vastly slower on Windows; I’m not certain of the current state though I think it improved a few years ago, but I believe it’s still considerably more than one millisecond; 10–20ms is the vague number that’s lodged in my memory, but not supported, and I can’t test it now. (I invite anyone running Windows to provide figures. You may well already have a true.exe somewhere, or there’s https://github.com/mlocati/true-for-windows https://github.com/mlocati/true-for-windows, or just write your own stub executable.) If you’re going to be using such calls often, spawning just one persistent process and using IPC for I/O is likely to be much faster.

dataflow 3y ago
> Historically spawning processes was vastly slower on Windows Indeed. > I’m not certain of the current state though I think it improved a few years ago It did? I feel like it gets slower and slower every time. > I believe it’s still considerably more than one millisecond; 10–20ms is the vague number that’s lodged in my memory, but not supported, and I can’t test it now. (I invite anyone running Windows to provide figures.) Yep. On the Windows 10 machine I have it's around 4ms, though that's antivirus and other similar sources of slowness disabled. You could probably get it closer to 2-3ms on a faster machine, but that's the ballpark.