4 ms·

Is there an equivalent for Node? I'm glad I haven't needed it since, but I couldn't find anything nearly so seamless when I last looked.

swyx 3y ago
why can't you just exec a python shell command to do the same thing? i'm actually slightly puzzled that "FFI" is its own thing when every language has a way to exec shell commands
- nathan_phoenix 3y ago
  You can if you are just executing a static python script never changes. But if you want to call some python functions with your TS input then you would need to serialize the TS input, pass it as an arg to the script, there deserialize it and then call the python function with the input. And if you wanted to call multiple python functions then your first arg to the python script would need to be the name of the function, only then the serialized data. Plus you would need to implement a switch case on the python side to execute the right function.
  - swyx 3y ago
    right. ok. thats not hard^? like at all? and in fact ive had multiple bugs that ultimately had to do with serialization being quietly abstracted over for me that if i had just handrolled it myself i probably wouldve been consciously aware of my bad choices. ^i know this is probably arrogance out of naivete, just teasing out what exactly i dont know that i dont know about what FFI does.
    - viraptor 3y ago
      There are limits to what the commandline arguments can take. And even if you fallback to files instead, you still need to serialise a potentially huge amount of data. (Wanna serialise a 1GB tree that is going to get only one lookup in each call to the script?) On top of that, you're missing any chance of in-memory caching, because the script call can't preserve that, while FFI could. Then there are things that are just not possible to serialise like pointers to mapped memory, file descriptors (outside of unix sockets), etc. Finally even if everything works, a command execution and parsing results is massively slower than a function call. So yeah, there's lots of restrictions on what you can practically do that way.
- djdev 3y ago
  deno_python exposes Python modules to JavaScript in a way that makes it possible access the API of any Python module out of the box, as if it were a JS API. Executing a Python shell command is quite different. Here, FFI is used to interface with the Python Interpreter C API. Argument for FFI against just executing shell commands is performance, efficiency, and that you can make calls from same process instead of spawning a subprocess. When doing [de]serialization over shell interface, a lot of things will be hit and miss. But for deno_python, type conversions are widely supported. You can even pass around objects by reference with zero copy, which would otherwise require copy over a shell interface.

swyx 3y ago

why can't you just exec a python shell command to do the same thing? i'm actually slightly puzzled that "FFI" is its own thing when every language has a way to exec shell commands

nathan_phoenix 3y ago
You can if you are just executing a static python script never changes. But if you want to call some python functions with your TS input then you would need to serialize the TS input, pass it as an arg to the script, there deserialize it and then call the python function with the input. And if you wanted to call multiple python functions then your first arg to the python script would need to be the name of the function, only then the serialized data. Plus you would need to implement a switch case on the python side to execute the right function.
- swyx 3y ago
  right. ok. thats not hard^? like at all? and in fact ive had multiple bugs that ultimately had to do with serialization being quietly abstracted over for me that if i had just handrolled it myself i probably wouldve been consciously aware of my bad choices. ^i know this is probably arrogance out of naivete, just teasing out what exactly i dont know that i dont know about what FFI does.
  - viraptor 3y ago
    There are limits to what the commandline arguments can take. And even if you fallback to files instead, you still need to serialise a potentially huge amount of data. (Wanna serialise a 1GB tree that is going to get only one lookup in each call to the script?) On top of that, you're missing any chance of in-memory caching, because the script call can't preserve that, while FFI could. Then there are things that are just not possible to serialise like pointers to mapped memory, file descriptors (outside of unix sockets), etc. Finally even if everything works, a command execution and parsing results is massively slower than a function call. So yeah, there's lots of restrictions on what you can practically do that way.
djdev 3y ago
deno_python exposes Python modules to JavaScript in a way that makes it possible access the API of any Python module out of the box, as if it were a JS API. Executing a Python shell command is quite different. Here, FFI is used to interface with the Python Interpreter C API. Argument for FFI against just executing shell commands is performance, efficiency, and that you can make calls from same process instead of spawning a subprocess. When doing [de]serialization over shell interface, a lot of things will be hit and miss. But for deno_python, type conversions are widely supported. You can even pass around objects by reference with zero copy, which would otherwise require copy over a shell interface.