3 ms·
If I understand the question correctly, you are saying: 1. There's a bunch of objects that need to pass down the ffi boundaries py->go 2. compute 3. There's a
by jondot 10y ago
If I understand the question correctly, you are saying:
1. There's a bunch of objects that need to pass down the ffi boundaries py->go
2. compute
3. There's a bunch of objects that need to pass up the ffi boundaries go->py
4. Python now continues as usual with a bunch of processed objects
In that case, yes this would be a problem. The way I'd resolve it is by planning the ffi boundaries accordingly. I'd make python do as less as possible, and pass just declarative "instructions" to go. In this case where's the file location and where's the sourcemap file location (and perhaps where to dump output to if that's the case). And go doing as much work as possible to make sure there's only a minimal number of objects passed back if any.
It may _feel_ like a hack but ultimately its the same approach if you were to make a "sourcemap server" making python code communicate with it over RPC.
If this is not the problem then I'd love an example of what you meant
- the_mitsuhiko 10y ago> 1. There's a bunch of objects that need to pass down the ffi boundaries py->go 2. compute 3. There's a bunch of objects that need to pass up the ffi boundaries go->py 4. Python now continues as usual with a bunch of processed objects You can look at the library in question. An object gets created in Rust but the ownership of that object is held in Python. When the Python GC runs we clean up the Rust object. > It may _feel_ like a hack but ultimately its the same approach if you were to make a "sourcemap server" making python code communicate with it over RPC. Sure, but that significantly complicates the problem. To the point in fact where I question if the Go solution makes any sense at all because it takes away the advantage you have where you can just drop an extension module in without much work. Once you need to restructure your system to be message based you might as well go in and run a separate process and use a unix pipe to communicate. We used to do that for a few things like our debug symbol symbolication and the downsides are just too big.
- jondot 10y agoI understand. So now if I may backpaddle a bit, why go through the trouble of having python own the objects? why not let Rust (or Go) deal with the entire bulk of the job at hand?
- the_mitsuhiko 10y agoBecause that would require a huge changes to our codebase. We pass those objects around in various places already.