3 ms·

Did anybody tried to generalize IO_URING a bit further with something like language interoperability - platform specific swift/kotlin/objective-c/javascript wit

by matesz 5y ago

Did anybody tried to generalize IO_URING a bit further with something like language interoperability - platform specific swift/kotlin/objective-c/javascript with closer to the metal languages like c/rust/c++ for instance? Let's say we have 2 main threads, one for platform language (we can't change that) and one for native language. Both of these threads share submission and completion queue buffers - so let's say Swift thread could submit to Rust thread what would be better suited for Rust and Rust could submit to Swift what had to be done in Swift. Just like in IO_URING - application thread (Rust) submits what has to be done in kernel (Swift) and perhaps the other way around too. Same for Android and Web. Of course sharing memory could be done also on other data like UI elements tree etc. I tried to test that, but gave up once I was getting some weird out of memory errors in Xcode when tried to start threads from rust side of things. Xcode debugger could step into rust code amazingly though. I'm really just guessing and have pretty much no experience in something like language interop, but I can't stop thinking it would be a good way to go. So, do you think IO_URING type abstraction could be a good approach to doing cross-platform development in general?

eska 5y ago
Aren't you back at the actor model at that point, where each thread has a message queue and can push tasks/results to the other thread's queue? That is indeed getting popular.
- jstimpfle 5y ago
  In my limited experience trying to make the Actor Model work, with the AM there is the danger of running into OOP type problems (overemphasis on encapsulation and isolated implementations) which quickly results in an unmaintainable mess. I've watched some presentations of success stories employing the AM, however my hunch is that the use there was a lot more limited and controlled, basically with a close to 1:1 mapping of physical devices to actors. Which I'm not sure is what most people think the Actor Model is. The Actor Model, in my mind, is the idea of writing "synchronous"/"blocking" code for a number of agent implementations, and then getting that code to scale by scheduling as many of these agents as possible with some fiber or green threads magic. IO completion queues is the actual mechanism that allows distributed computation (where "distributed" here includes normal syscall interactions with the OS) without deadlocks or bad performance due to high latencies.

by matesz 5y ago

eska 5y ago

Aren't you back at the actor model at that point, where each thread has a message queue and can push tasks/results to the other thread's queue? That is indeed getting popular.

jstimpfle 5y ago
In my limited experience trying to make the Actor Model work, with the AM there is the danger of running into OOP type problems (overemphasis on encapsulation and isolated implementations) which quickly results in an unmaintainable mess. I've watched some presentations of success stories employing the AM, however my hunch is that the use there was a lot more limited and controlled, basically with a close to 1:1 mapping of physical devices to actors. Which I'm not sure is what most people think the Actor Model is. The Actor Model, in my mind, is the idea of writing "synchronous"/"blocking" code for a number of agent implementations, and then getting that code to scale by scheduling as many of these agents as possible with some fiber or green threads magic. IO completion queues is the actual mechanism that allows distributed computation (where "distributed" here includes normal syscall interactions with the OS) without deadlocks or bad performance due to high latencies.