6 ms·

The batch system call part is not so hard on its own: https://github.com/c-blake/batch https://github.com/c-blake/batch

by cb321 6y ago

ithkuil 6y ago
io_uring interface is designed to allow to encode arbitrary syscalls, although it seems that only a handful are actually supported
- pengaru 6y ago
  It's under active development, more syscalls keep getting added. Contributions are welcome.
  - ithkuil 6y ago
    I'd love a TL;DR explaination for why a blanket interface is not possible. I can guess that there are different ways that syscalls handle parameters and there are different families of syscall behaviours, but I'm not sure if that's the reason. I'd love a quick intro and pointers to more details.
    - cb321 6y ago
      You can look at my code in the link I gave. It's pretty short and should work on kernels 3.x series to 5.x series. { EDIT: I realize this may make you ask your question even more strongly. :-) }
    - pengaru 6y ago
      While I won't be writing a TL;DR summary, this isn't exactly unexplored territory, and it's been documented by lwn: https://lwn.net/Articles/316806/ https://lwn.net/Articles/316806/ https://lwn.net/Articles/221887/ https://lwn.net/Articles/221887/ https://lwn.net/Articles/219954/ https://lwn.net/Articles/219954/
      - cb321 6y ago
        Those articles (and syslets) all seem to have this strong async focus. That could simply reflect pengaru's bias/interest. So, I do not know for sure, but I suspect the TL;DR is "async & scheduling in the mix makes it hard to get right". That complexity may also relate to the missing syscalls. My approach has both the virtue and curse of being too simple to worry about all that, but it _does_ remove basic syscall overhead.
      - IAmLiterallyAB 6y ago
        Another interesting idea was FlexSC https://www.usenix.org/legacy/events/osdi10/tech/full_papers/Soares.pdf https://www.usenix.org/legacy/events/osdi10/tech/full_papers...
  - cb321 6y ago
    Well, it was not hard for me to add batching for every syscall. The whole module is under 200 lines of C. Granted, I only did x86_64. I think it has some bearing to those using eBPF to just batch calls, too. Unless I am missing something, I do not think there needs to be any super-user/root/capability restriction on syscall batching since all the syscalls check permission "on the inside". That gives it maybe more scope for applications. That sys_batch is kind of a tiny "jump-forward-only" assembly language where you can use the output of prior calls in later ones. The jump forward only (no loops) I do should also guarantee termination { at least conditioned upon all syscalls terminating...but that's a whole other domain ;-) }. (EDIT: IIRC, the article that this conversation is about was excited about this aspect. In my examples/ I have an "mmap a whole file in one syscall" example.)
    - pengaru 6y ago
      I don't personally find such naive syscall batching flexible enough to be all that useful. No ability to do such common things as concatenate strings from syscall results, or branch according to stat() output for example, makes for a severely limited interface. io_uring is already in mainline, and it delivers syscall batching as a side-effect while bringing async to the table. I just don't see the point in adding another, severely limited syscall batching thingy, certainly not now, even moreso with talk about ebpf logic joining the party. BTW as mentioned in a sibling comment, you might want to check out mingo's syslets proposition, which has similar naive batching, but was also async. https://lwn.net/Articles/221887/ https://lwn.net/Articles/221887/
      - cb321 6y ago
        It can branch based on the syscall return and the ability to copy word-values like file descriptors from open to an fstat and so on. That copying could perhaps be extended to strings in some limited way, but I agree it is generally much more limited. Not needing superuser/any special capability is also nice, though. Not sure of the current status/plans, but I am pretty sure eBPF needed root that for a very long time. Anyway, I was not trying to "compete" with you or try to "get into mainline". A module works fine for me. Was just exhibiting an easy possibility about some points discussed. EDIT: and thanks for the pointer. I will check it out. EDIT2: and much like the word copy is a fake syscall, other fake syscalls like a "value test" could be added to forge a sort of if condition jump forward thing. My little repo there is more a proof of concept than anything else.

The batch system call part is not so hard on its own: https://github.com/c-blake/batch https://github.com/c-blake/batch

by cb321 6y ago

The batch system call part is not so hard on its own: https://github.com/c-blake/batch https://github.com/c-blake/batch

io_uring interface is designed to allow to encode arbitrary syscalls, although it seems that only a handful are actually supported

pengaru 6y ago
It's under active development, more syscalls keep getting added. Contributions are welcome.
- ithkuil 6y ago
  I'd love a TL;DR explaination for why a blanket interface is not possible. I can guess that there are different ways that syscalls handle parameters and there are different families of syscall behaviours, but I'm not sure if that's the reason. I'd love a quick intro and pointers to more details.
  - cb321 6y ago
    You can look at my code in the link I gave. It's pretty short and should work on kernels 3.x series to 5.x series. { EDIT: I realize this may make you ask your question even more strongly. :-) }
  - pengaru 6y ago
    While I won't be writing a TL;DR summary, this isn't exactly unexplored territory, and it's been documented by lwn: https://lwn.net/Articles/316806/ https://lwn.net/Articles/316806/ https://lwn.net/Articles/221887/ https://lwn.net/Articles/221887/ https://lwn.net/Articles/219954/ https://lwn.net/Articles/219954/
    - cb321 6y ago
      Those articles (and syslets) all seem to have this strong async focus. That could simply reflect pengaru's bias/interest. So, I do not know for sure, but I suspect the TL;DR is "async & scheduling in the mix makes it hard to get right". That complexity may also relate to the missing syscalls. My approach has both the virtue and curse of being too simple to worry about all that, but it _does_ remove basic syscall overhead.
    - IAmLiterallyAB 6y ago
      Another interesting idea was FlexSC https://www.usenix.org/legacy/events/osdi10/tech/full_papers/Soares.pdf https://www.usenix.org/legacy/events/osdi10/tech/full_papers...
- cb321 6y ago
  Well, it was not hard for me to add batching for every syscall. The whole module is under 200 lines of C. Granted, I only did x86_64. I think it has some bearing to those using eBPF to just batch calls, too. Unless I am missing something, I do not think there needs to be any super-user/root/capability restriction on syscall batching since all the syscalls check permission "on the inside". That gives it maybe more scope for applications. That sys_batch is kind of a tiny "jump-forward-only" assembly language where you can use the output of prior calls in later ones. The jump forward only (no loops) I do should also guarantee termination { at least conditioned upon all syscalls terminating...but that's a whole other domain ;-) }. (EDIT: IIRC, the article that this conversation is about was excited about this aspect. In my examples/ I have an "mmap a whole file in one syscall" example.)
  - pengaru 6y ago
    I don't personally find such naive syscall batching flexible enough to be all that useful. No ability to do such common things as concatenate strings from syscall results, or branch according to stat() output for example, makes for a severely limited interface. io_uring is already in mainline, and it delivers syscall batching as a side-effect while bringing async to the table. I just don't see the point in adding another, severely limited syscall batching thingy, certainly not now, even moreso with talk about ebpf logic joining the party. BTW as mentioned in a sibling comment, you might want to check out mingo's syslets proposition, which has similar naive batching, but was also async. https://lwn.net/Articles/221887/ https://lwn.net/Articles/221887/
    - cb321 6y ago
      It can branch based on the syscall return and the ability to copy word-values like file descriptors from open to an fstat and so on. That copying could perhaps be extended to strings in some limited way, but I agree it is generally much more limited. Not needing superuser/any special capability is also nice, though. Not sure of the current status/plans, but I am pretty sure eBPF needed root that for a very long time. Anyway, I was not trying to "compete" with you or try to "get into mainline". A module works fine for me. Was just exhibiting an easy possibility about some points discussed. EDIT: and thanks for the pointer. I will check it out. EDIT2: and much like the word copy is a fake syscall, other fake syscalls like a "value test" could be added to forge a sort of if condition jump forward thing. My little repo there is more a proof of concept than anything else.