5 ms·

Just to clarify, this would only be a problem if the user had network extensions installed (and potentially lots of them, depending on implementation)? It could

by _rs 7y ago

Just to clarify, this would only be a problem if the user had network extensions installed (and potentially lots of them, depending on implementation)? It could have negligible impact if the video editing workstation didn't have these installed, if I'm reading this right?

monocasa 7y ago
I'm just saying there's a lot of use cases where people actually saturate their network connections on workstations, and you shouldn't discount them just because 'I'm not running mac as a switch'. But yes, network perf is needed only for workflows that involve large remote resources, and not to all video editing use cases out there.
- flatiron 7y ago
  You can still saturate your network connection. Just takes a bit of context switching to user land. So if you have a crazy core hyperthreaded cpu your 1 gig network connection will easily get filled without a blink. This is like the ssl argument at the beginning of encrypting everything. The world was going to end and then it didn’t
  - monocasa 7y ago
    Context switches can absolutely cut into maximum bandwidth and leave you unable to saturate a network.
    - Dylan16807 7y ago
      In certain machines with very weak CPUs and/or many very powerful connections. For a workstation, assuming mild levels of competence, there's no issue.
      - monocasa 7y ago
        No, on workstations and servers, particularly in a post spectre world, putting your network drivers into user space will absolutely destroy your perf because of the added context switches. You'd maybe have a point if it were an L4, but mach ports are used as an example now of how not to do microkernel IPC because of how much overhead they use.
        Dylan16807 7y ago
        A few thousand context switches per second is minor enough even with spectre mitigations, and if you need more than that you failed the "mild levels of competence" test.
        monocasa 7y ago
        Because those DPDK guys are just a bunch of clowns I guess, trying to avoid even the normal one user/kernel transition.
        Dylan16807 7y ago
        They have a completely different goal, much harder than merely saturating a single network port.
        monocasa 7y ago
        ..no, you fundamentally have a 1 to 1 relationship with a core/port with DPDK. And a lot of the use case is very much normal server style work loads, it's not just people running network switches with it.
        Dylan16807 7y ago
        According to https://blog.selectel.com/introduction-dpdk-architecture-principles/ https://blog.selectel.com/introduction-dpdk-architecture-pri... they are largely trying to avoid bottlenecks that exist inside the Linux kernel itself, bottlenecks that happen even with zero context switches. That's a totally different problem. Also to avoid having a system call per packet, which falls under "mild levels of competence" for an API designed this decade. Userspace networking also exists to eke out absolute minimum latency, which you don't need just to saturate a port. When your only goal is to avoid throughput bottlenecks, you don't need anything fancy. Avoid having a context switch per packet and you're most of the way there. A context switch every millisecond, or something in that order of magnitude, is completely harmless to throughput. If it causes your core to process 10% fewer packets than if it had zero context switches, then use 1.5 cores. Context switches take nothing anywhere near a millisecond each.

by _rs 7y ago

monocasa 7y ago

I'm just saying there's a lot of use cases where people actually saturate their network connections on workstations, and you shouldn't discount them just because 'I'm not running mac as a switch'. But yes, network perf is needed only for workflows that involve large remote resources, and not to all video editing use cases out there.

flatiron 7y ago
You can still saturate your network connection. Just takes a bit of context switching to user land. So if you have a crazy core hyperthreaded cpu your 1 gig network connection will easily get filled without a blink. This is like the ssl argument at the beginning of encrypting everything. The world was going to end and then it didn’t
- monocasa 7y ago
  Context switches can absolutely cut into maximum bandwidth and leave you unable to saturate a network.
  - Dylan16807 7y ago
    In certain machines with very weak CPUs and/or many very powerful connections. For a workstation, assuming mild levels of competence, there's no issue.
    - monocasa 7y ago
      No, on workstations and servers, particularly in a post spectre world, putting your network drivers into user space will absolutely destroy your perf because of the added context switches. You'd maybe have a point if it were an L4, but mach ports are used as an example now of how not to do microkernel IPC because of how much overhead they use.
      - Dylan16807 7y ago
        A few thousand context switches per second is minor enough even with spectre mitigations, and if you need more than that you failed the "mild levels of competence" test.
        monocasa 7y ago
        Because those DPDK guys are just a bunch of clowns I guess, trying to avoid even the normal one user/kernel transition.
        Dylan16807 7y ago
        They have a completely different goal, much harder than merely saturating a single network port.
        monocasa 7y ago
        ..no, you fundamentally have a 1 to 1 relationship with a core/port with DPDK. And a lot of the use case is very much normal server style work loads, it's not just people running network switches with it.
        Dylan16807 7y ago
        According to https://blog.selectel.com/introduction-dpdk-architecture-principles/ https://blog.selectel.com/introduction-dpdk-architecture-pri... they are largely trying to avoid bottlenecks that exist inside the Linux kernel itself, bottlenecks that happen even with zero context switches. That's a totally different problem. Also to avoid having a system call per packet, which falls under "mild levels of competence" for an API designed this decade. Userspace networking also exists to eke out absolute minimum latency, which you don't need just to saturate a port. When your only goal is to avoid throughput bottlenecks, you don't need anything fancy. Avoid having a context switch per packet and you're most of the way there. A context switch every millisecond, or something in that order of magnitude, is completely harmless to throughput. If it causes your core to process 10% fewer packets than if it had zero context switches, then use 1.5 cores. Context switches take nothing anywhere near a millisecond each.