5 ms·
This exercise appears to be somewhat flawed, even if entertaining/informative. Instead of evaluating the speed at which complex problems are resolved, it predom
by jagrsw 3y ago
This exercise appears to be somewhat flawed, even if entertaining/informative. Instead of evaluating the speed at which complex problems are resolved, it predominantly tests a peripheral issue: the efficiency of extracting memory from one process and transferring it to another. This allows for the illusion that the second process continues to write to a console/file, even though, technically, it does not - executing pv >/dev/null is essentially a no-op, as the write system call returns almost instantly.
vmsplice grants access to a process' buffer/memory to another process - a shared mem equivalent. As the initial competition requirements are likely vague, I'd imagine it's unclear if this is still good wrt the rules.
- LegionMammal978 3y ago> As the initial competition requirements are likely vague, I'd imagine it's unclear if this is still good wrt the rules. You can scroll upward to the original question to see the initial requirements, and check the edit history to verify that they haven't changed since the start of the challenge: > Write your fizz buzz program. Run it. Pipe the output through <your_program> | pv > /dev/null. The higher the throughput, the better you did. > The program output must be exactly valid fizzbuzz. No playing tricks such as writing null bytes in between the valid output - null bytes that don't show up in the console but do count towards pv throughput. And vmsplice(2) indeed produces a stream of bytes in the standard output pipe that pv(1) can splice into /dev/null, or cat(1) can copy into the terminal. This submission was not the only one that uses vmsplice(2); others have found that it's far from a magic bullet. Once you pass the I/O hurdle, much work remains in generating the pages of output as quickly as possible.
- tkluck 3y agoI disagree, because one does not get to the bottleneck being _the efficiency of extracting memory from one process and transferring it to another_ without major fizzbuzz-specific optimizations first. For example, there's a clever bit representation to get base-10 carries to happen natively. The initial competition requirements are not particularly vague about this point: Measuring throughput with `<program> | pv > /dev/null` is prescribed, and it also says > Architecture specific optimizations / assembly is also allowed. This is not a real contest - I just want to see how people push fizz buzz to its limit - even if it only works in special circumstances/platforms.
- dahart 3y ago> it predominantly tests a peripheral issue: the efficiency of extracting memory form one process and transferring it to another. Isn’t this almost always the whole problem? Most code is bottlenecked on memory and I/O. Complex problems are usually held up by the speed of getting data from one place to another, and not very often on computing the data. As someone who spends his days optimizing GPU assembly, even in the rare cases when compute is the bottleneck, once you optimize it, memory becomes the bottleneck.
- dahfizz 3y agoI/O is something that literally every program has to do. Its also the bottleneck of 99% of code running on modern hardware. Moving bytes from one place to another is essential and relatively slow. Understanding how to deal with memory I/O and file I/O performantly is a relevant skill for every program and programmer.