4 ms·

Of course this is just a toy example. But suppose you had a different task that is I/O bound, like processing terabytes of jsonlines files. The argument is rat

• sophacles 3y ago
  This isn't a very strong generalization. * Where do those terabytes of json live? In the example case, the data is not read from anywhere, but rather generated and written to the pipe. All else being equal, I suspect it would be hard to feed such a program at 55GB/s w/ real data from some storage somewhere - in the end you'd be limited by the disk read speed (or network recv speed). * This example doesn't need much ram for processing. There's only a handful of variables that can be held nicely in registers. Even if you could get 55GB/s of input, there aren't a lot of cases of json processing that won't need to store variables in memory. At a minimum a lot of effort would be required to ensure cache friendliness for these variables to not thrash the cache - if it's possible at all. * The code for the example is tiny - json parsing requires a lot more instructions. Even in the simplest case - pulling a fixed number of bytes from fixed size records in the input stream requires a similar number of adds and compares (etc) as the example case, and the output memory alignment machinery will be about the same. As the code grows to handle actual parsing of json and processing of data, the instruction count grows quickly to a size where the instruction cache may become a concern. * speaking of instructions - parsing json has a lot of conditionals. I don't think it's possible to do it without any conditionals, or even without enough branch mispredictions and pipeline flushes to achieve such high throughput. Of course all of this is assuming that the IO of writing to a pipe is the same as IO in the "get from network or disk etc" sense. They really aren't the same - io to a pipe is IPC, the data never leaves the l2 cache - it's only io to the process not to the CPU. Once you have disks and networks and RAM involved, things change dramatically - the signals have to travel an order of magnitude (or more) further, they have to leave the chip so you must synchronize with a different device, there's protocol overhead for storage and network, etc. Point being - sure we can absolutely do better with software in a lot of cases, but this is not a good example to derive conclusions from.
• lmm 3y ago
  > The argument is rather the following: common knowledge is that it’s not worth optimizing the code and that programming languages don’t matter, because the IO is too slow anyway. This fizzbuzz shows just how bad we are at IO compared to the optimum. So if we were to improve the IO, then faster processing would also make a difference. Maybe? But given that we don't have a way to magically make IO go much faster, that seems pretty irrelevant to the real world. The processor can go faster than memory; sure, we knew that. > In this case you get an improvement of 179x compared to Python, i.e. your laptop will do instead of multiple clusters in the cloud. So a highly skilled programmer devoting months to what's typically a 10-minute interview problem (i.e. about 5000x as much programmer time - and the time investment would probably scale superlinearly with the complexity of the problem) was able to get... a measly two order of magnitude improvement. That's really not a good tradeoff most of the time.
• PragmaticPulp 3y ago
  > Of course this is just a toy example. But suppose you had a different task that is I/O bound, like processing terabytes of jsonlines files. If this task was the bottleneck in a large scale system then it would definitely get hand optimized after a proper analysis. But if this is an occasionally run task or something otherwise not business critical that doesn’t bottleneck anything, spending orders of magnitude more time hyper-optimizing it would be a waste of time and money. Match the solution to the job. Optimizing everything is one of the age-old mistakes in computer science.
  • bombolo 3y ago
    You still need the skills to do it.