4 ms·

Fine by me. The C version was a pain in the ass to compile. The Go version compiles in less than one minute, with a simple `go build`, on Windows, Linux or any

Fine by me. The C version was a pain in the ass to compile. The Go version compiles in less than one minute, with a simple `go build`, on Windows, Linux or any Go platform. I will take that over the faster speed every time.

tinix 5y ago
That's a fascinating perspective... This mentality is why modern software seems slower... Because it is. So many technological advances, computers are exponentially faster in so many ways, yet we regress in performance. It's a shame, really...
- svnpenn 5y ago
  Well, compared to Go, Rust, PHP, or many other modern tools, C is garbage in terms of build experience. Speed is important, but its not the only factor.
- marginalia_nu 5y ago
  It's basically Parkinson's law. If you give someone 10x the performance, they'll find a way of doing the same things they did before using 10x more resources. It's why you have user interfaces today that are often as slow as, or slower than the equivalent Windows 3.11 interfaces. In 1993 things took time because the processor was an actual potato and the data fetched off a floppy disk, today things are slow because the code being run is 1-10k times slower and messages need to be continuously sent to a datacenter half way around the world to inform an analytics server about your every input.
  - john_kerl 5y ago
    The 10x number was before improvements on https://github.com/johnkerl/miller/pull/786 https://github.com/johnkerl/miller/pull/786 et al. The earlier negative perf results were my fault, not Go's -- I was focusing initially on the port and feature development, leaving benchmarking and optimization until the end. That said, Go is a bit slower than C line for line; however, Miller 5 (in C) was single-threaded and Miller 6 (in Go) actively uses multicore. This is why complex processing chains now run much quicker in Go than in C -- due to multicore and pipelining which are much easier to do in Go.
jasonpeacock 5y ago
You'd rather optimize for compile time, which you do (maybe) 1x, over runtime, which you easily do >100x? You're focusing on the developer, not the user, but this is a user tool :(
- john_kerl 5y ago
  Agreed, very much a user tool -- once it's released most people will get it via brew/yum/apt/conda/chocolatey and compile time will not be relevant for most folks. Miller has always been a user tool and will remain so. I think the confusion arose because I did the port per se and feature-adds first (which has taken most of the time), and left analysis/optimization (even low-hanging fruit like output-buffering) until the very end -- https://github.com/johnkerl/miller/pull/786 https://github.com/johnkerl/miller/pull/786 for example merged just a couple days ago.
john_kerl 5y ago
Agreed; I've never really liked C compiles, and the Go build process is dreamy. Another win (besides more features, and recent performance improvements) -- the Windows version is now a snap. Now `mlr.exe` takes it rightful place alongside the Linux and Mac executables (https://github.com/johnkerl/miller/releases/tag/v6.0.0-beta https://github.com/johnkerl/miller/releases/tag/v6.0.0-beta). Go+Windows is heavenly & I'll never need to tweak MSYS2/Appveyor/DLLs/etc again. :) See also https://miller.readthedocs.io/en/latest/miller-on-windows/ https://miller.readthedocs.io/en/latest/miller-on-windows/

tinix 5y ago

That's a fascinating perspective... This mentality is why modern software seems slower... Because it is. So many technological advances, computers are exponentially faster in so many ways, yet we regress in performance. It's a shame, really...

svnpenn 5y ago
Well, compared to Go, Rust, PHP, or many other modern tools, C is garbage in terms of build experience. Speed is important, but its not the only factor.
marginalia_nu 5y ago
It's basically Parkinson's law. If you give someone 10x the performance, they'll find a way of doing the same things they did before using 10x more resources. It's why you have user interfaces today that are often as slow as, or slower than the equivalent Windows 3.11 interfaces. In 1993 things took time because the processor was an actual potato and the data fetched off a floppy disk, today things are slow because the code being run is 1-10k times slower and messages need to be continuously sent to a datacenter half way around the world to inform an analytics server about your every input.
- john_kerl 5y ago
  The 10x number was before improvements on https://github.com/johnkerl/miller/pull/786 https://github.com/johnkerl/miller/pull/786 et al. The earlier negative perf results were my fault, not Go's -- I was focusing initially on the port and feature development, leaving benchmarking and optimization until the end. That said, Go is a bit slower than C line for line; however, Miller 5 (in C) was single-threaded and Miller 6 (in Go) actively uses multicore. This is why complex processing chains now run much quicker in Go than in C -- due to multicore and pipelining which are much easier to do in Go.

jasonpeacock 5y ago

You'd rather optimize for compile time, which you do (maybe) 1x, over runtime, which you easily do >100x? You're focusing on the developer, not the user, but this is a user tool :(

john_kerl 5y ago
Agreed, very much a user tool -- once it's released most people will get it via brew/yum/apt/conda/chocolatey and compile time will not be relevant for most folks. Miller has always been a user tool and will remain so. I think the confusion arose because I did the port per se and feature-adds first (which has taken most of the time), and left analysis/optimization (even low-hanging fruit like output-buffering) until the very end -- https://github.com/johnkerl/miller/pull/786 https://github.com/johnkerl/miller/pull/786 for example merged just a couple days ago.

john_kerl 5y ago

Agreed; I've never really liked C compiles, and the Go build process is dreamy. Another win (besides more features, and recent performance improvements) -- the Windows version is now a snap. Now `mlr.exe` takes it rightful place alongside the Linux and Mac executables (https://github.com/johnkerl/miller/releases/tag/v6.0.0-beta https://github.com/johnkerl/miller/releases/tag/v6.0.0-beta). Go+Windows is heavenly & I'll never need to tweak MSYS2/Appveyor/DLLs/etc again. :) See also https://miller.readthedocs.io/en/latest/miller-on-windows/ https://miller.readthedocs.io/en/latest/miller-on-windows/