3 ms·
Take a deep look at Julia! Adding multithreaded parsing capabilities to CSV.jl was really a joy; basically just chunking up the file and spawning threaded tasks
by karbarcca 6y ago
Take a deep look at Julia! Adding multithreaded parsing capabilities to CSV.jl was really a joy; basically just chunking up the file and spawning threaded tasks to process each chunk with the existing parsing code.
My favorite favorite thing about developing in Julia is the ability to write "high-level" type code and usually get decent performance, BUT THEN have the ability to fine tune for that extra boost of performance. The code inspection tools (see lowered, typed, LLVM, or native IR code levels) and super nice Task-based multithreading really make performance tuning a delight.
- willvarfar 6y ago(Some of the comments in https://news.ycombinator.com/item?id=24736559 https://news.ycombinator.com/item?id=24736559 talk about new lines inside fields, making multi-threading difficult) (My quick search to find that recent story on HN made me find this too https://news.ycombinator.com/item?id=24740527 https://news.ycombinator.com/item?id=24740527)