6 ms·
Processing 256M publications records using Awk and parallel scripting
I participated in a organizational data challenge in 2018 and chose awk to process the massive data to solve several interesting challenges.
The repo is: https://github.com/ketancmaheshwari/SMC18
A report detailing the approach and results is here: https://github.com/ketancmaheshwari/SMC18/blob/master/report/SMC18_DataChallenge4.pdf
- yesenadam 6y agoYou might try posting this as a usual HN news item to get more response, or write a blog post about it and link to that. (Awk lover here)
- ketanmaheshwari 6y agoThanks! Will do so soon.
- mraza007 6y agoHey that’s pretty cool you should definitely write about it in a blog post