I see a dash of black magic was used... I'm extremely curious what optimizations and algorithms were used for such performance gains. >This takes about 1 secon

I see a dash of black magic was used... I'm extremely curious what optimizations and algorithms were used for such performance gains. >This takes about 1 second - sift processed 40 million records / 1 GB of data in just one second.