3 ms·
> He is limited by not having an "expensive macbook with 4 real cores" ?? He gave up on a test because it "paged out the 15GB of data". Note that he already de
by lambda 9y ago
> He is limited by not having an "expensive macbook with 4 real cores" ?? He gave up on a test because it "paged out the 15GB of data".
Note that he already demonstrated that for many of the tests, his older MacBook was faster than these 160 core clusters. There were a couple where the older MacBook was not faster than a 160 core cluster, but he suspects that a newer one might be.
The one that paged out happened to be one where he artificially broke it up into 160 pieces to simulate what it would be like if he had 160 similar cores, but that caused enough issues with memory locality that swapping dominated the results. This artificial partitioning was to give a sense for what results on a 160 core cluster "should" be, but the swapping distorts the results enough that for that problem it wouldn't even be a very good rough estimate.
> Is this guy for real?
This isn't supposed to be a publication-quality paper. This is a blog post, using the resources he has immediately at hand, to show why if you follow cutting edge distributed database research you might be led down a wrong, and very costly, path, which you could avoid with a much simpler implementation on much cheaper hardware.
Note that he references an earlier paper that he did publish on this topic; this blog post is mostly just doing a similar check on some more recent published work, to see if much has improved in the field:
https://www.usenix.org/system/files/conference/hotos15/hotos15-paper-mcsherry.pdf https://www.usenix.org/system/files/conference/hotos15/hotos...