LFENCEs of all things helped I don't really understand what was happening here, but it was a definite positive effect. I think the issue is that the early pre

LFENCEs of all things helped I don't really understand what was happening here, but it was a definite positive effect. I think the issue is that the early preloads filled the queue, preventing the stores from happening. The bright side is that getting good performance on Haswell is easier, as it supports 2 loads and 1 store per cycle (although note that address generation is still a bottleneck unless you make sure that the store is using a "simple" address). I've thought about designs of having one core do memory prefetching ahead of loosely synchronized other core doing actual computation I'm interested in this approach too, but haven't really tried it out. "srean" suggested this also: https://news.ycombinator.com/item?id=8711626 https://news.ycombinator.com/item?id=8711626 I think it could would work well if you know your access patterns in advance, the hard part would be keeping the cores in sync. In the absence of real-time synchronization, you'd need a really high throughput queue to make this work. Interesting write up and links here: http://www.infoq.com/articles/High-Performance-Java-Inter-Thread-Communications http://www.infoq.com/articles/High-Performance-Java-Inter-Th... It's a much easier problem if you can batch up the preloads rather than doing them one at a time. I have had success with a somewhat similar technique within a single core by issuing a batch of prefetches, then do processing on the previous data, then a batch of loads corresponding to the prefetches, then repeat. You can get 2-3x improvement of throughput on random access if you are able to tolerate the slightly increased latency. The key is figuring out how to keep the maximum number of requests in flight at all times.