In my experience, performance is dominated almost completely by memory throughput and/or memory latency (depending on whether the prefetcher succeeds). All the

In my experience, performance is dominated almost completely by memory throughput and/or memory latency (depending on whether the prefetcher succeeds). All the instruction latencies, VLIW optimizations, and so forth have a minor effect. However, the cases where these things do have an effect, are exactly the cases where it might make sense to go down to ASM level. The first avenue to improve performance, though, is improving memory access patterns. Compacting, aligning and rearranging data in cache windows. Even creating and maintaining redundant copies of data which is compacted or organized in a way that aids the cache can help. Another big improvement I've witnessed is helping the prefetcher. For example, I've had code that had to iterate a few linked lists to work on all elements. When I reorganized that code to interleave the iteration of 8 linked lists at a time (rather than iterating them all serially), I went up from being memory-latency-bound, to being closer to memory-throughput-bound. The prefetcher could predict the next 8 ptrs I'll dereference, and prefetch 8 cache lines, instead of 1 at a time.