I don't think its possible for a CPU to read memory without going through the cache. Moreover, the value of speculative execution drops really hard if you can't

I don't think its possible for a CPU to read memory without going through the cache. Moreover, the value of speculative execution drops really hard if you can't use any cache, as it means you are restricted to operating only on registers.