Its not about algorithms. The issue is mentioned in the paper: even if the cache-based side channels are stopped (lets say, by manually clearing any cached memo

Its not about algorithms. The issue is mentioned in the paper: even if the cache-based side channels are stopped (lets say, by manually clearing any cached memory as soon as its accessed, reducing performance to 0.1% of what it is now) there are other potential side channels like through limited CPU resources. A lot of the "fixes" might just be mitigations, that don't solve the problem entirely, but make it take much longer. Maybe you can cover up enough holes, and make the timers available poor enough that it takes a few hours or days to read a single byte. In other words, mitigations to make things less simple.