3 ms·
But the AST is effectively a "tiered system" lets say you have a 20TB of DB, 96GB fits in VRAM, 2 fits in your RAM, 17 and change are on your disks. Today the
by dogma1138 10y ago
But the AST is effectively a "tiered system" lets say you have a 20TB of DB, 96GB fits in VRAM, 2 fits in your RAM, 17 and change are on your disks.
Today the GPU can see this as a single continuous memory space, you have it does the AST and handles page faults when needed and tries to optimize the execution of each task by prefetching the needed data to the VRAM by it self through branch prediction or by programming the usage hints yourself.
Here is a short paper/write up on the unified memory in Pascal, look at the difference between optimizing/profiling your memory usage and using the prefetch hints, you going from increasing your data set by 4 times of max GPU memory and getting performance reduced by about 3.8-4 times relying on page faults alone to only losing less than 50% of your performance while going over 4 times your maximum available VRAM.
The same thing can be done on cards with 24GB, and the magic is that the performance drop actually levels off at about 2-3 times your maximum available VRAM which is why the difference between 2 and 4 times the amount of memory isn't that big in terms of performance hit/gain to over allocation ratio, so 8 times your amount of memory is not that far behind the penalty you already pay for using 4 times more.
And yes there is no scenario in which you do not lose performance, but the unified memory solution in NVIDIA GPUs is a very good solution to reduce the penalty and if you really need to go balls to the wall you'll actually benefit from the scaling.