OP here. The demo is running off of the equivalent of a single K40. We can use 8 K80s to scale to billions of records with the same response time in millisecond

OP here. The demo is running off of the equivalent of a single K40. We can use 8 K80s to scale to billions of records with the same response time in milliseconds. And database workloads are often heavily memory bound so the terabytes per second of memory bandwidth you can get on a GPU server can give you a huge speed up, and of course you have the prodigious compute power of the GPUs on tap if needed.