We call them GPUs because they are built from GPU chips. The Tesla M40 for example is based on a GM200 chip which is the same as in a GeForce GTX Titan X GPU.

We call them GPUs because they are built from GPU chips. The Tesla M40 for example is based on a GM200 chip which is the same as in a GeForce GTX Titan X GPU. Also Nvidia and other assemblers have done exactly that: separate boxes that you attach to servers by a PCIe cable. See for example the Tesla S2050 server: http://www.nvidia.com/docs/io/43395/nv-ds-tesla-s2050-june10-final-lores.pdf http://www.nvidia.com/docs/io/43395/nv-ds-tesla-s2050-june10... A good list of Tesla systems is at https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_processing_units#Tesla https://en.wikipedia.org/wiki/List_of_Nvidia_graphics_proces... (which, coincidentally, I have been heavily updating these last few days).