The problem is floating point math. Mostly we want GPU's (and TPU's) to crunch floating point match really quickly. However, FP units are complicated things and

The problem is floating point math. Mostly we want GPU's (and TPU's) to crunch floating point match really quickly. However, FP units are complicated things and take a disproportionate amount of FPGA fabric. Add to this the lower clock speed and suddenly GPU's start to look really cheap.