Firing up a GPU just to do a little bit of inference sounds quite expensive. CPUs (let alone CPUs talking to a GPU) spend huge numbers of cycles shunting data

Firing up a GPU just to do a little bit of inference sounds quite expensive. CPUs (let alone CPUs talking to a GPU) spend huge numbers of cycles shunting data around already.