The catch is that the token generation speed is going to be limited by network latency, making it unbearably slow to run over the internet. It can be great on

The catch is that the token generation speed is going to be limited by network latency, making it unbearably slow to run over the internet. It can be great on a local network though, especially if your workload is prefill-heavy (more text input to process than output tokens to emit).