2 ms·
I’m not sure what problem this would solve. The data sent to, and revived from, the LLM is small. Why would saving a couple theoretical hops on the network matt
by al_borland 1y ago
I’m not sure what problem this would solve. The data sent to, and revived from, the LLM is small. Why would saving a couple theoretical hops on the network matter? That’s not the bottleneck.
- dwa3592 1y agoThat's a fair thought. Sending my data to a local office would be much more quicker than sending it to Virginia or California.
- al_borland 1y agoWe’re talking about milliseconds here, when the processing of the query can take 10+ seconds. Local offices will likely be slower than the large regional data centers, unless that is offset by having fewer users. But does it scale? If we take this a few steps further, why not have the LLM running locally in your home?