3 ms·
The better use case is obviously voice assistant at the edge. As in voice 2 text 2 search/GPT 2 voice generated response. That is where ms matter but it is also
by unraveller 3y ago
The better use case is obviously voice assistant at the edge. As in voice 2 text 2 search/GPT 2 voice generated response. That is where ms matter but it is also a high abuse angle no one wants to associate with just yet. My guess is they are going to do this in another post, and if so they should make their own perplexity style online-gpt. For now they just wanted to see what else people can think up by making the introduction of it boring.
- ec109685 3y agoThere’s three options for inference: 1) On device inference 2) Inference “on the edge” 3) Inference in a data center Given fly is deployed in equinox data centers just like everyone else, fundamentally there isn’t much difference between #2 and #3.