3 ms·

They could also just call out to web services with cellular internet.

• ghayes 3y ago
  The RTT would need to be very low. It’s hard to have a conversation with 2-3 second pauses.
  • shadowgovt 3y ago
    The long poll in GPT isn't network; it's compute. Unfortunately, the tech probably isn't there yet for this use case.
    • lovemenot 3y ago
      This might be worked around with a characteristic verbal affectation that makes their utterances initially long-winded. Once the LLM's lag is passed, the proper content may begin to flow. Thinking fast and slow.
    • patapong 3y ago
      I am not sure about that! It seems like small models are emerging that are a bit more specific but can be very small, and thus have much lower latency. For example: https://arxiv.org/abs/2305.07759 https://arxiv.org/abs/2305.07759