4 ms·
What's the round trip latency on this? ask question -> response. Do you parse the question word by word and feed into llm or wait for the whole question before
by qazxcvbnmlp 1y ago
What's the round trip latency on this? ask question -> response. Do you parse the question word by word and feed into llm or wait for the whole question before feeling into llm?