2 ms·
Or maybe they think they should be sending each keystroke to a server and waiting for the response.
by stavros 10mo ago
Or maybe they think they should be sending each keystroke to a server and waiting for the response.
- amelius 10mo agoA server on Mars?
- stavros 10mo agoGotta have multiple AZs.
- me551ah 10mo agoThe server is probably running Python
- bigfishrunning 10mo agolol it's the flask debug server, "don't use this in production" banner and all
- JoelMcCracken 10mo agoNa each key press goes to a separate lambda invocation that gets submitted to a Kafka queue, and what happens after that is a mystery to all involved. We can make crazy latency ourselves just fine, no space transmission necessary
- labcomputer 10mo agoNo, not a mystery, in fact. Each keypress is appended to an 80 line prompt (key name along with timestamp of keypress and current text shown on the screen) and fed to a frontier LLM. Some of the office staff banged on the keypad for a few hours to generate training data to fine-tune the LLM on the task of denouncing key presses. Thanks to some optimizations with Triton and running multi-GPU instances, latency is down to just a few seconds per digit entered. You see, we needed to hit our genAI onboarding KPIs this quarter…
- trinix912 10mo agoProbably a Celeron-powered PC tower barely keeping up with Windows Server 2008 R2 in a closet of a public office ;)