3 ms·
Making LLM workflows human friendly
- k2so 2y agoLLMs = Latency? That's how most of us perceive it. When examining the timing breakdown of a request on Claude, you'll notice that the majority of the time is spent in Content Download—essentially, decoding output tokens. In the blog, I discuss how partial json validation can help in workflow driven LLM products. Would love feedback on how I can improve, thanks!