3 ms·
I get the argument, but let's be blunt: every serious cache system deals with consistency, partial reuse, and correctness. That’s standard engineering - regardl
by varispeed 1y ago
I get the argument, but let's be blunt: every serious cache system deals with consistency, partial reuse, and correctness. That’s standard engineering - regardless of how much intimidating jargon you layer over it. Useful, sure. But watching the industry throw a circus around basic cache management, complete with papers and corporate branding, is exactly why so much of modern tech feels like a hype-driven clown show rather than a disciplined craft.
- bGl2YW5j 1y agoI’m with you. It’s a bit shocking.
- vlovich123 1y agoI really don’t understand what you’re saying. This isn’t about consistency of the data. If you don’t figure out a mathematically valid way to combine the precomputed values of snippets of text, then the LLM just doesn’t work properly. Prefix cache management which is just normal systems engineering is not all this is doing. Stitching together cache fragments such that the LLM is actually still reasoning correctly about the text is hard. Have you read the paper?
- imtringued 1y agoYou're bragging about the easy parts and rolling your eyes at the hard parts. Meanwhile the AI engineers are doing the exact opposite. Bragging about the hard parts and rolling their eyes at the easy parts.