3 ms·
I don't see AI system too special in terms of back-end engineering, except maybe for agentic system, things are inherently stateful. But considering how limite
by karmasimida 2y ago
I don't see AI system too special in terms of back-end engineering, except maybe for agentic system, things are inherently stateful.
But considering how limited RPM/TPM with regards mainstream LLMs, states saving/loading is hardly the bottleneck I feel.
- pcalcado 2y agoSame here. I've had data-intensive systems and classifiers on critical paths for non-AI apps, and the same tools I used before seem to work fine with GenAI. The primary real difference I've found has to do with when agents make decisions; this creates arbitrary call graphs in your distributed architecture and makes it harder to provision things, optimize, and do anomaly detection.