4 ms·
What happens when your LLM of choice goes on an infinite loop failing to solve a problem? What happens when your LLM provider goes down during an incident? Wh
by cassianoleal 7mo ago
What happens when your LLM of choice goes on an infinite loop failing to solve a problem?
What happens when your LLM provider goes down during an incident?
What happens when you have an incident on a distributed system so complex that no LLM can maintain a good enough understanding of the system as a whole in a single session to spot the problem?
What happens when the LLM providers stop offering loss leader subscriptions?
- junofan 7mo agoAFAIK everything I use has timeouts, retries, and some way of throwing up its hands and turning things back to me. I use several providers interchangeably. I stay away from overly complex distributed systems and use the simplest thing possible. I plan to wait for some guys in China to train a model on traces that I can run locally, benefitting from their national “diffusion” strategy and lack of access to bleeding-edge chips. I’m not worried.