3 ms·
It depends on the time the job itself takes. If you're having the LLM handle a training run for another model, the LLM is probably spending most of its time wai
by hgoel 1mo ago
It depends on the time the job itself takes. If you're having the LLM handle a training run for another model, the LLM is probably spending most of its time waiting for iterations rather than consuming tokens.
For a task I left a local model running on overnight, only ~100k tokens were used because most of the time was just waiting on tests to finish, then waking up, tweaking a few settings and trying again.
- roy3838 1mo agoHi!! This is basically why I built Observer (https://github.com/Roy3838/Observer https://github.com/Roy3838/Observer, FOSS). It runs a small local model that watches your screen/terminal and pings or calls you when the thing you're waiting for actually happens: tests finished, run crashed, agent stuck on a prompt. So an overnight loop like yours wakes you only when it needs a decision, instead of you waking on a timer to check. But in this case it would be triple LLM inception, one training another and a third one monitoring everything is done correctly :p