But for tiny models on CPUs, that kind of thing doesn't seem practical, it seems like you get at most one LLM call to a 360M model with maybe a few paragraphs o

But for tiny models on CPUs, that kind of thing doesn't seem practical, it seems like you get at most one LLM call to a 360M model with maybe a few paragraphs of context, before a user will probably be tired of waiting.