4 ms·
It's not a formal must, it's a practical "must". Paging and streaming and distribution (and collection) all bring their own overhead, and given the architecture
by swatcoder 3y ago
It's not a formal must, it's a practical "must". Paging and streaming and distribution (and collection) all bring their own overhead, and given the architecture of most of these systems, the overhead of those techniques is very large.
So for researchers who can access a system with enough memory, which is most of the people professionally exploring these models, nobody needs to invest much preparatory effort into those other techniques or reducing their overhead. For them, there's basically no ROI for it.
But a lot of that work to optimize for those other techniques is gradually being accomplished in the open source community, where people don't have access to expensive clouds and lab systems. It takes time though, and can still only achieve so much.