2 ms·
> context management, and keeping utilization in the 40%-60% range (depends on complexity of the problem). Is this a rule of thumb? Will the cheaper (fewer par
by aitchnyu 1y ago
> context management, and keeping utilization in the 40%-60% range (depends on complexity of the problem).
Is this a rule of thumb? Will the cheaper (fewer params) models dumb down at 25%?