3 ms·
is 70B not commercially useful because of the model storage requirements, or total inference performance, or additional memory per session that's inferencing, o
by paulmd 3y ago
is 70B not commercially useful because of the model storage requirements, or total inference performance, or additional memory per session that's inferencing, or what?
is the output better such that it's desirable, or is this just a case of "too much performance hit for a marginal gain"?