3 ms·
Let’s say TTFT needed the most improvement. At some point, loading the model with enough context size may take tens of seconds in some macs.
by freehorse 8mo ago
Let’s say TTFT needed the most improvement. At some point, loading the model with enough context size may take tens of seconds in some macs.