3 ms·
People interested in running multiple LLM queries in parallel are not people who would consider buying Apple Silicon.
by ryao 2y ago
People interested in running multiple LLM queries in parallel are not people who would consider buying Apple Silicon.
- int_19h 2y agoThere are other ways to parallelize even a single query for faster output, e.g. speculative decoding with small draft models.