2 ms·
"The speed people are just creating wrappers or minor changes" To be fair, the overhead of running LLMs at scale -- the cost per interaction -- is the limiting
by llm_nerd 3y ago
"The speed people are just creating wrappers or minor changes"
To be fair, the overhead of running LLMs at scale -- the cost per interaction -- is the limiting factor for commercial deployments. Efficiency is an enormous advantage.