3 ms·

The main reason LLMs may appear disruptive at this point is because the real cost has yet to be fully factored in. Not too many applications can justify spendi

• ben_w 2y ago
  The main cost is training rather than inference, and compute is still getting cheaper, so I don't expect this to limit the use of LLMs. We may finally see people realising that just because they can use a hammer on a screw, doesn't make it the best choice.
  • red-iron-pine 2y ago
    cheaper, but still crazy damn expensive it makes sense because big money is betting it's worth the investment -- but it may not be
• Ukv 2y ago
  I'm not so certain. You can already run a GPT-4-quality model locally on a decent desktop, and GPT-3-quality models on low-powered chips - plus data centers will benefit from scale. A lot of third-party services are using paid APIs that (based on cases like Mistral where some models are publicly available) appear to more than cover the inference costs. There are also plenty of uses for LLMs beyond generating hopefully-accurate answers, such as for fictional content or use as foundation models for tasks like translation. Though we are definitely in the "throw things at the wall and see what sticks" stage currently.