3 ms·

Just playing casually with NanoGPT (https://github.com/karpathy/nanoGPT https://github.com/karpathy/nanoGPT) with a desktop holding a 2080ti, it's really really

Just playing casually with NanoGPT (https://github.com/karpathy/nanoGPT https://github.com/karpathy/nanoGPT) with a desktop holding a 2080ti, it's really really really clear to me that the path to get to a pre-fine-tuned LLM is remarkably easy. RLHF is the piece above this which appears to also be surprisingly easy (if Sam Altman is to be believed). The juice is making these tools incredibly easy. I think the barrier to entry here is low. OpenAI is ahead now, but I doubt that lives forever.

yieldcrv 4y ago
Exactly, almost nobody is going to need a trillion parameter LLM Its going to go client side, at the OS level and have like 1% of the mental capacity and be good enough
- Closi 4y ago
  This comment might end up being a bit like "640kb of RAM is more than anyone will ever need!" GPT3 already used almost 1/5th of that figure at 180bn parameters, and PaML uses 500bn.
  - yieldcrv 4y ago
    Okay, if client side resources expand then larger parameter LLMs will be used. There. The point is that it will be ubiquitously client side, and it will happen faster than newer hardware comes out. Current hardware is very limited and slow in getting output from LLMs.
    - Closi 4y ago
      I'm not convinced - at least on current hardware this seems well positioned for the cloud. My tiny Alexa puck isn't going to run a 180bn parameter LLM that runs best on 10 graphics cards any time soon, however it can already call a simple API and get a response in only 50ms more. I suspect people will prefer the cloud overhead of 50ms for a better response for a bigger model for a lot of queries. But who knows at this stage! I guess it could go either way depending on how both hardware and these models advance. I just assume that in the close future, most people are going to be interacting with LLM's on low-cost devices with limited/varied compute.

yieldcrv 4y ago

Exactly, almost nobody is going to need a trillion parameter LLM Its going to go client side, at the OS level and have like 1% of the mental capacity and be good enough

Closi 4y ago
This comment might end up being a bit like "640kb of RAM is more than anyone will ever need!" GPT3 already used almost 1/5th of that figure at 180bn parameters, and PaML uses 500bn.
- yieldcrv 4y ago
  Okay, if client side resources expand then larger parameter LLMs will be used. There. The point is that it will be ubiquitously client side, and it will happen faster than newer hardware comes out. Current hardware is very limited and slow in getting output from LLMs.
  - Closi 4y ago
    I'm not convinced - at least on current hardware this seems well positioned for the cloud. My tiny Alexa puck isn't going to run a 180bn parameter LLM that runs best on 10 graphics cards any time soon, however it can already call a simple API and get a response in only 50ms more. I suspect people will prefer the cloud overhead of 50ms for a better response for a bigger model for a lot of queries. But who knows at this stage! I guess it could go either way depending on how both hardware and these models advance. I just assume that in the close future, most people are going to be interacting with LLM's on low-cost devices with limited/varied compute.