4 ms·

I hate seeing these guys succeed because everyone of their successes is a new day that AI becomes less accessible to the average person and more locked behind t

I hate seeing these guys succeed because everyone of their successes is a new day that AI becomes less accessible to the average person and more locked behind their APIs.

sashank_1509 3y ago
I see this statement a lot and have no idea how people come to this conclusion. I have a beefy 16k$ workstation with 2 4090s and I could barely run the LLAMA 65B model at a very slow pace. Let us say we do have the model weights to GPT-4 and GPT3.5, me as the average consumer I don't know how this helps me in any way. I need to shell at least 25k (possibly much more for GPT-4) before I can run these models for even inference, and even then it will be a slow, unpolished experience. On the other hand OpenAI’s API makes things blazingly fast and dirt cheap to the average consumer. It honestly does feel like they have enabled the power of AI to be accessible to anyone with a laptop. If that requires fending off competition from Behemoths like Google, Meta by not releasing model weights then so be it. This critique would be more apt to Nvidia who are artificially increasing datacenter GPU prices thus pricing out the average consumer. OpenAI is doing the opposite.
- hackernewds 3y ago
  An "average" person is not someone who knows how to call an API. Perhaps only on HN
  - IMTDb 3y ago
    Then an "average" person is certainly not someone who is able to download and run an LLM on their device.
  - rolisz 3y ago
    If they don't know how to call an API, they won't know how to run local models (at the moment it's quite a pain to set-up all the dependencies)
    - bioemerl 3y ago
      It's actually not bad, the hard part is getting the hardware. Kobold will install itself most of the time with a double click.
  - theshrike79 3y ago
    "AI" as we know it is hardly 6 months old now, just wait a while and it'll be grandma accessible.
    - esafak 3y ago
      You exaggerate a bit! Machine learning and language models have been around for decades. OpenAI itself has been around since 2015.
- wellthisisgreat 3y ago
  > LLAMA 65B model at a very slow pace How does it compare to GPT 3.5, or 4? I mean if you ask the same questions. Is it usable at all? I tried the models that work with 4090 and they were completely useless for anything practical (code questions, etc.). Curiosities sure, but on Eliza level.
  - kgwgk 3y ago
    Is there a simple question / answer that you would find illuminating?
    - wellthisisgreat 3y ago
      the one that I used for GPT-4 and the local ones was a bit obscure: "how to configure internal pull-up resistor on PCA9557 from NXP in firmware" the GPT4 would give a paragraph of > The PCA9557 from NXP is an 8-bit I/O expander with an I2C interface. This device does not have an internal pull-up resistor on its I/O pins, but it does have software programmable input and output states. and then write a somewhat meaningful code. the local LLMs failed even at the paragraph stage could you try that?
- deleted 3y ago
  [deleted]
- bioemerl 3y ago
  > . I need to shell at least 25k (possibly much more for GPT-4) before I can run these models for even inference Give it a decade and you might be able to, but without the model you'll never have the option.
- ilaksh 3y ago
  I have been thinking about trying to do LoRA style fine tuning of Flacon-40b or Falcon-7b on RunPod. The new OpenAI 16k context and functions thinking made me lose the urge to get into that. Was questionable whether it could really write code consistently anyway even if very well fine tuned. But at least that is something that can be attempted without $25k.
error9348 3y ago
Less accessible compared to what?
egonschiele 3y ago
- The API is extremely cheap - There are plenty of open source tools built on top if it (example list: https://github.com/heartly/awesome-writing-tools https://github.com/heartly/awesome-writing-tools) While I wish this work was open, they are both the best and cheapest option out there... by a mile.
- bioemerl 3y ago
  This is exactly my problem. They are doing quite well, and closing the door behind them. Open AI isn't your friend and reserves the right to screw you down the line.
  - egonschiele 3y ago
    Oh, I 100% agree. I just haven't seen another model come close. I'd love to hear someone tell me why.
    - sp332 3y ago
      Lots and lots of human labor. https://gizmodo.com/chatgpt-openai-ai-contractors-15-dollars-per-hour-1850415474 https://gizmodo.com/chatgpt-openai-ai-contractors-15-dollars...
TeMPOraL 3y ago
Thing is, as long as the field is growing in capabilities as fast as it is, there isn't going to be any kind of "democratizing" for an average person, or even average developer. Anything you or me can come up with to do with LLM, some company or startup will do better, and they'll have people working full-time to productize it. Maybe it's FOMO and depression, but with $dayjob and family, I don't feel like there's any chance of doing anything useful with LLMs. Not when MS is about to integrate GPT-4 with Windows itself. Not when AI models scale superlinearly with amount of money you can throw at them. I mean, it's cool that some LLAMA model can run on a PC, provided it's beefy enough. I can afford one. Joe Random Startup Developing Shitty Fully Integrated SaaS Experience can afford 100 of them, plus an equivalent of 1000 of them in the cloud. Etc. Yeah, I guess it is FOMO and depression.

I hate seeing these guys succeed because everyone of their successes is a new day that AI becomes less accessible to the average person and more locked behind their APIs.

sashank_1509 3y ago

I see this statement a lot and have no idea how people come to this conclusion. I have a beefy 16k$ workstation with 2 4090s and I could barely run the LLAMA 65B model at a very slow pace. Let us say we do have the model weights to GPT-4 and GPT3.5, me as the average consumer I don't know how this helps me in any way. I need to shell at least 25k (possibly much more for GPT-4) before I can run these models for even inference, and even then it will be a slow, unpolished experience. On the other hand OpenAI’s API makes things blazingly fast and dirt cheap to the average consumer. It honestly does feel like they have enabled the power of AI to be accessible to anyone with a laptop. If that requires fending off competition from Behemoths like Google, Meta by not releasing model weights then so be it. This critique would be more apt to Nvidia who are artificially increasing datacenter GPU prices thus pricing out the average consumer. OpenAI is doing the opposite.

hackernewds 3y ago
An "average" person is not someone who knows how to call an API. Perhaps only on HN
- IMTDb 3y ago
  Then an "average" person is certainly not someone who is able to download and run an LLM on their device.
- rolisz 3y ago
  If they don't know how to call an API, they won't know how to run local models (at the moment it's quite a pain to set-up all the dependencies)
  - bioemerl 3y ago
    It's actually not bad, the hard part is getting the hardware. Kobold will install itself most of the time with a double click.
- theshrike79 3y ago
  "AI" as we know it is hardly 6 months old now, just wait a while and it'll be grandma accessible.
  - esafak 3y ago
    You exaggerate a bit! Machine learning and language models have been around for decades. OpenAI itself has been around since 2015.
wellthisisgreat 3y ago
> LLAMA 65B model at a very slow pace How does it compare to GPT 3.5, or 4? I mean if you ask the same questions. Is it usable at all? I tried the models that work with 4090 and they were completely useless for anything practical (code questions, etc.). Curiosities sure, but on Eliza level.
- kgwgk 3y ago
  Is there a simple question / answer that you would find illuminating?
  - wellthisisgreat 3y ago
    the one that I used for GPT-4 and the local ones was a bit obscure: "how to configure internal pull-up resistor on PCA9557 from NXP in firmware" the GPT4 would give a paragraph of > The PCA9557 from NXP is an 8-bit I/O expander with an I2C interface. This device does not have an internal pull-up resistor on its I/O pins, but it does have software programmable input and output states. and then write a somewhat meaningful code. the local LLMs failed even at the paragraph stage could you try that?
deleted 3y ago
[deleted]
bioemerl 3y ago
> . I need to shell at least 25k (possibly much more for GPT-4) before I can run these models for even inference Give it a decade and you might be able to, but without the model you'll never have the option.
ilaksh 3y ago
I have been thinking about trying to do LoRA style fine tuning of Flacon-40b or Falcon-7b on RunPod. The new OpenAI 16k context and functions thinking made me lose the urge to get into that. Was questionable whether it could really write code consistently anyway even if very well fine tuned. But at least that is something that can be attempted without $25k.

error9348 3y ago

Less accessible compared to what?

egonschiele 3y ago

- The API is extremely cheap - There are plenty of open source tools built on top if it (example list: https://github.com/heartly/awesome-writing-tools https://github.com/heartly/awesome-writing-tools) While I wish this work was open, they are both the best and cheapest option out there... by a mile.

bioemerl 3y ago
This is exactly my problem. They are doing quite well, and closing the door behind them. Open AI isn't your friend and reserves the right to screw you down the line.
- egonschiele 3y ago
  Oh, I 100% agree. I just haven't seen another model come close. I'd love to hear someone tell me why.
  - sp332 3y ago
    Lots and lots of human labor. https://gizmodo.com/chatgpt-openai-ai-contractors-15-dollars-per-hour-1850415474 https://gizmodo.com/chatgpt-openai-ai-contractors-15-dollars...

TeMPOraL 3y ago

Thing is, as long as the field is growing in capabilities as fast as it is, there isn't going to be any kind of "democratizing" for an average person, or even average developer. Anything you or me can come up with to do with LLM, some company or startup will do better, and they'll have people working full-time to productize it. Maybe it's FOMO and depression, but with $dayjob and family, I don't feel like there's any chance of doing anything useful with LLMs. Not when MS is about to integrate GPT-4 with Windows itself. Not when AI models scale superlinearly with amount of money you can throw at them. I mean, it's cool that some LLAMA model can run on a PC, provided it's beefy enough. I can afford one. Joe Random Startup Developing Shitty Fully Integrated SaaS Experience can afford 100 of them, plus an equivalent of 1000 of them in the cloud. Etc. Yeah, I guess it is FOMO and depression.