5 ms·

Locally ran models exist, and more will come and they'll become cheaper and easier to run.

throwaway290 3y ago
Even if ClosedAI doesn't get government to ban sales of powerful GPUs to the public. Homebrew models will not be of comparable quality in near future. Don't be fooled by how cheap ChatGPT is, it is not profitable.
- MacsHeadroom 3y ago
  "Homebrew" Stable Diffusion beat DALL-E 2. There's zero reason this won't repeat for LLMs. All indications are that it will.
  - throwaway290 3y ago
    Sorry I don't think if you need to wait half an hour to see a result it's the same thing really. You don't need to worry about deluge of AI content if it can only come from a few crazy people with top of the line hardware spending days iterating on prompts. Especially if really good GPUs start requiring a license. At some point it's just faster to learn to write properly.
    - dingledork69 3y ago
      You are fooling yourself if you think the next generation of CPUs won't have ML coprocessors.
      - throwaway290 3y ago
        Apple's neural engine has been around in phones and laptops for years. Not even close to what's required for offline chatgpt. What's your point? Which of us is fooling themselves? Average consumer hardware is not showing any signs of being capable of this in near term. Hype is hype, but use your own head. Don't forget that these homegrown models also require training data, scraped and cleaned, an individual or nonprofit can't do that.
        srslack 3y ago
        >an individual or nonprofit can't do that. https://laion.ai/ https://laion.ai/ https://www.mosaicml.com/blog/mpt-7b https://www.mosaicml.com/blog/mpt-7b There's dozens if not hundreds more individuals and nonprofits doing these things.
        throwaway290 3y ago
        Yes, these are used by mainstream commercial AI thingies. But they are not their only sources. Plus they have armies of people preparing and cleaning this data, are you ready to employ one?
        srslack 3y ago
        People are literally doing this for free. You said that there is no individuals, and no non-profits, that can do this. Which is laughable. They're doing it right now: https://arxiv.org/abs/2305.11206 https://arxiv.org/abs/2305.11206 It may even be the case that all of that RLHF training that OpenAI does simply lessens the quality of generations, as suggested by one of their own papers and the paper above.
        throwaway290 3y ago
        65 billion parameters, and what hardware does this require to run fast enough to be usable?)
        srslack 3y ago
        https://lawrencecchen.com/llama-65b-m1-max https://lawrencecchen.com/llama-65b-m1-max Quantized, LLaMa can run on an Pixel 6: https://twitter.com/thiteanish/status/1635678053853536256 https://twitter.com/thiteanish/status/1635678053853536256 https://simonwillison.net/2023/Mar/13/alpaca/ https://simonwillison.net/2023/Mar/13/alpaca/
        3y ago
    - sieongioetnio 3y ago
      We already have publicly-available models that are good enough for spam and scams. When running on CPU they are already faster than most people can type. The cat is out of the bag. While the "AI alignment" jackasses were writing their Terminator fanfiction and wringing their hands about paperclips, they had already destroyed the world wide web as we know it.
      - throwaway290 3y ago
        Keep in mind the topic, it's not about WWW which can be totally swamped by a handful of dedicated malicious actors with powerful hardware, it's about verifying applicants that they did not cheat. These models you talk about are either not good or unbearably slow, the output of a model that today runs "as fast as you can type" on average hardware will never be reliably mistaken for the real thing. If you try to cheat with it it would be more likely to fail you than if you spend $20 on a human freelancer to write stuff The only factor that breaks this today is cheap availability of chatgpt and such. They are reasonably high quality but unprofitable to run, they are subsidized to hook public up so that later MS can safely jack up prices (ideally after getting an exclusive AI license from the government).
        MacsHeadroom 3y ago
        You could have said the same things about image generation 18 months ago.
        throwaway290 3y ago
        I could, and actually I did, and if I did not then I will now, say the same things about image generation today. Unless you know something I don't know or have top of the line hardware, general purpose image generation with homegrown models is either unbearably slow or poor quality.
        srslack 3y ago
        You can run Stable Diffusion on an iPhone 11 and it completes in under a minute or two. Running on CPU generally takes around 5 minutes. My almost top of the line macbook runs a batch of four in around 30 seconds on Metal, and I'm sure it's much faster with a mid-range GPU considering how unoptimized Metal is with Torch. And yes, you can go take a look around reddit and 4chan, the vast majority of those aren't dreambooth/MJ/remote models. That's not even taking into account local LoRAs and scripts that are possible instead of some company's untweakable crap. The open source around this is healthy, has pushed past DALL-E, and there's no real roadblock to Open Source LLMs except of course, the training cost. Even still, people are getting $200k+ models in their hands for free from various training runs and donated computing and LoRAing them and fine tuning them all to make them comparable to the closed off remote models. Any "cryptographic" scheme with the generations of these will just catch the lazy. The lazy already include the confabulated sources in their papers, and don't try to normalize the Error Level Analysis in generated images (probably the quickest way to determine whether an image is generated), so I don't think it's actually a net benefit. It's a cat and mouse game, and will push the mice further into the walls. You can't possibly say that generative images like this are "poor quality" https://www.reddit.com/r/StableDiffusion/comments/131lpks/my_collection_of_brokers_bankers_and_lawyers_into/ https://www.reddit.com/r/StableDiffusion/comments/131lpks/my... https://i.imgur.com/3iDf43z.png https://i.imgur.com/3iDf43z.png

Even if ClosedAI doesn't get government to ban sales of powerful GPUs to the public. Homebrew models will not be of comparable quality in near future. Don't be fooled by how cheap ChatGPT is, it is not profitable.

MacsHeadroom 3y ago
"Homebrew" Stable Diffusion beat DALL-E 2. There's zero reason this won't repeat for LLMs. All indications are that it will.
- throwaway290 3y ago
  Sorry I don't think if you need to wait half an hour to see a result it's the same thing really. You don't need to worry about deluge of AI content if it can only come from a few crazy people with top of the line hardware spending days iterating on prompts. Especially if really good GPUs start requiring a license. At some point it's just faster to learn to write properly.
  - dingledork69 3y ago
    You are fooling yourself if you think the next generation of CPUs won't have ML coprocessors.
    - throwaway290 3y ago
      Apple's neural engine has been around in phones and laptops for years. Not even close to what's required for offline chatgpt. What's your point? Which of us is fooling themselves? Average consumer hardware is not showing any signs of being capable of this in near term. Hype is hype, but use your own head. Don't forget that these homegrown models also require training data, scraped and cleaned, an individual or nonprofit can't do that.
      - srslack 3y ago
        >an individual or nonprofit can't do that. https://laion.ai/ https://laion.ai/ https://www.mosaicml.com/blog/mpt-7b https://www.mosaicml.com/blog/mpt-7b There's dozens if not hundreds more individuals and nonprofits doing these things.
        throwaway290 3y ago
        Yes, these are used by mainstream commercial AI thingies. But they are not their only sources. Plus they have armies of people preparing and cleaning this data, are you ready to employ one?
        srslack 3y ago
        People are literally doing this for free. You said that there is no individuals, and no non-profits, that can do this. Which is laughable. They're doing it right now: https://arxiv.org/abs/2305.11206 https://arxiv.org/abs/2305.11206 It may even be the case that all of that RLHF training that OpenAI does simply lessens the quality of generations, as suggested by one of their own papers and the paper above.
        throwaway290 3y ago
        65 billion parameters, and what hardware does this require to run fast enough to be usable?)
        srslack 3y ago
        https://lawrencecchen.com/llama-65b-m1-max https://lawrencecchen.com/llama-65b-m1-max Quantized, LLaMa can run on an Pixel 6: https://twitter.com/thiteanish/status/1635678053853536256 https://twitter.com/thiteanish/status/1635678053853536256 https://simonwillison.net/2023/Mar/13/alpaca/ https://simonwillison.net/2023/Mar/13/alpaca/
        3y ago
  - sieongioetnio 3y ago
    We already have publicly-available models that are good enough for spam and scams. When running on CPU they are already faster than most people can type. The cat is out of the bag. While the "AI alignment" jackasses were writing their Terminator fanfiction and wringing their hands about paperclips, they had already destroyed the world wide web as we know it.
    - throwaway290 3y ago
      Keep in mind the topic, it's not about WWW which can be totally swamped by a handful of dedicated malicious actors with powerful hardware, it's about verifying applicants that they did not cheat. These models you talk about are either not good or unbearably slow, the output of a model that today runs "as fast as you can type" on average hardware will never be reliably mistaken for the real thing. If you try to cheat with it it would be more likely to fail you than if you spend $20 on a human freelancer to write stuff The only factor that breaks this today is cheap availability of chatgpt and such. They are reasonably high quality but unprofitable to run, they are subsidized to hook public up so that later MS can safely jack up prices (ideally after getting an exclusive AI license from the government).
      - MacsHeadroom 3y ago
        You could have said the same things about image generation 18 months ago.
        throwaway290 3y ago
        I could, and actually I did, and if I did not then I will now, say the same things about image generation today. Unless you know something I don't know or have top of the line hardware, general purpose image generation with homegrown models is either unbearably slow or poor quality.
        srslack 3y ago
        You can run Stable Diffusion on an iPhone 11 and it completes in under a minute or two. Running on CPU generally takes around 5 minutes. My almost top of the line macbook runs a batch of four in around 30 seconds on Metal, and I'm sure it's much faster with a mid-range GPU considering how unoptimized Metal is with Torch. And yes, you can go take a look around reddit and 4chan, the vast majority of those aren't dreambooth/MJ/remote models. That's not even taking into account local LoRAs and scripts that are possible instead of some company's untweakable crap. The open source around this is healthy, has pushed past DALL-E, and there's no real roadblock to Open Source LLMs except of course, the training cost. Even still, people are getting $200k+ models in their hands for free from various training runs and donated computing and LoRAing them and fine tuning them all to make them comparable to the closed off remote models. Any "cryptographic" scheme with the generations of these will just catch the lazy. The lazy already include the confabulated sources in their papers, and don't try to normalize the Error Level Analysis in generated images (probably the quickest way to determine whether an image is generated), so I don't think it's actually a net benefit. It's a cat and mouse game, and will push the mice further into the walls. You can't possibly say that generative images like this are "poor quality" https://www.reddit.com/r/StableDiffusion/comments/131lpks/my_collection_of_brokers_bankers_and_lawyers_into/ https://www.reddit.com/r/StableDiffusion/comments/131lpks/my... https://i.imgur.com/3iDf43z.png https://i.imgur.com/3iDf43z.png