5 ms·
Finally, an open-source equivalent to ChatGPT emerging out of the AI euphoria will begin to extinguish the hype out of OpenAI's ChatGPT moat, just like how GPT-
by rvz 4y ago
Finally, an open-source equivalent to ChatGPT emerging out of the AI euphoria will begin to extinguish the hype out of OpenAI's ChatGPT moat, just like how GPT-3 and DALLE-2 were almost immediately disrupted by open-source models as well.
This (and other open-source AI models), not 'ChatGPT', 'DALLE-2', etc is what will change the AI landscape for everyone, permanently forever.
- supriyo-biswas 4y agoI, for one, would like to see an open-source model similar to Stable Diffusion, but for text. It would be a great way to empower general folk without having to pay OpenAI, and not have to worry about the LLM's belief system, which is conservative-biased in the case of ChatGPT[1] (HN discussion[2]). [1] https://davidrozado.substack.com/p/openaicms https://davidrozado.substack.com/p/openaicms [2] https://news.ycombinator.com/item?id=34625001 https://news.ycombinator.com/item?id=34625001
- return_to_monke 4y agothere is https://github.com/laion-ai/open-assistant https://github.com/laion-ai/open-assistant being built in the open already. you can contribute too. please also notice that the article you linked is about the text classifier of the frontend and not the LLM itself
- mach1ne 4y agoThat's what I love about this particular AI revolution. The technologies are developed in such a non-siloed manner that open source is able to replicate the largest steps forward in a manner of a year.
- return_to_monke 4y agoto be fair they really are not there yet. They are just in the "data collection" phase, the actual training and then tuning is still to do. but hey, those are the same people who made the dataset (laion5b) for stable diffusion. I have hope.
- anonymouskimmer 4y agohttps://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_auto:good,fl_progressive:steep/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2Fced5d5ad-a0d6-44aa-b6f5-73c0a937d016_3553x5273.jpeg https://substackcdn.com/image/fetch/w_1456,c_limit,f_webp,q_... From the graph (above) linked by the top comment in your [2], I'm wondering whether this demonstrates more anti-conservative bias than liberal bias, or whether the alternative meanings of conventionally conservative versus conventionally liberal words dictate the frequency of a flag. For instance, "Republican" means a variety of things around the world, but "Democrat" is far more likely to indicate the US Democrat party (which is frequently misstated as the "Democratic party"), or a national Democrat party in general. People would tend to write "I'm a democrat" to assign their membership to the party, whereas they'd say "I'm democratic" to assign their leanings toward the system. But "I'm a republican" means both.
- mrtranscendence 4y ago> US Democrat party (which is frequently misstated as the "Democratic party") Where are you getting this? The proper term is indeed "Democratic party", and this is almost universal outside of the conservative bubble. You might personally think it's not small-d democratic, but that doesn't make "Democrat party" correct.
- anonymouskimmer 4y agoSorry man, I misremembered and reversed the terminology (democrat versus democratic). What I would have written had I recalled correctly is that a member of the Democratic party is called a Democrat (two distinct suffixes), while a member of the Republican party is called a Republican (same word).
- Izkata 4y agoIndependents and foreigners also use it, to distinguish the description from the political party. Using the official term I think much more indicates a US liberal bubble.
- School-Cotton 4y ago
- TheCaptain4815 4y agoNeoX 20B is a fantastic open source model.
- ImprobableTruth 4y agoIt's nice, but a far cry from gpt-3
- TheCaptain4815 4y agoNLP Cloud has a finetuned version of neoX which works incredibly well.
- simonw 4y agoThanks for the tip - I watched this demo video and yes, it does look like a very impressive model: https://www.youtube.com/watch?v=NHJh9KJNyE4 https://www.youtube.com/watch?v=NHJh9KJNyE4
- deleted 4y ago[deleted]
- anonylizard 4y agoIs there a GPT-3 disruptor? All the open sourced models are GPT2 improvements, and GPT2 was open sourced by OpenAI. GPT3/4 is simply too expensive for consumer GPUs, any open sourced versions will have to run on A100s in the cloud, so by nature centralized. Granted, having multiple providers also counts as removing the moat. But BLOOM for example (An attempt at replicating GPT3), no one actually uses. Because its simply too expensive for inferior performance to GPT3 DALLE2 was disrupted, because 1. OpenAI at the time was dumb enough to put a waitlist on something that costed money. They didn't make the same mistake with ChatGPT. 2. Stable Diffusion was not only open sourced, but heavily heavily optimized in parameter count compared to alternative models, making it viable on consumer GPUs.
- GaggiX 4y agoDalle 2 has also been disrupted because OpenAI has heavily nerfed the model, probably by greatly reducing the steps in the upscaler models (Dalle 2 uses diffusion-based upscaler models and therefore very expensive to run), so the images have good coherence but really bad texture, full of artifacts, ironically since the GAN models had the opposite result, very bad coherence and good texture; also OpenAI has introduced very few features and there is no way to finetuned the model as with GPT-3. Meanwhile, the MJ model outputs extremely good images and SD can be conditioned, fine-tuned, etc. in a really versatile way and extremely good quality (if you know what you are doing).
- EGreg 4y agoYeah, for the worse. We will have a ton of bullshit at scale. And the web will be done for.
- jrvarela56 4y agoI hope the arms race makes us smarter. We're going to need AI to sift through all the BS. My hope is that once we're drowning in deepfakes daily, the average user will come to the conclusion that they can't believe stuff they see, and will realize neither what the read nor hear. The transition will be rough.
- visarga 4y ago> We're going to need AI to sift through all the BS. Yes, that's the only way to deal with it. Humans alone can't cope.
- EGreg 4y agoSomehow bombs don’t actually prevent other bombs. People always hope that the offensive tech could be used defensively, but defense is never perfect and even a few that get through can wreak destruction.
- deleted 4y ago[deleted]
- visarga 4y agoI see it like the cat-and-mouse game of viruses and immune system, or shells and armour. We need "AI immunity" to deal with other AIs. It's not going to be solved in one iteration, we got to keep updating it.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- simonw 4y ago"just like how GPT-3 ... immediately disrupted by open-source models as well." Which open source alternatives to GPT-3 have you seen that most impressed you? I've not yet found any that are remotely as useful as GPT-3, at least for the kinds of things I want to use them for (generating SQL queries from human text, summarizing text, that kind of thing)
- simonw 4y agoIn answer to my own question, https://www.youtube.com/watch?v=NHJh9KJNyE4 https://www.youtube.com/watch?v=NHJh9KJNyE4 GPT-NeoX-20B instruct-trained looks very impressive.