3 ms·
I'm still waiting for Hacker News to band together and simply start building an open model that does not require a corporate daddy like OpenAI, Microsoft, Baidu
by aliljet 4y ago
I'm still waiting for Hacker News to band together and simply start building an open model that does not require a corporate daddy like OpenAI, Microsoft, Baidu, Google, or anyone else. These models have no place being owned by corporations. The services that use these models are certainly up for grabs, but the models should be open. Who is trying to rally the community to this cause?
- binarymax 4y agoWell, there's the new GPT-Neo models and BLOOM models, and you can spin them up with something like this: https://github.com/togethercomputer/OpenChatKit https://github.com/togethercomputer/OpenChatKit
- rcarmo 4y agoHN seldom bands together to do anything but quip at things, or discuss how it should have been done better. Oh, wait.
- sho_hn 4y agoMore exciting software projects get started on 4chan than Hacker News.
- morsch 4y agoMost people around here have a job.
- sho_hn 4y agoOr want to have/generate one. What I mean is, HN is a place where people keep good ideas to themselves in the hopes of launching a business on its back. It's a hustling/entepreneurship community, not a "let's build stuff" one, like, say, the Open Source communities. Collaboration has an opportunity cost around these parts.
- rcarmo 4y agoI'll take "exciting" in technical terms rather than "exciting" in legally-challenging, raging mob ones...
- deleted 4y ago[deleted]
- zer0-c00l 4y agoWhy not you?
- jeffbee 4y agoThis isn’t folding@home. These models need as near to synchronous parameter updates as possible. You need a real facility to do this.
- sebzim4500 4y agoYeah but we could definitely crowdsource enough money to make an extremely effective finetuned version of LLaMA. The challenge there is probably more about finding and cleaning data.
- hiddencost 4y agoTake a peek at EleutherAI.
- janeway 4y agoDidn’t they spend hundreds of millions running the training data or something? How are some open-source hackers going to do it?
- kristjansson 4y agoThere are at least two completely open models comparably scaled to GPT-3, and at least one open distributed-hosting project.
- Steve0 4y agoCheck out Llama, Alpaca, Langchain, Dalai.
- deleted 4y ago[deleted]
- deleted 4y ago[deleted]
- spacephysics 4y agoFully agree, we need a competitive (even if slightly behind) truly open initiative. The potential (and current) censorship of these models is an oligarch’s wet dream. With google vs DuckDuckGo etc you can see what websites are returned, and get a sense of the kind of authoritative sources used With this kind of tech, there’s no real way to know except by comparing precise prompts with one another. But if they’re all behind corporate walls, and they share incentives to say similar things, we’re none the wiser
- feyman_r 4y agoBing Chat provides sources as in its response. I think that is not perfect, but directionally correct in terms of identifying what web results were used as authoritative sources.
- ludocode 4y agoI don't know about Hacker News, but there is a community building an open large language model using compute time donated by cloud hosting companies: https://en.wikipedia.org/wiki/EleutherAI https://en.wikipedia.org/wiki/EleutherAI They built GPT-J and GPT-NeoX.
- tensor 4y agoThere is https://github.com/BlinkDL/ChatRWKV https://github.com/BlinkDL/ChatRWKV you could try.
- briandear 4y agoHave you set up the GitHub repository and done the initial commit yet? If you think “we” should take the first step, then go-ahead, take it. Lead the way. I am being semi-snarky because often people talk about some grand ideas about things “we” should do, but rarely do those advocating for some utopian outcome actually do anything towards achieving it. If it is to be, it’s up to me.
- TylerE 4y agoConsidering that the CEO of OpenAI is the former president of Y Colbinator…