8 ms·
Ask HN: Can I download GPT / ChatGPT to my desktop?
I would love to be able to run GPT / ChatGPT on my desktop and remove some of the length limits on text.
How can I do that, and where can I download it from?
- PlotCitizen 4y agoIt’s not possible currently but there’s another story on the HN front page with an open source alternative which I haven’t tried
- 5e92cb50239222b 4y agoOpenAI is 'open' in the name only, so no. I don't think they have any plans of opening full access to the public either, considering that their previous model (that ChatGPT builds upon) was sold to Microsoft for exclusive use: https://en.wikipedia.org/wiki/GPT-3 https://en.wikipedia.org/wiki/GPT-3
- dilap 4y agoIt's really obnoxious for non-open companies to include "open" in the name. Stolen valour, basically.
- blacksmith_tb 4y agoOpen as in "open for business" not as in "open source", you might say.
- throwaway2016a 4y agoMaybe if you compare it to open source. But if you consider that many of these algorithms were historically invented by a private company and kept private / proprietary as a competitive advantage I think the fact OpenAI puts no unreasonable restrictions on who can use it makes it fairly "open" Though, I too, would rather be able to run the model myself.
- dilap 4y agoPublishing research is pretty normal for traditional companies. The pledge to make patents available is a small step towards open. But I think the obvious and natural thing for a company in the business of training machine learning models that claims to be open to do is make the models themselves available. OpenAI does rougly the opposite: not only do you not have acccess to the underlying models, even the API access you are given is to a model itself deliberately trained to avoid answering certain classes of queries. To me that's the opposite of open; it's closed, restricted, and centrally controlled. (Very impressive results tho!)
- throwaway2016a 4y ago> Publishing research is pretty normal for traditional companies. Published research is "open" to the extent that it is transparent but it is not "open" to the extent that it can be used and accessed by people. Unless you are an AI researcher, half these papers (to be generous) might as well not exist. My argument is from that perspective (ability for the average person to use it), academic research only gives the illusion of openness. Not only that but the training data is often -- but not always -- omitted from academic research. So reproducing the exact results they did is often out of reach without a significant investment in building your own collection of training data. For example: Facebook and Google have both announced similar technology to OpenAI yet neither is usable out of the box (or at all for practical purposes) where as OpenAI despite being "closed" I can get started in 5 minutes. Take by contrast to both of those, Stable Diffusion. Which I think is miles ahead of DALL-E... their code and their pre-trained weights are very easy to use as well as being open.
- 4y ago
- AHappyCamper 4y agoTruly a shame. Good marketing tho
- consumer451 4y agoI think a more accurate name might be Alignment AI. [0] I realize it's a charitable interpretation of their behavior, and I used to be very angry about them not being truly open. However, after playing with ChatGPT I think I am beginning to understand and even support their behavior. [1] My personal sea change was realizing that giving dual-use tools to the global @everyone and hoping for the best might not be the greatest plan. I came to this realization thinking about bio-tech and GNC software, but it may apply to ML products as well. [2] While I used to think universally/religiously "it should be open, will be one day anyway, etc," I now think about these things on a case by case basis. [0] https://openai.com/alignment/ https://openai.com/alignment/ [1] https://news.ycombinator.com/item?id=33928400 https://news.ycombinator.com/item?id=33928400 [2] Imagine how many nodes this bot farm would have if it wasn't limited by the existing C&C bottleneck. ChatGPT is a productivity multiplier. This is productivity I want to put off multiplying as long as possible: https://news.ycombinator.com/item?id=34165350 https://news.ycombinator.com/item?id=34165350
- deleted 4y ago[deleted]
- turkeygizzard 4y agoI'm pretty sure the GPT model is huge and does not fit on any conventional GPU. Even if they open-sourced the weights, I don't think most people would be running it at home. Also regarding the text limits, AFAIK, there's just an inherent limit in the architecture. Transformers are trained on finite-length sequences (I think their latest uses 4096 tokens). I have been trying to understand how ChatGPT seems to be able to manage context/understanding beyond this window length
- Sharlin 4y agoI don't think ChatGPT does. I have had long discussions with it, with some rules agreed upon in the beginning, and at some point it clearly begins to forget the exact rules and has to be reminded of them. (Specifically, AI Dungeon type games where ChatGPT is the DM and the human the protagonist, or vice versa. The most common failure mode seems to be that it forgets whether it's playing the DM or the protagonist. To be fair, it performs admirably well despite the limitations.)
- rzzzt 4y agoIn a previous thread (which I can not find right now) the recommendation was to either ask it to summarize what happened earlier, or do this job yourself from time to time.
- puffybuf 4y agoI read that it just re-reads the discussion so far every time you submit. So it must hit a limit of what it can remember since they limit the amount of training tokens it can read for a submission.
- Sharlin 4y agoYes, I know. It’s a pure function with no mutable state.
- throwaway2016a 4y agoIs Chat-GTP it's own model? I thought ChatGTP was just GTP-3 with an easier to use interface.
- luckyme123 4y agoLuckily, no. Otherwise you (others) could hack the safeties and ask it how to cheaply kill a lot of people or so .. better not make bad people too intelligent! (Obviously not talking about you)
- PM_me_your_math 4y agoRename it and ask it to find & kill the virus it created?
- xaml 4y agoI don't think that I can PM you, but could you explain how Green's theorem can be applied to regions with singularities?
- PM_me_your_math 4y agoIt was a (bad) SkyNet joke, quoting the third terminator movie.
- burkaman 4y agoIt's not magic, it doesn't know anything that isn't easily findable public information.
- Fatnino 4y agoAnd much of that it doesn't know either. "in Google sheets, I have dates in the first column. How would I make the second column indicate if daylight savings time is active on that date?" ChatGPT will confidently provide non working suggestion after non working suggestion, but ultimately can't make this work.
- Sharlin 4y agoEven if it were freely available, there's no way to run GPT3 or ChatGPT on any existing desktop hardware. The exact hardware requirements aren't public either (yes, very "open") but a full 175-billion-parameter GPT3 instance requires hundreds of gigabytes of GPU memory, and even though ChatGPT is "smaller and better", when it comes to conversational dialogue, there's no way to fit it in current consumer hardware.
- lambdaba 4y agoWill it ever be an option, perhaps the first option, given some evolution in the software and/or hardware?
- Sharlin 4y agoEver? Likely. Hardware keeps improving, and better training techniques will almost certainly keep shrinking model sizes ceteris paribus. But one should also remember that these are static models that can't learn anything that was not present in the original training corpus, so for some use cases that rely on current information they're simply not a good match. And training a model like this requires vastly more hardware (and human) resources than just using it. Never mind the issue of collecting a corpus in the first place.
- htns 4y agoI don't much follow AI news beyond what I randomly happen to see on HN, but this might still be the largest open source model: https://github.com/yandex/YaLM-100B https://github.com/yandex/YaLM-100B . There's discussion of it here: https://old.reddit.com/r/MachineLearning/comments/vpn0r1/d_has_anyone_got_yalm100b_to_run/ https://old.reddit.com/r/MachineLearning/comments/vpn0r1/d_h... - at the bottom of that page is a comment from someone who actually ran it in the cloud.
- mellosouls 4y agoIt's not possible post GPT2 for the reasons given by others. Open communities with potential for involving yourself include Hugging Face and EleutherAI, the former perhaps more accessible, the latter an active Discord. It's been a while since I spent time looking at them, I'm not sure if there is something you can easily get up and running with. https://huggingface.co/ https://huggingface.co/ https://www.eleuther.ai/ https://www.eleuther.ai/
- chamwislo 4y ago[dead]
- navjack27 4y agoYou can do gpt-j https://gist.github.com/navjack/32197772df1c0a8dbb8628676bc4e25a https://gist.github.com/navjack/32197772df1c0a8dbb8628676bc4... I mean yeah after you set it up like this you still have to prompt engineer to get it to behave like a chat but I mean it's better than GPT - 2
- dragonwriter 4y agoThere are non-OpenAI models based on the same GPT paper as the OpenAI GPT-series, e.g., GPT-NeoX [0], GPT-J, etc., that are actually Open Source, unlike OpenAI which is “open” only in the sense of “we might let you use it, either as a free preview or a paid service”. You probably won't be able to run (or especially train) them on typical desktops, though. [0] https://www.eleuther.ai/projects/gpt-neox/ https://www.eleuther.ai/projects/gpt-neox/
- qup 4y agoAre they good?
- speedgoose 4y agoIt depends on what you use them for. To replace ChatGPT, no they are not good enough.
- beckingz 4y agoWith some effort they're close to ChatGPT in terms of quality. More of one off input output as opposed to a conversation though.
- dragonwriter 4y agoThat... depends. If you are looking at a pretrained model that competes with ChatGPT out of the box, no. GPT-NeoX apparently compares favorably with GPT-3 on some measures, but ChatGPT is a 175 billion parameter GPT-3 model, and the big pretrained GPT-NeoX model available is a 20 billion parameter model. Could you rival ChatGPT with the right settings and training set and sufficient hardware for training? Well, if you want to try, you can with GPT-NeoX, and you can chosr whether or how to filter the output. With OpenAI’s models, you get what they give you, on the terms they are willing you give you access to it, with filters that exist tonprotect OpenAI’s image and liability.
- chamwislothe2nd 4y agohttps://github.com/bigscience-workshop/petals https://github.com/bigscience-workshop/petals Since my other account is shadow banned for some unexplained reason, I just wanted to mention the petal project. It's an attempt to bittorrent style distribute the load of running these large models. Good luck!
- fred967 4y agoYou can download his ancestor here: https://winworldpc.com/product/dr-sbaitso/2x https://winworldpc.com/product/dr-sbaitso/2x
- rzzzt 4y agoHELLO FRED967, MY NAME IS DOCTOR SBAITSO. I AM HERE TO HELP YOU. SAY WHATEVER IS IN YOUR MIND FREELY, OUR CONVERSATION WILL BE KEPT IN STRICT CONFIDENCE. MEMORY CONTENTS WILL BE WIPED OFF AFTER YOU LEAVE, SO, TELL ME ABOUT YOUR PROBLEMS. That falling intonation is very reassuring.
- trilbyglens 4y agoMy feeling is that even if it were available to download, the compute requirements to run it at production speeds would likely be blistering.