4 ms·
I'm hoping of one day running GPT3/ChatGPT on my local computer, similarly to how one can run Stable Diffusion now. I would love to have a personal conversatio
by anhner 4y ago
I'm hoping of one day running GPT3/ChatGPT on my local computer, similarly to how one can run Stable Diffusion now.
I would love to have a personal conversation with these AI systems, use them as a sort of assistant, without the worry of being spied on. At the moment I can't use it as more than a glorified search engine, because of the privacy implications of running it on the cloud.
- jillesvangurp 4y agoThere will always be this uncomfortable balance between need to know information to be useful being essentially the same as everything an adversary would need to really act against you. With something like chat gpt being plugged into some voice assistant thing and having access to all your documents, emails, and other content, you could imagine having conversations about work content, content creation, calendar management, etc. Basically it would become like a secretary that is able to write letters based on your input, manage your calendar, etc. It could be pro-active and remind you about things, summarize incoming messages, search through your documents, message history, etc. That's where AI becomes really useful. But the issue of trust is a big one. I don't think a lot of this requires a lot of breakthroughs either just a lot of integration work and engineering. Chat gpt is more a proof of concept than a well integrated thing at this point. It basically is running in isolation and it's only window to the world is chat. Changing that should not be that hard. Running things locally might help with this but it may not be a hard requirement for this. All depends on how useful this is.
- jacooper 4y ago> That's where AI becomes really useful. But the issue of trust is a big one Well google already has access to all of this data, the difference is unlike google assistant, ChatGPT actually can do something useful.
- spi 4y agoThere's not much of an alternative: even if compute power gets extraordinarily cheap / models are very optimized, either you have a very, very large hard drive, or you have to use the internet for that. You just can't hope to have a model that is trained to know everything in a smallish file, the weights need to be at least as heavy as an ideally compressed version of all the things it knows (which is of course much less than the huge amount of data it is trained on, mainly due to redundancy, but still a lot) Of course, right now you also need at least 8 super-expensive A100 GPUs and not just your laptop CPU, but maybe that's going to change eventually.
- anhner 4y agoDoes it need all that much space though? I mean Stable Diffusion was trained on hundreds of terrabytes of images, and the model only needs 4-5 GB of hardware space and a decent GPU to run. I haven't read anywhere any stats about GPT3/ChatGPT yet (like how big the model is)
- _pdp_ 4y agoThese models are way too big for consumer hardware.
- anhner 4y agoIs it bigger than the Stable Diffusion model? (4-5 GB)
- danielbln 4y agoThere will always be an edge to these massively operated central models that you can't easily run at home, due to compute and cost. Things like 4-bit quantization will help making these foundational models significantly easier to operate on smaller hardware, and of course you might not need a foundational model for manby local usecases, but maybe a smaller, more specialized LLM is enough (those can even be trained by the bigger models).