23 ms·
GPT4All Chat – Locally-running AI chat application powered by the GPT4All-J
- manyoso 3y agoGPT4All Chat is a locally-running AI chat application powered by the GPT4All-J Apache 2 Licensed chatbot. The model runs on your computers CPU, works without an internet connection and sends no chat data to external servers (unless you opt-in to have your chat data be used to improve future GPT4All models). It allows you to communicate with a large language model (LLM) to get helpful answers, insights, and suggestions. GPT4All Chat is available for Windows, Linux, and macOS. NOTE: Windows and Linux require at least an AVX2 chipset. Both intel and arm macOS machines should work. A future update should allow non-AVX2 chipsets for older machines. Check back frequently and will announce here when it is ready.
- manyoso 3y agoNew version was just released for those who already have it installed with a number of bugfixes and improvements to the model. If you have it, then click "Check for updates" in the drawer that slides out of the hamburger menu button.
- anditherobot 3y agohow to install on linux step by step?
- manyoso 3y agoDownload the installer and run it. That is it.
- senectus1 3y agooh very cool, now make one for mobile phones :-D
- poglet 3y agoSilly question, how does this differ from ChatGPT? Edit: Other then offline functionality.
- manyoso 3y agoIt is entirely different model. Based on GPT-J which is an open source LLM.
- pazimzadeh 3y agoSo is this GPT-4 4 All, or GPT-3 4 ALL?
- jb510 3y agoIt was based on GPT 3.5 Turbo, but it just changed to a new model GPT4All-J which is improved from 3.5 turbo, but still small enough to run locally. It’s still it’s a small model compared to GPT4 so can’t imagine to competes head to head with that best.
- bheadmaster 3y agoI've tried it, I remember it being more akin to cleverbot than GPT. One thing I love about GPT - context recognition - seems somewhat broken, having model reply with random nonsense unrelated to my prompt each time. I really hope it manages to catch up eventually. In terms of privacy, a local LLM is lightyears ahead of a SaaSS LLM controlled by Microsoft.
- manyoso 3y agoThere is a big update released today… maybe give it another go?
- bheadmaster 3y agoWhat exactly was updated, the model weights, the code, or both? Can I find release notes anywhere? I am a little wary of the opaque installer binary that requires root permission...
- manyoso 3y agoAll of the above and the root permission will be fixed jn future release if yuh want to wait
- sbierwagen 3y agoIt's fine tuned on ChatGPT answers, but the actual underlying model is GPT-J, a 6 billion parameter model. Roughly one twentieth the size of GPT-3, and it's capabilities are about what you'd expect from that number. I installed it, asked a few dozen questions, then uninstalled when I got garbled nonsense in response. (Not just wrong answers, but incoherent noise, repeating chunks of training prompts back to me unrelated to the question, random python code, etc)
- master004 3y agoIs this GPT from AliExpress?
- manyoso 3y agoNo; this is an open source project sponsored by nomic.ai
- submeta 3y agoSo there is no censoring in this? But does it work with the whole text corpus that openai‘s models work with?
- Zetobal 3y agoNo
- HeartStrings 3y ago"Mom, I want a GPT” "We already have GPT at home” GPT at home…
- tudorw 3y agoMy daughter and I used GPT to summon up some Sanrio characters for a tea party, it was adorable.
- laserbeam 3y agoThe naming is confusing and evil. 1. This is not based on GPT4. If you're going to include the words "GPT4" in your name and not base it on GPT4, what else can I not trust you about? 2. There's no direct link to the actual model used, just a reference to GPT-J. 3. I searched for GPT-J, huggingface docs describes it as a GPT-2-like model, the repo for it was started in august 2021, and hasn't really had significant updates in a long while. Honestly, I don't want to spend time figuring out if this is worth it, trust is already out the window.
- j45 3y ago4 in the name isn’t terribly misleading. The name could also age well after GPT5 comes out. It stood out to me as “for” more then the current GPT model.
- FooBarWidget 3y agoNot sure why this post was downvoted but I concur. I never read GPT4All as "(GPT4)-all" but as "GPT for all", as in "a ChatGPT-like thing that everyone can run locally". It surprises me somewhat that someone reads this as "related to GPT4", but it annoys me even more that the person also immediately assumes malice.
- laserbeam 3y agoYou can refer to it as GPT for all, but it sure looks like it uses the name to bank on the fact that GPT4 is popular, even if it has no relation to it.
- FooBarWidget 3y agoIt banks on the fact that "GPT" is popular, definitely, but not "GPT-4" (the specific version) in particular.
- harvey9 3y agoI read it as 'for' but I'd still call it a bit misleading regarding what it can do. However, this is a very fast moving topic and a month from now this entire post will be about as relevant as the Saxon Chronicles.
- Oras 3y agoI was skeptical when I noticed the file size for Mac (18.6MB), but it downloads the model during installation (2.5 GB). Text generation wise, it works pretty well. The text is not as fast as GPT3.5 or even GPT4. I noticed it is a lot slower when generating code. I asked it to write a code to search Google and return the first 10 links. Here is the conversation: Prompt: Write a python code to search Google and return the links for the first 10 results Response: Here is the python code to search Google and return the links for the first 10 results: ```python import requests import re url = 'https://www.google.com/search https://www.google.com/search' # set the number of results to display num_results = 10 # set the search query query = 'python' # make the request to Google response = requests.get(url) # extract the links from the response links = response.text.split('\n') # extract the links from the response and sort them in descending order sorted_links = [link.split('\n')[0] for link in links[::-1]] # print the first 10 links print(sorted_links[:num_results]) ``` Prompt: Rewrite the same code in javascript Response: Here is the rewritten javascript code to search Google and return the links for the first 10 results: ```javascript function searchGoogle() { var query = 'python'; var num_results = 10; var url = 'https://www.google.com/search'; https://www.google.com/search'; var response = ''; var links = ''; // make the request to Google var xhr = new XMLHttpRequest(); xhr.open('GET', url); xhr.onload = function() { if (xhr.status === 200) { response = xhr.responseText; links = response.split('\n'); links.sort(); links.forEach(function(link) { links.push(link.split('\n')[0]); }); console.log(links[:num_results]); } }; xhr.send(); } ``` The code above uses XMLHttpRequest to make a request to Google's search engine and extract the links for the first 10 results. The links are then sorted in descending order and printed to the console. edit: formatting
- bananamerica 3y agoDoes it have less censorship than the online version?
- manyoso 3y agoIt isn’t affiliated with openai’s gpt4
- hyperionplays 3y agoI wonder if they will have cluster support, i have a few old fat servers with a few RTX spec GPU's laying around, my own fast ChatGPT running in the garage would be cool.
- speedylight 3y agoI’d say try Open Assistant imo it’s better than gpt4all interms of response quality.
- ospray 3y agoThis plus the the vicuna models is probably as close as you can get in the near term. https://github.com/oobabooga/text-generation-webui https://github.com/oobabooga/text-generation-webui
- manyoso 3y agohttps://youtu.be/8ZW1E017VEc https://youtu.be/8ZW1E017VEc Video by the founder of the project to put in perspective what this is and is not
- 2Gkashmiri 3y agolooks like they already changed GPL to MIT. Oh well, i might be in the minority but i like GPL license for the exact reason people want to move away from it and call it a bad license.
- diffeomorphism 3y agoHonest question: A random .io website with a punny 4all name and "download and run this file. Totally not a virus." How are users expected to determine whether this is a good idea? Edit: Also no info about the author, seemingly no affiliation to openai (despite being called gpt4all), no license info on the page. Click on the github link and find some "nomic-ai" with no info other than an email address. Is it made by a Nigerian prince or something?
- Kelamir 3y agoDepends on whether you trust their base project https://github.com/nomic-ai/gpt4all https://github.com/nomic-ai/gpt4all . gpt4all is actively distributed over torrent, I actually have over 2.0 ratio
- diffeomorphism 3y ago> is actively distributed over torrent, I actually have over 2.0 ratio That, of course, sounds extremely convincing and instantly trustworthy.
- tudorw 3y agothey have around 11k users on Discord, I'm new to Discord but it seems like there and Tiktok are where the juicy bits are. The web is too noisy, unaccountable, slow and broken.
- tudorw 3y agonow 13.5k with 2000 online, the older autoGPT has 3k online and 22k members.
- diffeomorphism 3y ago> I'm new to Discord but it seems like there and Tiktok are where the juicy bits are. You are making this sound even worse.
- kgeist 3y agoIf I'm not mistaken, they use a fork of an old version of llama.cpp I've been running Vicuna locally for several days now using llama.cpp (i.e. CPU only, because my laptop lacks a good GPU). It's not that hard to set it up yourself from scratch. Compiling llama.cpp is straightforward under Linux. The model (13B parameters, 4 bit) can be downloaded from HuggingFace. There are several difficulties, however: 1) Vanilla llama.cpp doesn't appear to have a simple-to-use interface to interact with it from another process (your app), such as REST API (an idea found in Fabrice Bellard's text-synth) -- you want your main app to be decoupled from a process which consumes a lot of CPU and RAM and can crash. I solved it by using llamacpphtmld project which provides HTTP access to the model. It's a pretty simple project and I think of making my own Go wrappers. 2) Vicuna was trained on ChatGPT output so it often responds with garbage such as "As an AI language model..." and refuses to discuss "controversial topics". I solved it with prompt engineering where I add things like "%bot_name% never moralizes", "%bot_name% is rude" etc. (it's not always rude but that somehow stops it from moralizing) 3) it's pretty slow, at least on my laptop (1-3 minutes to process a phrase -- although I include dialog history as well) So far I'm very pleased with the results (aside from the fact that it's very slow) -- the bot feels GPT3-level when you use it as a chatbot, or as a story generator. In fact, in my tests, it feels like it actually exceeds GPT3 in that regard. I run two bots on my IRC channel, one is Vicuna and one is GPT3-based, and I feed them same prompts to compare. Vicuna feels better as a general-purpose chatbot which can talk pretty much about anything, and it's pretty imaginative. GPT3 often refuses to talk about things, and it loses track of our dialog more quickly. Although it's not that hard to set it up, it takes time to get things right, so I thought maybe open-source my findings as some kind of middleware to quickly integrate with other projects (localhost REST API + read-to-use weights and preset prompts). So far most projects I've seen are either very low-level (like llama.cpp) or very high-level (web-based chat).
- ankraft 3y agoThat sounds interesting. How can I obtain this Vicuna model that works with llama.cpp?
- kgeist 3y agohttps://medium.com/@martin-thissen/vicuna-on-your-cpu-gpu-best-free-chatbot-according-to-gpt-4-c24b322a193a https://medium.com/@martin-thissen/vicuna-on-your-cpu-gpu-be... See the section "CPU Installation (GGML Quantised)" You need Python to download the model from HuggingFace using the official API. After that, all you need is the binary file with weights and a compiled binary of llama.cpp P.S. The author seems to have renamed their repo to "eachadea/legacy-vicuna-13b" on HuggingFace
- abrookewood 3y agoThere is a fairly brief technical report discussing the training etc [0]. The abstract: "This preliminary technical report describes the development of GPT4All, a chatbot trained over a massive curated corpus of assistant interactions including word problems, story descriptions, multi-turn dialogue, and code. We openly release the collected data, data curation procedure, training code, and final model weights to promote open research and reproducibility. Additionally, we release quantized 4-bit versions of the model allowing virtually anyone to run the model on CPU." [0] https://tinyurl.com/yj4yj8xe https://tinyurl.com/yj4yj8xe
- tudorw 3y agoCan everyone watch the video linked below and then comment, there are far more interesting aspects to discuss than what it's called. I picked up on no intention of deception.
- Double_a_92 3y agoI swear to god from now on I will just assume anything named GPT (that isn't the official thing) is just bullshit.
- deleted 3y ago[deleted]
- jerrygenser 3y agoGPT is an acronym describing a particular strategy for training a language model. Many models can be trained in this fashion. It's very possible that models not trained by the original inventors of the method will be better than the current models operated by the inventors.
- deleted 3y ago[deleted]
- pxf 3y agoYou can already run it on your browser with webgpu https://mlc.ai/web-llm/ https://mlc.ai/web-llm/
- pluijzer 3y agoI have just installed this to play with this a little and it is actually not, that, bad. I had asked it to write some simple Python code and it gave me correct results. I wouldn't have expected this for a model of this size.
- manyoso 3y agoThis latest release included a ton of bugfixes and a new model. Hope people are pleasantly surprised. Still needs work, but we're making progress quite fast.
- denysonique 3y agoInterestingly, for some reason while trying to access this from the UK I get the following message (which I've never seen before): Dear Customer, We have a legal obligation and a responsibility to ensure that our subscribers are not able to reach sites which are barred by law or that are likely to cause malicious damage to their online reputation or devices. You have tried to access a site at https://gpt4all.io/index.html https://gpt4all.io/index.html which we consider to be harmful. If you consider this to be incorrect and need access to this site, please contact us on address team@internetty.uk or call us on 0333 102 0000 or visit https://www.internetty.uk https://www.internetty.uk and we will review our policy for the site. We apologise for any inconvenience that this restriction may cause you but wish to assure you that we are constantly reviewing threats on the internet to ensure that all our subscribers remain safe.
- vorticalbox 3y agoseems your ISP is blocking the It. The interesting question is this the effect of a) barred by law or b) considered harmful.
- anoncow 3y agoTo ensure credibility and trust are not lost, please rename the project to: GPTFORALL OR OPENGPT OR GNUGPT
- Kelamir 3y agoIs there any explanation to the negativity expressed here over the name? Because GPT4All was posted 20 days ago here https://news.ycombinator.com/item?id=35349608 https://news.ycombinator.com/item?id=35349608 with 593 points and 303 comments, and nobody mentioned it.