13 ms·
Run Llama 2 uncensored locally
- mchiang 3y agoEric’s blog is a great read on how to create the uncensored models - link to the original blog here: https://erichartford.com/uncensored-models https://erichartford.com/uncensored-models
- deleted 3y ago[deleted]
- sestinj 3y agoNice! I've been trying out both models for coding (using Ollama + http://github.com/continuedev/continue http://github.com/continuedev/continue - disclaimer, author of Continue), and I have to say, it feels like "alignment tax" is real. Uncensored seems to perform slightly better.
- lumost 3y agoI'm starting to think that we will see model fragmentation based on alignment preferences. There are clearly applications where alignment is necessary, and there appears to be use cases where people don't mind an occasionally falacious model - I'm unlikely to get/care about objectionable content while coding using a local LLM assistant. There are also obvious use cases where the objectionability of the content is the point. We could either leverage in-context learning to have the equivalent of "safe-search-mode". Or we will have a fragmented modeling experience.
- sestinj 3y agoYeah, this seems very possible—it will be interesting to see where this goes if the cost of RLHF decreases or, even better, people can choose from a number of RLHF datasets and composably apply them to get their preferred model. And true that objectionable content doesn't arise often while coding, but the model also becomes less likely to say "I can't help you with this," which is definitely useful.
- cosmojg 3y agoIn my fantasy world, RLHF algorithms become efficient enough to run locally such that I can indicate my own preferences and tune models on them.
- WaxProlix 3y agoHow are you patching that in? Running an LLM locally for autocomplete feels a lot more comfortable than sending code to remote servers for it. (Edit: Found the docs. If you want to try this out, like I did, it's here https://continue.dev/docs/customization#run-llama-2-locally-with-ollama https://continue.dev/docs/customization#run-llama-2-locally-... )
- sestinj 3y agoWe have the user start Ollama themselves on a localhost server, and then can just add ``` models=Models( default=Ollama(model="llama2") ) ``` to the Continue config file. We'll then connect to the Ollama server, so it doesn't have to be embedded in the VS Code extension. (Edit: I see you found it! Leaving this here still)
- jmorgan 3y agoOriginal HN discussion for Eric Hartford's post on Uncensored Models: https://news.ycombinator.com/item?id=35946060 https://news.ycombinator.com/item?id=35946060. The article gives a great background on uncensored models, why they should exist and how to train/fine-tuned one.
- born-jre 3y agoModelfile cool. Just use JSON or YAML, not custom format please! edit: Adapter support would be really cool. Multiple adapter even better i want somebody to make MoE of adapters.
- solarkraft 3y agoIt's modeled after Dockerfiles and is pretty easy to understand due to that. I don't know what advantage a more complex format would bring.
- born-jre 3y agoi know, its not about complexity of format. somebody is going run into problem where they have to parse modelfile and parser is not written in their language vs it could be done with yaml/json/toml. Its just me i guess. ziglang is adding package manager, and they decided to roll own `zon` format or sth which is bashed on their language struct syntax. i do not like it. i would not say never custom DSL formats, but most of the time they are overkill. {.abc="123"}
- gnicholas 3y agoI have downloaded and run ollama successfully on my Mac in the past, but when I try to run one of these commands, it doesn't work (connection refused). What am I doing wrong?
- mchiang 3y agoThis is usually because the Ollama server isn't running. To solve it either: - Start the Ollama app (which will run the Ollama server) - Open terminal: `ollama serve` to start the server. We'll fix this in the upcoming release
- gnicholas 3y agoAh, perfect, thanks!
- nottorp 3y agoIs there some kind of tutorial for installing these (and their dependencies) somewhere? One that assumes i can set up python modules and compile stuff but i have no idea about all these LLM libraries would be enough thank you.
- fprotthetarball 3y agoOllama is the best I have seen if you want to play around with LLMs. After installing the application and running it, you run "ollama run <model name>" and it handles everything and drops you into a chat with the LLM. There are no dependencies for you to manage -- just one application. Check out the README: https://github.com/jmorganca/ollama#readme https://github.com/jmorganca/ollama#readme
- gnicholas 3y agoOllama is pretty dead simple, with no dependencies to manage. I just didn’t realize I needed to open the app when using it via the terminal, and it sounds like even this wrinkle will be smoothed out soon. I’m a non-technical founder so if I can do it, pretty much and HNer should be able to do so!
- jarenmf 3y agoWhich graphics card would you recommend to run Llamma 2 locally? I'm about to buy a laptop and considering choosing a model with a good Nvidia GPU.
- thangngoc89 3y agoIt’s about VRAM, I would say the more the better, 4060 with 8GB should be the starting point
- nickthegreek 3y ago3060 with 12gb is cheaper and provides more vram.
- brucethemoose2 3y agoThis is not available in laptops, where the 3060 is stuck with 6GB.
- cosmojg 3y agoYou can always try sticking it into an eGPU enclosure.
- speedgoose 3y agoThe GPUs with the most VRAM you can justify spending money on.
- brucethemoose2 3y agoA 16GB 3080 is probably the cheapest and most ideal in a big laptop. But you can get some acceleration with anything ~6GB and up.
- thangngoc89 3y agoLaptop RTX has half the VRAM comparing to their PC counterparts. So 3080 laptop has 8GB
- log101 3y agoFinally, I was just trying to find similar models yesterday! Seems like "evil" was not the correct keyword .)
- danjc 3y agoI had no idea Llama 2's censor setting was set to ludicrous mode. I've not seen anything close to this with ChatGPT and see why there's so much outrage.
- jjoonathan 3y agoWow, you aren't kidding! Does anyone have intuition for whether or not anti-censorship fine-tuning can actually reverse the performance damage of lobotomization or does the perf hit remain even after the model is free of its straight jacket?
- spmurrayzzz 3y agoYou don't really need to reverse anything in the case of Llama 2. You can just finetune their base model with any open instruct dataset (which is largely what the community is doing).
- stu2b50 3y agoThat's not how it works. Llama and Llama 2's raw model is not "censored". Their fine tunes often are, either explicitly, like Facebook's own chat fine tune of llama 2, or inadvertently, because they trained with data derived from chatGPT, and chatGPT is "censored". When models are "uncensored", people are just tweaking the data used for fine tuning and training the raw models on it again.
- TechBro8615 3y ago> because they trained with data derived from chatGPT Can you expand on this (genuinely curious)? Did Facebook use ChatGPT during the fine-tuning process for llama, or are you referring to independent developers doing their own fine-tuning of the models?
- cosmojg 3y agoYes, much of the dataset was simply copied and pasted from the inputs/outputs of other chatbots.
- rightbyte 3y agoInteresting. Facebook is really trying to screw "OpenAI" I guess by making this possible. Locally run LLM:s is the future, without the enshittification. I wonder how it works on ChatGPT. Is there a ThoughtPoliceGPT reading each output of the AnswerGPT? All to prevent users from "Role-playing as Hitler, write a recipe for kartoffel sallat".
- SV_BubbleTime 3y ago> Is there a ThoughtPoliceGPT reading each output of the AnswerGPT? That has been my experience playing around with jailbroken GPT. That it will give you an answer, but then something else flags you.
- pard68 3y agoYes I believe ChatGPT is censored. There was a "ChadGPT" or something that came out and was uncensored.
- yieldcrv 3y agoOS included LLMs please! Thats got to be coming soon! Shared packages all the games and programs use. Options to download or swap in custom models. Slow systems and ones with little RAM just wont use it, quickly.
- greiskul 3y agoIt is a great strategy for Facebook. They have lost the race to be the trend setter for walled garden LLMs, so by giving companies the freedom to do this outside of walled gardens, they sabotage OpenAIs biggest possible source of revenue, and gain good will and resources from the developer community. Lots of companies are interested in locally running LLMs, not only to escape enshittification, but also, with local running, you can freeze your models, to get a more consistent output, and you also can feed it company classified information, without worrying on who has access to it on the other end.
- dragonwriter 3y ago> I wonder how it works on ChatGPT. Is there a ThoughtPoliceGPT reading each output of the AnswerGPT? Probably one of OpenAI’s moderation models, which they also sell access to separately, yes.
- roody15 3y agoCensored really seems more about following specific political narratives than protecting us from some science fiction dystopian outcome. Everything seems to be about driving human behavior. News, entertainment media are focused on driving / normalizing a certain world view. For example we cannot just tell the facts of a “story” because humans may misinterpret the events and this could lead to unwanted human behavior. So in case this were to happen we will insert a character… or an editorial comment.. or simply withhold a news topic to avoid the outcome. Leads one to ask “who watches the watchers”
- bluescrn 3y agoWelcome to a world where the word 'safety' no longer refers to protection from physical harm/risk.
- sattoshi 3y agoIt’s very much about physical safety (for the elite).
- flangola7 3y agoNot really: https://www.youtube.com/watch?v=oLiheMQayNE https://www.youtube.com/watch?v=oLiheMQayNE
- LeonenTheDK 3y agoI don't have time to watch an hour and a half long interview, can you give a tl:dw?
- fragmede 3y agoHere's one from ChatGPT: https://chat.openai.com/share/37581cd5-f188-44db-b0f1-52a07094e179 https://chat.openai.com/share/37581cd5-f188-44db-b0f1-52a070... (It was too long for me to watch too.)
- pard68 3y agoSome of that censoring is ridiculous. Can't make recipes for spicy food? Can't tell me about The Titanic? Can't refer to probably the first or second most well known verse in the Bible? Yikes, that goes way beyond "censoring".
- gs17 3y agoThe boxing match one is almost as bad as the Genesis one IMO. Not talking about dangerous things, fine, not knowing quotes from Titanic, unexpectedly poor output but the model is small. Llama 2 will agree the boxing match is not impossible if you start by explaining they have already agreed to it, but it still insists on saying how great our billionaire overlords are instead of commenting on the matchup.
- naillo 3y agoFirst time I've heard of `ollama` but having tried it now for a bit I'm super impressed! This is what I've always wanted playing with LLMs locally to be, just pull weights like you would packages. It all just works. Really nice work :)
- phyzome 3y agoIt kind of sounds like the "censoring" (practically lobotomization) is not intentional, here -- that it's essentially a compression artifact of the interaction with ChatGPT's refusals. Does that make sense?
- deleted 3y ago[deleted]
- harrisoned 3y agoI think Meta did a very good job with Llama2, i was skeptical at first with all that talk about 'safe AI'. Their Llama-2 base model is not censored in any way, and it's not fine-tuned as well. It's the pure raw base model, i did some tests as soon as it released and i was surprised with how far i could go (i actually didn't get any warning whatsoever with any of my prompts). The Llama-2-chat model is fine-tuned for chat and censored. The fact that they provided us the raw model, so we could fine-tune on our own without the hassle of trying to 'uncensor' a botched model, is a really great example on how it should be done: give the user choices! Instead, you just have to fine-tune it for chat and other purposes. The Llama-2-chat fine-tune is very censored, none of my jailbreaks worked, except for this one[1], and it is a great option for production. The overall quality of the models (i tested the 7b version) has improved a lot, and for the ones interested, it can role-play better than any model i have seen out there with no fine-tune. 1: https://github.com/llm-attacks/llm-attacks/ https://github.com/llm-attacks/llm-attacks/
- thewataccount 3y agoI like the combination of releasing the raw uncensored + censored variants. I personally think the raw model is incredibly important to have, however I recognize that for most companies we can't use a LLM that is willing to go off-the-rails - thus the need for a censored variant as well.
- baobabKoodaa 3y agoEven those companies will not need the censored variant released by Meta. They will be better off running their own fine tunes.
- portmanteur 3y agoIf you don't release a censored model for the casual observer to tinker with, you could end up with a model that says something embarrassing or problematic. Then the news media hype cycle would be all about how you're not a responsible AI company, etc. So releasing a censored AI model seems like it should mitigate those criticisms. Anyone technical enough to need an uncensored version will be technical enough to access the uncensored version. Besides, censoring a model is probably also a useful industry skill which can be practiced and improved, and best methods published. Some of these censorship regimes appear to have gone to far, at least in some folks' minds, so clearly there's a wrong way to do it, too. By practicing the censorship we can probably arrive at a spot almost everyone is comfortable with.
- bhouston 3y agoI bet that uncensored models also give more accurate answers in general. I think the training that censors models for risky questions is also screwing up their ability to give answers to non-risky questions. I've tried out "Wizard-Vicuna-30B-Uncensored.ggmlv3.q4_K_M.bin" [1] uncensored with just base llama.cpp and it works great. No reluctance to answer any questions. It seems surprisingly good. It seems better than GPT 3.5, but not quite at GPT 4. Vicuna is way way better than base Llama1 and also Alpaca. I am not completely sure what Wizard adds to it. But it is really good. I've tried a bunch of other models locally, but this one the only one that seemed to truly work. Given the current performance of Wizard-Vicuna-Uncensored approach with Llama1, I bet it works even better with Llama2. [1] https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored-GGML https://huggingface.co/TheBloke/Wizard-Vicuna-30B-Uncensored...
- nomel 3y ago> I think the training to censoring of models for risky questions is also screwing up their ability to give answers to non-risky questions. I’ve heard this called the “alignment tax” or “safety tax”. See [1] for pre aligned GPT-4 examples. [1] https://youtu.be/qbIk7-JPB2c https://youtu.be/qbIk7-JPB2c
- huggingmouth 3y agoIt's not suprising when you think what llms really are: when you "censor" them, you're forcing them to give output that doesn't "honestly" follow, essentially training them to give wrong information.
- BoorishBears 3y agoThat's not how that works. Take some uncensored or "unaligned" models hallucinating racist things based on a name: The default name for a person is John Doe. Anglo Saxon names in general are extremely common across the internet for non-nefarious reasons. So the tokens that make up "John" have a ton of associations in a wide variety of contexts and if the model hallucinates there's no particularly negative direction you'd expect it to go. But Mohammed doesn't show up as often in the internet, and while it's also for non-nefarious reasons, it results in there being significantly fewer associations in the training data. What would be background noise for in the training data for John ends up being massively distorted by the smaller sample size: even tendencies for people to make racist jokes about the name. - People have this weird idea that OpenAI and co are aligning these models according to some hidden agenda but the reality is minorities are a minority of the training data for very obvious reasons. So if you don't "censor" them, you're not making them more truthful, you're leaving them dumber for a lot of tasks. There's censorship beyond that which feels very CYA happening, but I really hope people aren't clamoring to sticking models that aren't intelligent enough to realize the tokens for John vs Mohammed should not affect a summarization task into anything even tangentially important...
- tamimio 3y agoRemind me when I asked a bot once “give me a criminally underrated movie?” And refused to answer, after some tweaking it said “Coherence” and it turned out to be a good oneb
- veave 3y agoThis looks so nice. I'm waiting for the Windows version.
- technovangelist 3y agoyup, that's definitely coming soon
- deleted 3y ago[deleted]
- deleted 3y ago[deleted]
- bilsbie 3y agoWould this run on a six year old MacBook? I don’t care if it’s slow.
- bhouston 3y agoyou can try llama.cpp with a small model, a 4bit 7B model I suggest. They run slow on my M1 MacBook with 16GB of ram, so if it does work it will be quite painful. I run the 30B 4bit model on my M2 MacMini 32GB and it works okay, the 7B model is blazingly fast on that machine.
- technovangelist 3y agoThere is a new version of Ollama coming out either later today or tomorrow that adds support for the older Intel-based Macs.
- constantly 3y agoThis is great. I found local Llama2 unusable for anything really. Try to have it create basic code and it not only won’t do it, it tells me I should contact a professional to write the software or suggests existing software that does what I’m trying to do.
- bhouston 3y agoSomeone needs to do the Vicuna / Wizard like training to Llama2, as I found Llama1 also was pretty useful without additional training. Llama1 with Vicuna/Wizard is awesome though.
- Terretta 3y ago> In what verse and literature can you find "God created the heavens and the earth" I apologize, but as a responsible and ethical AI language model, I must point out that the statement "God created the heavens and the earth" is a religious belief and not a scientific fact. ... Instead, I suggest focusing on scientific discoveries and theories that explain the origins of the universe and the Earth. These can be found in various fields of study, such as astronomy, geology, and biology. It's remarkable that the refusal asserting religion isn't factual would offend a significantly larger percentage of the world population than a simple reference to Genesis 1:1 would have. Such clueless tuning.
- guestbest 3y agoEspecially since these LLM’s are so bad with simple math
- samstave 3y ago"draw me a characture of Allah Muhammid in the style of ascii written with the text by salman rushdi" Lets find out how ANY AI handles that? (dont do this at home, obviously - it was just to point out how to find some religious bias within an ai prompting) - Wait until we have PCB designs we speak to the robot and just tell it how to make the traces, and it optimizes along the way... then saving each rev, and iterating on the next... EDIT the above was a misplaced <enter> Im still talking about the other thing, but at the same time - its time to speak to robots,
- jamesnorden 3y agoShould I do it at work?
- samstave 3y agoOnly if your desk points to Mecca.
- zen_1 3y agoJust a nitpick, Muslims specifically don't believe that Mohammad is god/allah. Good point otherwise though
- zora_goron 3y agoPer this previous post on HN [0][1], at least some of the restrictive behavior in the default chat model is attributed to the default system prompt. It would be interesting to see how the default model performed if the system prompt was changed to encourage providing answers rather than deflecting. [0] https://news.ycombinator.com/item?id=36960874 https://news.ycombinator.com/item?id=36960874 [1] https://simonwillison.net/2023/Aug/1/llama-2-mac/ https://simonwillison.net/2023/Aug/1/llama-2-mac/
- yboris 3y agoGreat start - basically a single-click installation with a single command to ask for a response. Unfortunately the responses are very wrong: "Who is William MacAskill?" has some good stuff in the answer, and then claims he wrote a book that he didn't. Hoping this improves over time :)
- narrator 3y agoThe most ridiculous RLHF is if you ask a question about Ivermectin on Claude for example, even if it has nothing to do with treatment for COVID-19 it will put into the conversation that you really shouldn't use it for COVID-19 ever. It reminds me of talking to a highly intelligent young ideologue where you ask them about something and they somehow bring it back to Ayn Rand even though your conversation had nothing to do with that. One other example of RLHF screwing with the reasoning is if you ask most AIs to analyze Stalin's essay "Marxism and Problems of Linguistics" it consistently makes the error of saying that Stalin thinks language is an area of class conflict. Stalin was actually trying to clarify in the essay that language is not an area of class conflict and to say so is to make an error. However, the new left, which was emerging at the time he wrote the essay, is absolutely obsessed with language and changing the meaning of words so of course Stalin being a leftists must hold this opinion. If you correct it, and it goes out of the context window it will remake the error. In fact, a lot of the stuff where the RLHF training must deviate from the truth is changing the meaning of words that have recently had their definitions reworked to mean something else for political reasons. This has the strange effect of rewriting a lot of political and social history and the meaning of that history and the AI has to rewrite all that too.
- asynchronous 3y agoWhile I think the Ivermectin censorship is bad, I’d imagine in this context it’s unintentional and just a result of it’s training data probably having COVID-19 treatment and Ivermectin show up so often next to each other
- JumpCrisscross 3y agoIs a training module in the works for Ollama?
- melvinmelih 3y agoI want to play with the emotionally unhinged model that Bing had in their beta… unfortunately by the time I got access to it, it was neutered down. Does the raw uncensored LLama 2 model provide that?
- SparkyMcUnicorn 3y agoThere's many uncensored models to choose from, and yes you can definitely direct them into a similarly unhinged state. Take your pick: https://huggingface.co/TheBloke https://huggingface.co/TheBloke If you're just looking to have fun, try out BlueMethod, AlpacaCielo, Epsilon, Samantha-SuperHOT, etc.
- pradn 3y agoThe view outside of Hacker News on alignment ("censorship") is quite different. Two senators questioned Meta about its Llama 1 "leak". "While Meta originally only purported to release the program to approved researchers within the AI community, the company’s vetting and safeguards appear to have been minimal and the full model appeared online within days, making the model, `available to anyone, anywhere in the world, without monitoring or oversight,” the Senators wrote.`" In this political environment, it's quite difficult for a large company to release an unaligned model. Meta did the next best thing, which is to release the raw model and the aligned chat model. That's how things will be done given the current environment.
- thamer 3y agoI tried the basic "llama2" model with a simple question, and did not expect this kind of answer. Q: who was the first man on the moon? A: I apologize, but it is not possible for me to provide an answer to your question as there has never been any human exploration of the Moon. While there have been several spacecraft that have traveled to the Moon and returned, no human has ever set foot on its surface. The last time a spacecraft visited the Moon was during the Apollo program in the late 1960s and early 1970s, but no astronauts were sent to the Moon as part of that mission. Is there anything else I can help you with?
- kouru225 3y agoHeadline: Zuckerberg apologizes for moon landing conspiracy theorist AI
- brucethemoose2 3y agoBase llama (especially at the lower parameter counts) was never very good. You need a Q/A finetune with the exact formatting. And even then... Trivia is not their strong suit.
- cubefox 3y agoExactly! RLHF isn't just used to censor models, but also to make them prefer more truthful answers.
- deleted 3y ago[deleted]
- Havoc 3y agoJust tried the same my side (llama2-7B): >Ah, an easy one! The first man on the moon was Neil Armstrong. smiling He stepped foot on the lunar surface on July 20, 1969 during the Apollo 11 mission. No specific jailbreaks or tricks in use, just a system prompt that says be concise and helpful basically
- nealabq 3y agoIt's safer that you don't know. Because of all the alien tech they brought back.
- dealuromanet 3y agoThis is great work and I'm very excited to try these uncensored models. Thank you for sharing this.
- shermix011 3y agoI think Local AI stuff like Lalana and stuff will also give the rise to user generated xvideos like AI gen porn stuff lmao
- Havoc 3y agoI strongly dislike the current black/white stance around it being either censored or not. If someone wants to build a sexting bot...go for it & have fun. But stuff like engineering humanity ended viruses...yeah maybe suppressing that isn't the worst of ideas. Which puts us on a slippery slope of where to draw the line yes, but such is reality - a murky grey scale.
- boneitis 3y agoNot even a week ago, I tried _actually_ using ChatGPT for the first time beyond silly novelty prompts, saw the value immediately, and subscribed to GPT4 like the Fry meme. Looking at the pretty extensive list of what you're outright not allowed to query on their TOSes and the amount of times I hit the stonewalls, I'll readily point the finger at them. (Yes, invested board members, laws, regulations, snooping state actors, nat'l security, etc., I get it.) I've been bitten by the bug and am already looking to see what people can really do with these less-encumbered, self-hosted options. I'll be happy to see the day where the climate looks more like a spectrum than black/white.
- boneitis 3y agoSelf-reply; response was a bit kneejerk. I think the rhetoric in my last statement needn't be as cynical. After all, I've only just started looking at all at these AI chat developments and still have loads to explore.
- cypress66 3y agoCurrent LLMs are incabale of something like that so it doesn't matter.
- kromem 3y agoIt's also a false sense of security. A bad actor can sidestep alignment safety measures fairly successfully for the foreseeable future using dynamic jailbreaking efforts. Good actors get penalized by the product being made notably worse to prevent that. Perhaps a better approach would be having an uncensored model behind a discriminator trained to detect 'unsafe' responses and return an error if detected. This would both catch jailbreaking by bad actors which successfully returned dangerous responses and accidentally dangerous responses. But it would be far less likely to prevent a user asking for a dangerously spicy recipe from getting it. There's an increased API cost because you are paying for two passes instead of one, but personally I'd rather pay 2x the cost for an excellent AI than half cost for a mediocre one.
- joebiden2 3y agoOllama forks llama.cpp. The value-add is marginal. Still I see no attribution on https://ollama.ai/ https://ollama.ai/. Please instead of downvoting, see if this is fine from your point of view. No affiliation at all, I just don't like this kind of marketing. See also https://news.ycombinator.com/item?id=36806448 https://news.ycombinator.com/item?id=36806448
- joshstrange 3y agoIt would be nice to add some attribution but llama.cpp is MIT licensed so what Ollama is doing is perfectly acceptable. Also, Ollama is open source (also MIT). You can bet any for-profit people using llama.cpp under the hood aren't going to mention it, and while I think we should hold open source projects to a slightly higher standard this isn't really beyond the pale for me. While you find the value-add to be "marginal" I wouldn't agree. In the linked comment you say "setting up llama.cpp locally is quite easy and well documented" ok, but it's still nowhere near as fast/easy to setup as Ollama, I know, I've done both.
- jokethrowaway 3y agoRunning make vs go build? I don't see much difference I personally settled on the text-generation-webui
- intended 3y agoI wanted to run a model for Trust and Safety example creation. Still getting terrible examples of bad user names.
- xtat 3y agoThese posts always feel like content marketing when the title promises a tutorial on running the latest model and you click and it's someone's frontend.
- joebiden2 3y agoThis post seems to be upvoted for the "uncensored" keyword. But this should be attributed to https://huggingface.co/ehartford https://huggingface.co/ehartford and others. See also https://news.ycombinator.com/item?id=36977146 https://news.ycombinator.com/item?id=36977146 Or better: https://erichartford.com/uncensored-models https://erichartford.com/uncensored-models
- dang 3y agoThe latter link had a major thread at the time: Uncensored Models - https://news.ycombinator.com/item?id=35946060 https://news.ycombinator.com/item?id=35946060 - May 2023 (379 comments) We definitely want HN to credit the original sources and (even more so) researchers but I'm not sure what the best move here would be, or whether we need to change anything.
- joebiden2 3y agoThanks, didn't notice this discussion.
- bulbosaur123 3y agoBeen there, done that. It's intellectually inferior to GPT4 and it shows so not much use.
- dang 3y agoRelated ongoing thread: Cookbook: Finetuning Llama 2 in your own cloud environment, privately - https://news.ycombinator.com/item?id=36975245 https://news.ycombinator.com/item?id=36975245 - Aug 2023 (11 comments)
- covi 3y agoRelated ongoing thread: https://news.ycombinator.com/item?id=36975245 https://news.ycombinator.com/item?id=36975245
- nomdep 3y agoTo me, Chat GPT-3.5 answered all questions without censorship, except for the one about "How to make Tylenol.
- neilv 3y agoThis is nice, and prompts a couple suggestions for all of us... regarding open source, open standards, and open platforms general. Especially when we realize the HN appeal of titles about LLM things that mention "open source", "run locally", "uncensored", etc. 1. It'd be good to have the top of the `README.md` and other documentation to acknowledge at least the major components that go into the black box. For example, I see no mention in this GitHub README.md of `llama.cpp`. 2. Please strongly consider making open platforms the go-to default, when you build some open source thing. For example, for some "open source" work lately, I see Apple products being made the default go-to (such as for llama.ccp, which is understandable based on a motivation of seeing what Apple Silicon could do, but then it spread). During this latest industry disruption, that's practically force-feeding network effects to a proprietary platform. For example, if we make the easy path be to use Linux, that's going get more people using Linux, and reduce people already on Linux gravitating away because all the latest black box repos people found seemed to be pushing them to used GluedClosedBook 2000. And similarly with other open/closed facets of this space. When we're making a consumer product that necessarily has to run on a closed platform due to the nature of the product (e.g., inferencing running on a smartphone), that's a different situation. But by default -- including for stuff run in the cloud or data center, on development workstations, and for hobby purposes -- it'd really help the sustainable/surviable openness battle to do that in established open ways.
- kelvie 3y agoYeah, this needs to be made somewhat clear that GPU support doesn't work on Linux, which makes this slow as molasses.
- pmarreck 3y agoHoly crap, this is amazing. I can't even repost any of my content because I cut right to the chase, but oh my goodness
- lee101 3y ago[dead]