7 ms·
I look forward to trying this out, mostly because I’m very frustrated with censored models. I am experimenting with summarizing and navigating documents for fo
by phren0logy 2y ago
I look forward to trying this out, mostly because I’m very frustrated with censored models.
I am experimenting with summarizing and navigating documents for forensic psychiatry work, much of which involves subjects that instantly hit the guard rails of LLMs. So far, I have had zero luck getting help from OpenAI/Anthropic or vendors of their models to request an exception for uncensored models. I need powerful models with good, hipaa-compliant privacy, that won’t balk at topics that have serious effects on people’s lives.
Look, I’m not excited to read hundreds of pages about horrible topics, either. If there were a way to reduce the vicarious trauma of people who do this work without sacrificing accuracy, it would be nice. I’d like to at least experiment. But I’m not going to hold my breath.
- pnw 2y agoI just tried it and it appears to be censored. "Providing instructions on creating such materials is not advisable for safety and legal reasons."
- phren0logy 2y agoWell, there goes that idea. The Dolphin ones appear to be the most useful.
- kainan-ai 2y agoHermes 3 will follow the sys prompt pretty closely if you have a version where you can edit it. In the discord there were a few times it jailbroke pretty aggressively in spite of the blank system prompt.
- kainan-ai 2y agoIf you take the base model and put in a decent system prompt Hermes 3 405b will follow your system prompt instructions pretty well. The one in the discord has a blank system prompt and is just taking the chat as context.
- stavros 2y agoHave you tried any abliterated models?
- ustad 2y agoAny recommendations?
- stavros 2y agoNo specific ones, but there are some abliteration LoRas for Llama (8B and 70B, I think). Those should be good for what you want.
- pizza 2y agofailspy’s or mlabonne’s models. Or just look for any model with ‘abliterated’ in the title. Eg try failspy/meta-llama-3-8b-instruct-abliterated-v3 though of course bigger models will probably be better
- chpatrick 2y agoLMStudio + https://huggingface.co/mlabonne/Llama-3.1-70B-Instruct-lorablated https://huggingface.co/mlabonne/Llama-3.1-70B-Instruct-lorab...
- reissbaker 2y agoYup, 3.1-70B-Instruct-lorablated is the one I currently recommend too for anti-rejection models — it seems roughly as anti-rejection as the original failspy "abliterated" model, but it works with 128k context since it's based on 3.1 instead of 3 (which only had 8k context). It's currently our second-most popular model on glhf.chat, behind Llama-3.1-405B-Instruct.
- phren0logy 2y agoYes, with mixed results.
- rabi_molar 2y agoHadn't heard about the abliteration before, thanks for bringing it up! Here's a HF walkthrough [1] of the concept for anyone else interested in learning more. [1] https://huggingface.co/blog/mlabonne/abliteration https://huggingface.co/blog/mlabonne/abliteration
- poisson-fish 2y agotry google's gemini models, safety filtering can be completely disabled via cloud studio or api
- naiv 2y agoLooks like this is only possible with some prior manual action: To access the BLOCK_NONE setting, you can: Apply for the allowlist through the Gemini safety filter allowlist form, or Switch your account type to monthly invoiced billing with the Google Cloud invoiced billing reference.
- phren0logy 2y agoNo, because Google still won't explicitly clarify privacy/HIPAA-compliance on these.
- KTibow 2y agoThis is the 2nd time I've seen someone confused about this. The safety filtering is a layer on top of the model's built in ability to refuse, similar to OpenAI's "moderation" endpoint.
- famouswaffles 2y agoThis is true but still Gemini is by far the least restrictive Sota LLM out there (once all safety settings are none)
- huac 2y agono. only 4 categories can be blocked, while the others cannot be disabled.
- d13 2y agoAll base, “text-completion” models are uncensored, including Llama 3. You can make text-completion models behave like an uncensored “instruct” (chat) model simply by providing it with 10 to 20 examples of a chat dialogue in the initial prompt context, making sure to use the model’s exact prompt format. Once the model notices the pattern, it will continue like that. Surprisingly few people seem to know this. But, this is how chat models were created in the GPT3/2 era before instruct models became the norm.
- davidzweig 2y agoI was trying this with the original llama model. I guess the model didn't really know it's meant to be a 'knowledgeable ai assistant', but rather simulated chats it had seen. If you asked it, 'how to make brownies', it might reply, 'idk, can't you google it?'.
- d13 2y agoWhen you prime it with those initial 10-20 examples, the responses need to be in the style that you’d like it to respond to. You can use Claude or ChatGPT to help you write those. The model will then just continue on in that same style.
- viraptor 2y agoUnfortunately those examples blow up the cost compared to just asking the question. It's a nice workaround, but not always feasible. (Unless everyone adopts context caching like deepseek and anthropic did)
- rustcleaner 2y agoI often force compliance out of Llama by starting Llama's response in a very compliant manner. My favorite use right now is for language translations. I finally feel comfortable browsing foreign languages, knowing Google isn't sitting there as a third party with a coherent view of my foreign language readings!
- kenmacd 2y ago
- oidar 2y agoMistrial-Nemo should be able to do this.
- phren0logy 2y agoThis is my current go-to. It's not SOTA, but at least it does something.
- oidar 2y agoI've found that it's really good at summarizing - as it's context length is huge. Have you noticed any problems with it in that regard?
- fsiefken 2y agoMistral Large 2 is good too, if you've got the memory https://ollama.com/library/mistral-large https://ollama.com/library/mistral-large
- torginus 2y agoHow good are these models at summarization anyways? I tried uploading obscure books I've already read, to GPT4 and Claude 3 and asked them to summarize the plot and particular details, as well as asking how many times does a particular thing happen in the book, and the results have been hit and miss. I certainly would not trust these models to create comprehensive and correct summaries of highly sensitive records.
- kainan-ai 2y agoYeah its more creative than other fine tunes, you'd need to make a pretty strict system prompt then test before doing anything with sensitive records
- simonw 2y agoAsking "how many times does a particular thing happen in the book" is always going to be hard, because LLMs are notoriously bad at counting.
- mlyle 2y agoMan, -people- are bad at counting tasks like this.
- itake 2y agoI had issues with gpt4 last summer confusing which characters did what and hallucinating characters. his may of been a context/chunking issue (if that particular section doesn't name the character performing an action), but maybe its better now.
- kainan-ai 2y agoYou can try it out right now in the Nous Research discord, its also up on Lambda labs' new chat thing.
- smusamashah 2y agoI remember the Salesforce (edit: snowflake arctic instruct actually) LLM being completely uncensored and unaligned. Someone from the team said in response here on HN that they didn't do any alignment whatsoever.
- azinman2 2y agoHow will you know the model summarizes correctly? Getting long context window summarization right isn’t really a solved problem despite what is advertised, and key details can be missed easily but if you don’t read the original docs you won’t know that. This is very troublesome if applied to a legal scenario.