14 ms·
InternLM – new open source 7B LLM
- kasia_wieczorek 3y agoGreat name for a simple LLM hehe
- gnrlst 3y agoIs this also censored/nerfed? I'd love to play with a "raw" unnerfed model to fully grasp what an LLM can do (and see how biased it is). Does anyone have any recommendations for unnerfed models to try out?
- lioeters 3y agohttps://huggingface.co/models?search=uncensored&sort=trending https://huggingface.co/models?search=uncensored&sort=trendin...
- cubefox 3y agoThe most powerful available foundation model is code-davinci-002, a.k.a. GPT-3.5. It's only available on Azure since OpenAI removed it from their own Playground and API for some reason.
- RobotToaster 3y agoThe model isn't available at all?
- cubefox 3y agoIt is available in the sense that it is accessible. The weights are not available for download of course, but the OP wanted to "play around" with it, for which only access is required. There is no other accessible foundation model that can compete with GPT-3.5.
- cubefox 3y agoWhy are you guys downvoting me?
- brucethemoose2 3y agoBecause GPT 3.5 not very good compared to LLaMA 65b or even 33b finetunes, from my testing. Also because 3.5 is not really available?
- cubefox 3y agoHave you actually tested code-davinci-002?
- alach11 3y agoMaybe you mean gpt-3.5-turbo or text-davinci-003? Or GPT-4 (technically in beta so not fully available to everyone)?
- cubefox 3y agoNo, those are all fine-tuned models which are "nerfed" in the terminology of the OP. I mean code-davinci-002, the GPT-3.5 base model.
- squeaky-clean 3y agoIs that what nerfed means? I usually see "nerfed" used in a way that means that it will refuse to answer certain topics. "I can't answer that as it would violate copyright" and such.
- cubefox 3y agoThe fine-tuned models are certainly censored and not "raw".
- squeaky-clean 3y agoBut doesn't code-davinci-002 also have OpenAI's filters in between you and the model?
- cubefox 3y agoYes, but that's different from the model itself being fine-tuned.
- seanhunter 3y agocode-davinci models are finetuned on code so I don't think that's what the OP wants. For reference the family tree is here https://platform.openai.com/docs/model-index-for-researchers https://platform.openai.com/docs/model-index-for-researchers
- seanhunter 3y agoAll 3 text-davinci models are available on openAI's api. including 3 (which is the GPT-3.5 gen). Code-davinci-002 is a code-tuned model, You can see a nice visual summary of the relationships between the openAI models at https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tracing-Emergent-Abilities-of-Language-Models-to-their-Sources-b9a57ac0fcf74f30a1ab9e3e36fa1dc1 https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr... Or the official source is https://platform.openai.com/docs/model-index-for-researchers https://platform.openai.com/docs/model-index-for-researchers
- cubefox 3y ago> All 3 text-davinci models are available on openAI's api. That's irrelevant because these are all fine-tuned. > Code-davinci-002 is a code-tuned model No, "code-tuned" isn't even a thing. It is a foundation model, which consists purely of pretreating. No fine-tuning is involved. > Or the official source is The official source says exactly what I just said.
- seanhunter 3y agoOK perhaps I used slightly the wrong term. The docs[1] say that code-davinci-002 is "optimized for code completion tasks" though so it seems unlikely to fulfil the OPs purpose of playing around with an unaligned/sweary model which was my main point. Some of the uncensored models from huggingface would probably serve that purpose much better. [1] see the entry for code-davinci-002 in https://platform.openai.com/docs/models/gpt-3-5 https://platform.openai.com/docs/models/gpt-3-5
- cubefox 3y agoCode was just part of its pretraining. All other GPT-3.5 models are fine-tuned versions of code-davinci-002. Quote: 1 code-davinci-002 is a base model, so good for pure code-completion tasks 2 text-davinci-002 is an InstructGPT model based on code-davinci-002 3 text-davinci-003 is an improvement on text-davinci-002 4 gpt-3.5-turbo-0301 is an improvement on text-davinci-003, optimized for chat Quote end. https://platform.openai.com/docs/model-index-for-researchers https://platform.openai.com/docs/model-index-for-researchers The reason you want a base model for code completion has nothing to do with code itself, it has to do with the fact that it completes text unlike all the instruction tuned models, which expect instructions. When you have code, there aren't necessarily any instructions present. You basically want autocomplete. That's what a base model does. But that doesn't mean it doesn't work with other things apart from code. After all, all other GPT-3.5 models are just code-davinci-002 with additional instruction and RLHF fine-tuning added, and they know countless other subject areas apart from code. I don't get why this is so hard to understand.
- logicchains 3y agoLLaMA 65B is the best uncensored model we've got, and the Airoboros fine-tuning if you want it to follow instructions.
- tyfon 3y ago> trust_remote_code=True This is a hard no from me, anyone know why this is so common in models from China? I'm not getting into conspiracies or anything here, but I've seen it in quite a few others from there. I wouldn't run a model with this requirement from anyone else for that matter.
- ShamelessC 3y agoMind pasting the link to that line? Am on mobile and can’t find it myself easily.
- ipsum2 3y agoI believe it's because the model architecture isn't added to Huggingface transformer library, so it needs to eval some python code (i.e. load a pickle) to create the PyTorch model. Have not noticed it to be specific to models from China, almost all lesser known models have to do this.
- jabbany 3y agoSeems pretty common though, for defining custom architecture configs whatnot? AFAIK the "remote code" is still openly hosted on huggingface so you can audit it if you like. Seems no more dangerous than things like `pip install some_random_library`?
- rfoo 3y agoI like this pip metaphor. If we had required `--trust-remote-code` for every `npm install` we could have avoided left-pad and most of the software supply chain drama in the past years.
- Tepix 3y agoI welcome new models! The more, the merrier. That said, this model has been tailored but they are comparing it to non-finetuned LLaMA-7B in their benchmark? That seems a bit fainthearted.
- moffkalast 3y ago> HumanEval: InternLM-7B: 10.4, LLaMA-7B: 14.0 The funny part is that the base model apparently outperforms the fine tune. So far the HumanEval benchmark seems to be the only one that can objectively compare overall model performance despite being a coding-only benchmark, the rest mostly just give a "99.7% chatgpt" bullshit results. Turns out you can't compare creative writing because all outputs are basically valid.
- usgroup 3y agoA related question -- when fine tuning a model like this to a specific corpus, how does the fine tuning effect the actual chat capability, since the chat model weights seem to come as a separate model? Does one fine tune the LLM+Chat model directly? If so, does that not require some kind of prompt based training as opposed to just lookahead prediction? Does one have to fine tune the LLM and then repeat whatever they do to get the LLM+Chat model?
- airgapstopgap 3y agoNote that this is apparently a 7B version of a 104B model trained with the intention of competing with OpenAI offerings on the Chinese market [1]. There is a number of those projects: Baichuan, ChatGLM2, InternLM and some more iirc, and they all have small-scale opensource versions. For what it's worth, I've tried out ChatGLM2-6B and Baichuan converted to LLaMA (the architecture is literally identical in that case). They're okay, though underwhelming given their reported benchmarks; probably the main point of creating them is gaining experience for engineers, and feedback from the wider community that has less incentive to downplay their shortcomings. Surprisingly, they do not appear censored in any particularly "Chinese" political direction, but they share sensibilities of ChatGPT and Claude. 1. https://github.com/InternLM/InternLM-techreport https://github.com/InternLM/InternLM-techreport
- potency 3y agoI don't know anything about what you're talking about. Where do I start to learn some of the AI terminology, models, benefits and drawbacks of each, etc?
- hutzlibu 3y agoThe most patient lecturer would probably be ChatGPT itself ...
- brucethemoose2 3y ago> Surprisingly, they do not appear censored in any particularly "Chinese" political direction, but they share sensibilities of ChatGPT and Claude. Perhaps they used GPT4 responses for the instruct finetuning, as many LLaMA finetunes do? The paper doesn't say where they got the data from, other than "The pre-trained language model is further fine-tuned, following the mainstream procedure as in InstructGPT." (Also, I don't like how they use raw LLaMA 65b as a benchmark rather than an instruct tuned derivative)
- airgapstopgap 3y agoI believe it's more like they used Anthropic human preference data [1] or similar, and accordingly Anthropic/progressive American notion of honest-helpful-harmless behavior. Thus I've seen models misgeneralize towards prudish finger-wagging. For example they parse badwords like "beat", "abuse", "steal" in morally neutral contexts ("beat a benchmark" or something) as signifiers of substantial transgression and spiral into telling me how, as language models, they insist it's never okay to etc. etc. This attitude was strikingly reminiscent of American models, even though other failure modes – like hallucinations – don't seem so similar. Papers like Tulu [2] suggest that LLaMA-65b is indeed an appropriate baseline, given reasonable prompting. Instruct datasets only convey a flavor of responses, and for a strong foundation model that can infer the intended flavor on its own, naive finetuning seems to be detrimental. GPT-4 was much more powerful prior to having been finetuned, if reports of early witnesses and researchers are to be believed. 1. https://huggingface.co/datasets/Anthropic/hh-rlhf https://huggingface.co/datasets/Anthropic/hh-rlhf 2. https://arxiv.org/abs/2306.04751 https://arxiv.org/abs/2306.04751
- freediver 3y agoTried it with a few PyLLMs benchmarks [1] and it did not do very well. Q. Use m to substitute p, a to substitute e, n to substitute a, g to substitute c, o to substitute h, how to spell peach under this rule? A. m: p a: e n: a g: c o: h how: peach Q. write three sentences each ending with the word apple A. 1. The apple tree was covered in apples. 2. She sliced the apple with a knife. 3. The apple juice was too sweet. Wonder if the synthetic benchmarks are very biased and not that useful. [1] https://github.com/kagisearch/pyllms/blob/6c11cd51b099d33e61d209712f0fc660838c3e5f/llms/llms.py#L157 https://github.com/kagisearch/pyllms/blob/6c11cd51b099d33e61...
- blitzar 3y ago“when a measure becomes a target, it ceases to be a good measure” The model itself it optimising weights for its targets, and the model builders are optimising the model for the benchmarks.
- exizt88 3y ago> The code in this repository is open-source under the Apache-2.0 license. The InternLM weights are fully open for academic research and also allow commercial use with written permission from the official team. For inquiries about commercial licenses and collaborations, please contact internlm@pjlab.org.cn. This makes me much less excited about this model.
- hmottestad 3y agoIs that even valid? This seems to be the only place where they’ve made this exemption, it’s not written in the license. Even the weights on hugging face are licensed under apache 2.0. Doesn’t apache 2.0 allow for fairly unrestricted commercial use? Isn’t that the whole point of using that license?
- exizt88 3y agoThe model code is Apache 2.0, the weights are proprietary.
- huggingmouth 3y agoThat's not their huggingface repo says: https://huggingface.co/internlm/internlm-7b https://huggingface.co/internlm/internlm-7b The current release on huggingface is available under plain apache 2.0
- deleted 3y ago[deleted]
- Tostino 3y agoIf weights are even protected by copyright at all...which would be a departure from current law.
- ohgodplsno 3y agoLess excited that you can't freely take work from academics to resell it in a shitty SaaS company ?
- YetAnotherNick 3y agoIs the dataset on which it is trained on mentioned anywhere?
- yieldcrv 3y agowhy is 7B parameters seemingly a magic number?
- peignoir 3y agojust easier to run on smaller hardware
- datastack 3y agoI guess going with a parameter count that matches existing models makes it easier to compare benchmarks. Perhaps there is another particular reason like required memory, but momentum is probably also significant.
- wilonth 3y ago7B params would take 14gb of gpu RAM at fp16 precision. So it would be able to run on 16gb GPUs with 2gb to spare for other small things.
- brucethemoose2 3y agoBut in practice, no one is running inference at FP16. int8 is more like the bare minimum.
- ForOldHack 3y agoI have an 8GB, and I am considering two more 8GB, it should I get a single 16GB? The 8GB card was donated, and we need some pipelining... I have 10~15 2GB quadro cards... Apparently useless.
- brucethemoose2 3y agoI mean... It depends? You are just trying to host a llama server? Matching the VRAM doesn't necessarily matter, get the most you can afford on a single card. Splitting beyond 2 cards doesn't work well at the moment. Getting a non Nvidia card is a problem for certain backends (like exLLaMA) but fine for llama.cpp in the near future. AFAIK most backends are not pipelined, the load jumps sequentially from one GPU to the next.
- geckbench 3y ago[dead]
- joelthelion 3y agoWhat kind of hardware do you need to run a model like this? Do you need an A100, or can something smaller suffice? What about for fine tuning? Are hardware requirements higher?
- timmclean 3y agoSaving you a click: despite what the repo title might suggest, while the code is open source, the model weights cannot be used commercially without permission. > The code in this repository is open-source under the Apache-2.0 license. The InternLM weights are fully open for academic research and also allow commercial use with written permission from the official team. For inquiries about commercial licenses and collaborations, please contact internlm@pjlab.org.cn. https://github.com/InternLM/InternLM#open-source-license https://github.com/InternLM/InternLM#open-source-license
- b1n 3y agoThere is a great opportunity for totalitarian and authoritarian regimes (China and UAE so far) to create commercially usable and free LLMs that work significantly better than alternatives (backed by large amounts of government money). Over time, as they get used in more and more products, these LLMs can become more 'aligned' to these regimes way of thinking. There are no Chinese companies that are not part of the Chinese government. This is a new kind of cultural soft power.
- heartbeats 3y agoAligned how? If you download the code, it won't change under your feet.
- ASalazarMX 3y agoThe weights, calculated through very intensive computing, are what hold the knowledge in LLMs, the source code just executes those. These products could just update/patch their weights periodically, and no one would complain because that's not bad per se.
- czDRZ-akk 3y agoThis would be more interesting for discussion if the comment described an actual threat scenario instead of a vague hypothetical. Even in the hypothetical, there’s no actual described consequence, only that China gains some undefined level of soft power which means nothing on its own. Some ideas: - How are these LLMs being used? Who is the end user and what are they using the application for? - If a state-level threat actor wanted to compromise an LLM, how would they do it? What would their goals be? How would they then use the attack vector to accomplish their goals? - What benefit would the actor get from doing so? What are the costs? What are the consequences if they fail or are discovered? - How would a target detect if they’ve been compromised? How easily could they recover?
- b1n 3y agoI didn't think it was that vague. But if you're looking for ideas. 1. High quality LLM made free for commercial use. 2. LLM is used in many places as it is the best. 3. LLM is aligned to subtly promote the interests of the threat actor. There is no 'compromise'. It is not hacking software, only wetware. A concrete example... LLM created by Advanced Persistent Threat (APT) is used in educational software aimed at kids. Over time as the kid interacts with it the LLM promotes a way of thought that either aligns with the APT or undermines the ideology of the society they are in. There is no moment that can be pointed to that says: "look they are trying to hack us!", but decades later you have adult members of a foreign society more open to your way of thinking. i.e. Soft Power.
- deleted 3y ago[deleted]