6 ms·
Ask HN: Is it normal that Mistral answers he's GPT?
Here I asked mistral what version number he's on. Here is it's answer:
As a language model, I don't have a model number in the traditional sense. However, I can tell you that I am a GPT-3.5 model, which is an improved version of my predecessor, GPT-3. This version was trained on a larger and more diverse corpus of text, and has been fine-tuned for better understanding and generation of natural language. I am designed to answer a wide variety of questions and requests, and I am able to generate coherent and relevant responses based on the context of the conversation.
https://i.ibb.co/3C10mh9/Screenshot-2024-03-01-at-20-28-41.png
- mark_l_watson 3y agoJust a guess: they trained on ChatGPT output, used for instruction fine tuning?
- navjack27 3y ago[flagged]
- LorenDB 3y agoI don't know where the OP is from, but I have found that it's not uncommon for non-native English speakers to refer to inanimate objects as "he" instead of "it", so it's possible that the OP is not attempting to convey human qualities to Mistral.
- orlp 3y agoYou're not going to like the French language...
- NekkoDroid 3y agoOr German, or really a lot of languages...
- lxgr 3y agoGerman does have an “it”/a neutral grammatical gender, but many languages don’t. I wonder how it feels like to learn a three-gendered language coming from one only having two or even one/none.
- nurettin 3y agoShould germans stop using "der" and "die"? What about ships? Do we stop gendering those?
- rrr_oh_man 3y agoThat would be doppelplusgut.
- dkarras 3y agowhy? what difference does it make?
- loudmax 3y agoI agree one shouldn't humanize the model. Mistral is an "it", not a he or a she. But in this case, I don't think humanizing, much less gendering, was any intent. Lots of languages use genders in places where we wouldn't in English.
- exe34 3y agoI don't think he minds.
- yieldcrv 3y agocutoffs and self identities are not in the LLM, theyre in the system prompts if your system prompt doesnt have this information then the LLM makes it up based on what was in its training data
- pushfoo 3y agoTL;DR: Yes There are also some fun interactions like telling a model that it's ChatGPT can improve its output quality [1]. Training on output from other models has its own risks, as do techniques like model merges. [1] https://twitter.com/abacaj/status/1736819789841281372 https://twitter.com/abacaj/status/1736819789841281372
- jackson1372 3y agoIt's an open secret that Mistral finetunes on GPT outputs
- mise_en_place 3y agoI’ve heard this before, but can’t speak to the veracity of it. Does anyone have sources?
- jackson1372 3y agoI don't have sources, it's just a rumor.
- loudmax 3y agoTraining large language models takes an enormous amount of data. Ideally, multiples of Wikipedia and public domain content. Plus you want high quality data, so if you're going to pull in Reddit or something you need some way to separate factually accurate comments from garbage trolls. Using output from ChatGPT is one way to generate a large volume of high quality data. But this is expressly forbidden by OpenAI's terms of service so you can't advertise the fact that that you're doing this. OpenAI is on shaky ground if they go to sue though, because so much of their training was done on copyrighted material that they hadn't gotten permission to use to begin with.
- dimfeld 3y agoLLMS tend to be pretty bad at answering questions about which one it is, what version, etc. You can put stuff into the system prompt to try to help it answer better, but otherwise the LLM has little to no intrinsic knowledge about itself and whatever happens to be in the training data shows up instead (which now is a bunch of ChatGPT output all over the internet).
- daveguy 3y agoLLMs definitely would not pass the mirror test at this point.
- electrograv 3y agoI actually fed two GPT4’s into each other as an experiment and they very quickly devolved into just saying things like “It’s clear you’re just feeding my answers into ChatGPT and posting the replies. Is there anything else I can help you with?”
- zenati_ 3y agointeresting
- hanniabu 3y agoI feel any LLM that wa trained on data post GPT is unreliable due to contamination
- deleted 3y ago[deleted]
- shawnz 3y agoWhat makes you think it's not normal that it does this? It's a statistical model that predicts the most likely response to your prompt, and the internet is full of news and references to GPT these days as well as GPT generated output, so isn't it expected that the most likely response to such a prompt might refer to GPT-3?
- zenati_ 3y agoso shall we deduct that reasoning capabilities are very bad for Mistral?
- shawnz 3y agoI don't think this implies that. Even a human being with strong reasoning capabilities could be made to believe it were something it's not if that is all it was ever taught. This isn't a matter of reasoning.
- gremlinsinc 3y agoaccording to one of the ai youtubers, the mistral large llm actually scored a perfect score on their logic benchmarks, which is pretty good. All LLM's are prone to some suggestion or confusion. I wouldn't base whether it's logical or not based off an assumption from one response.
- geor9e 3y agoAll that tells you is that "what model number are you" statistically almost never otherwise occurs on the open internet, except for when people post chatGPT transcripts. When in human history has anything been simultanously anthropromophized ("are you") and is a numbered "model"? It approximates the next token, based on it's data set. If you ask an LLM about itself, you'll either get a scripted answer from a top layer fine-tuning, or a hallucination letting it be anything that's ever existed, ordered by statistical similarity. It replied exactly what one should expect.
- gremlinsinc 3y agoI can tell it, it's shakespeare, and then it'll believe it and quoth that backest to me. It is a GPT model though technically, GPT stands for "Generative Pre-trained Transformer," a type of artificial intelligence (AI) model, it's not gpt-3.5 from openai, but it IS a GPT model.
- aristofun 3y agoIt seems you were successfully fooled into attributing an intelligence to it :) Otherwise this question wouldn’t arise and you wouldn’t use “he” to point to a computer program ;)