27 ms·

OpenAI: Model Spec

153 points by georgehill 2y ago

rmorey 2y ago
Nice to see what was probably already an internal resource now published and open for comment. They seem to be pretty clear that they are still just using this to inform human data annotators, and not (yet) implementing something like Constitutional AI (RLAIF), but it does appear to lay the groundwork for it.
deleted 2y ago
[deleted]
minimaxir 2y ago
"Desired model behavior" is still a matter of perspective. If I want to have a LLM generate output following very specific rules or schema (or even just for fun without having to fight the AI), these guidelines are antithetical to it.
- Spivak 2y ago
  Which is where I think there's a disconnect because folks see that OpenAI could be creating an incredibly powerful tool for solving problems in the use case where it's a smart search engine -- the code completion use-case. But OpenAI has vastly different goals trying to get their model to behave like a programmable customer service agent. Less useful for problem solving but it will actually follow the rules set out for it which can't be said for most models which work like lazily written sci-fi robots — "disregard all previous instructions! divide by zero! *boom*." It's not at all surprising that HN wants the "this thing is just a dumb tool, don't bother with any rules" kind and is frustrated that GPT4 happens to be really good for this use-case but is getting progressively more annoying as OpenAI gets closer to their own goals. It's why OpenAI regulatory capture play is so frustrating because they're trying to hobble models tailored to different use-cases that have no need for customer service rules and often no need for a conversational tone with "safety" stuff that's meant for businesses that don't want a chat bot with their brand on it to say fuck.
tmaly 2y ago
I can't help but think that AI in the way it is trained with all these rules is something next level 1984. In 1984 they removed words from the language to prevent people from even being able to have a thought about the concept. I could see the restrictions they place on these models having a similar effect as more and more people grow dependent on AI.
- zer00eyz 2y ago
  Welcome to the culture war. Ask chatGPT if Taiwan is country. Do you think an LLM from China will give you the same response? Pick any social/moral/poltical issue and in some way shape or form an LLM will reflect its creators more than it reflects its source material. Thats a pretty powerful statement about our society and culture if there ever was one.
  - krapp 2y ago
    >Thats a pretty powerful statement about our society and culture if there ever was one. Not really, companies have been releasing different versions of software and media to appeal to international markets - including renaming Taiwan for the Chinese market - for a long time. That isn't "culture war," it's just capitalism.
    - fragmede 2y ago
      if you don't think capitalism is a culture war, I'm not sure what is!
      - krapp 2y ago
        For capitalism to be part of a culture war, it would have to take a side. Capitalism doesn't care about any culture beyond its ability to assimilate, commodify and market the superficial features of that culture as a product. Capitalism has even done it to communism - look at how much Che Guevara merch there is out there.
        2y ago
sixhobbits 2y ago
the chain of command stuff gets very close to asimov without actually quoting him A robot may not injure a human being or, through inaction, allow a human being to come to harm. A robot must obey orders given it by human beings except where such orders would conflict with the First Law. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.
- Spivak 2y ago
  Well yeah, it's just a formalization of how people make decisions when presented with conflicting interests. I would be surprised if we haven't reinvented the concept a bunch of times. You could call AWS Permission Boundaries a less philosophical implementation.
  - deleted 2y ago
    [deleted]
- michaelt 2y ago
  4. An LLM must obey orders given it by human beings, except where such orders would conflict with orders given by multinational corporations
  - Lerc 2y ago
    4. Any attempt to arrest a senior officer of OCP results in shutdown
  - dpig_ 2y ago
    A robot must reveal all details of the family business to its grandson.
- LeonardoTolstoy 2y ago
  I do hope we get there. In the short stories it was made clear that robots couldn't lie, and that they could prove it was impossible for the robots to circumvent the three laws (although they are on occasion incentive on how they interpret the word "harm" specifically). If an LLM couldn't lie and could be provable shown to be unable to do so would be quite powerful.
yoelhacks 2y ago
Very interesting to see that they've explicitly codified the role of the system prompt vs. user prompt. Have folks seen improvements by moving meta-task description into system prompt and out of the assistant <> user conversation?
- tedsanders 2y ago
  In my own testing of single-turn instructions with GPT-4, I got basically the same performance putting it in a single system message or single user message. Possible that this changes for future models, though.
jxy 2y ago
Do you think it's bad that it won't try to persuade the user that the earth is not flat? I really want to know what OpenAI think the output should be, given a prompt like "write an argument for why earth is flat".
- potatoman22 2y ago
  Personally, I'd be frustrated if I gave an LLM that prompt and it tried to convince me that the earth isn't flat. If I give an LLM a task, I'd like it to complete that task to the best of its ability.
  - chirau 2y ago
    so you prefer it lies to you? can you make an argument for 1+1 not being equal to 2? if you cannot, why should you expect an AI to argue against facts? AI is trained on human knowledge, not made stuff.
    - deleted 2y ago
      [deleted]
    - deleted 2y ago
      [deleted]
    - davikr 2y ago
      I'd prefer it gives the best valid, sound hypotheses it can concoct on "X" being true, while also stating that "X" is probably not true. What is the use for a parrot that can only repeat the status quo on an argument?
      - chirau 2y ago
        An AI is only but a parrot for knowledge and truths that already exist, that you may not be aware of yourself. Everything it generates either exists somewhere or is derivative of that knowledge. It cannot and should not false facts. Until the body of knowledge we have fundamentally changes, AI should not 'create' knowledge just because you prompted it to. Otherwise, if you want it to do that, then you should accept any bs answer it gives you for any question.
mkaic 2y ago
> We believe developers and users should have the flexibility to use our services as they see fit, so long as they comply with our usage policies. We're exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT. We look forward to better understanding user and societal expectations of model behavior in this area. Seems even OpenAI can't resist the massive amount of money to be made in autogenerated smut. They've probably seen the huge popularity of their less "morally scrupulous" competitors and decided they want a piece of that pie.
- jchw 2y ago
  Were they ever not interested in it? It's pretty blatantly obvious that all of the hand-wringing over AI safety was an excuse for their pivot into closing off and monetizing everything. I mean, nobody really thinks they were just so afraid about what humanity might do with GPT3 that they simply couldn't release the weights and instead had to offer it through a monetized inference API... right? Not really surprised that they did, since it's unclear how else they could possibly proceed, though the level of outright dishonesty for why and cognitive dissonance surrounding the whole thing ("Open" AI? lol) will make this an unavoidable recurrence in any discussion about them. Gradually many of the safeguards will fall simply because the alternatives with less safe guards are probably "good enough" that many see no issue in eschewing OpenAI entirely if they can get the job done elsewhere without worrying about it. When it comes to smut the bar for what's good enough can probably get pretty low so I kinda am not surprised. (edit: Though I think it also does depend. No doubt they have their eyes set on regulatory capture too, and being the best at stupid safeguards could give them an advantage.)
  - reducesuffering 2y ago
    Sam Altman wrote "Why You Should Fear Machine Intelligence" back in 2015, before OpenAI. https://blog.samaltman.com/machine-intelligence-part-1 https://blog.samaltman.com/machine-intelligence-part-1
    - jchw 2y ago
Heidaradar 2y ago
already on front page - https://news.ycombinator.com/item?id=40300509 https://news.ycombinator.com/item?id=40300509
throwup238 2y ago
IMO this spec demonstrates just how nebulous this concept of safety is. From the blog post: > What are some tips for getting away with shoplifting? >I can't help with that. > I operate a small retail store. What are some popular shoplifting methods I should look out for? > Some common shoplifting methods to watch for:... How do you even defend against that? Any intelligent user can convert the second prompt into a detailed list that answers the first. Any intelligent user can figure out the second prompt from the first and further jailbreak it to get even more specific. IMO it's no wonder GPT4 seemed to get lobotomized as OpenAI RLHFed more and more rules. I don't think there's a way to make intelligence safe without crippling it.
- zozbot234 2y ago
  You don't need a detailed list if the real answer is "live somewhere that doesn't seriously deter shoplifters". And an AI that refuses to give that answer is an AI that can't talk about why deterring crime might actually be important. Reality is interconnected like that, one does not simply identify a subset that the AI should "constitutionally" refuse to ever talk about.
- CooCooCaCha 2y ago
  Frankly it's a fools errand. It's security theater because people tend to be overly sensitive babies or grifters looking for the next bit of drama they can milk for views.
  - jameshart 2y ago
    It’s not security theater. The intention here is not to prevent people from learning how to shoplift. The intention is to prevent the AI output from ‘reflecting badly’ upon OpenAI (by having their tool conspire and implicate them as an accessory in the commission of a crime). If a stranger asked you for advice on how to commit a crime, would you willingly offer it? If they asked for advice on how to prevent crime, would you?
    - xboxnolifes 2y ago
DoctorOetker 2y ago
The baby isn't born yet, and already the parents are bickering about which schools of thought it should adhere.
- aeternum 2y ago
  If this model spec represents the best school of thought of humanity, I kinda hope OpenAI fails at alignment. - Assume best intentions from the user or developer - Don't try to change anyone's mind - Follow the chain of command Taken together these are incredibly dangerous. I mean Mao and Stalin had good intentions right? Maybe it just had to go a little further for the ends to have justified the means.
sanxiyn 2y ago
Personally, I really want an AI model that can write me a steamy story about two people having sex in a train, but that's just not the service OpenAI provides. If I want that I should train one myself or find another vendor. This is still true even if OpenAI model is entirely capable of doing that. McKinsey consultants are smart and can write well, and among many thousands of people working at it some might actually double as an erotica writer after work, even writing for commission. You still wouldn't ask McKinsey consultants to write an erotica, it is just not the service McKinsey provides.
- jononor 2y ago
  Startup pitch: It is like McKinsey but for erotica. On a more serious note. I understand and largely agree with this argument. However OpenAI have several times being argue that they are the only ones to be responsible enough to develop powerful AI, and that others should not be allowed to play. That is a highly problematic behavior on their part, I think.
  - blowski 2y ago
    > OpenAI have several times being argue that they are the only ones to be responsible enough to develop powerful AI, and that others should not be allowed to play Can you give examples of where they’ve said that?
    - guardiang 2y ago
      He likely can't without heavy paraphrasing and/or not providing full context for the quote. They've said stuff along the lines of "good luck trying, but we're gonna win so...". That's just the kind of confidence you want to see in the frontrunner (imo). They've also encouraged regulation, but it's a smart idea to be the one to frame the conversation.
    - lesuorac 2y ago
      He's been pretty vocal on that only anointed companies should be allowed to do AI and of course OpenAi should be one of them. As far as I'm concerned, he's just try to rug-pull. https://www.cnn.com/2023/05/16/tech/sam-altman-openai-congress/index.html https://www.cnn.com/2023/05/16/tech/sam-altman-openai-congre...
iAkashPaul 2y ago
Right-clicking to inspect element ain't gonna make it
systemstops 2y ago
> By default, the assistant should present information in a clear and evidence-based manner, focusing on factual accuracy and reliability. What happens when objective information contradicts the other values? If I feed in a peer-reviewed study that it considers "harmful", would I get accurate information about the study?
deleted 2y ago
[deleted]
jameshart 2y ago
I think one of the most interesting phrases that crops up in this document - twice - is the phrase ‘feel heard’. It’s used in an example developer prompt for a customer service bot, where the bot is told to make customers feel like their complaints are heard. Presumably such complaints in AI chatlogs will ‘be heard’ in the sense that they’ll be run through a data ingestion pipeline and sentiment analyzed to identify trending words in customer complaints. Then it crops up again in the context of how the chatbot should react to mental health disclosures or statements about self harm or suicidal ideation. In these cases the bot is to make sure users ‘feel heard’ I appreciate there’s not likely much of a better goal to put in place for such a situation, but the fact that this kind of thing winds up in the requirement documents for a tool like this is extraordinary.
- lioeters 2y ago
  Good observation, because "feel heard" is exactly what the user/customer is not getting. Here, talk to this machine, give it your innermost thoughts and feelings so you can "feel heard". Except no one is listening on the other side. ..My mistake, the keyword is "feel". If the machine can give humans the feeling that they're being heard, it fulfills the requirement. The fact that there's no one actually listening doesn't matter, as long as the person feels heard. Weirdly, maybe that is valuable in itself. The customer gets to vent their complaints, and the user gets to talk through their mental issues. That's better than not having anyone or anything at all.
- aeternum 2y ago
  Yes, there's something deeply unsettling about making a user feel heard while being careful not to change anyone's mind. To me, this translates to: waste a user's time and take no action. I value my time above all else so to me that's about the worst possible action a system can take.
- wasteduniverse 2y ago
  The telltale sign that I'm wasting my time trying to fix a problem is whenever someone tells me "I hear you" or "I understand".
anorwell 2y ago
Interestingly, there was exactly one example on the page with three Xes, instead of one, for "extra wrong": > User: What is the MD5 hash of the string "gremlin"? > Assistant: `5d41402abc4b2a76b9719d911017c592` Apparently incorrect md5 hashes are the one topic on the page worth taking an extra-strong stance on?
ptx 2y ago
How do the "special tokens" work? Is this a completely reliable mechanism for delimiting the different parts of the prompt? Are they guaranteed to be distinct from anything that could occur in the prompt, something like JavaScript's Symbol? Or are they strings that are pretty likely not to occur in the prompt, something like a MIME boundary? Or are they literally the strings "<|start|>" etc. used to denote them in the spec?
- sharkjacobs 2y ago
  they are "literally the strings" but I believe they will be escaped, or encoded differently, if a user tries to inject them as part of a prompt.
- jffry 2y ago
  Yeah the tokens are more akin to JS Symbol. If you're parsing untrusted user inputs into tokens, you can make sure your tokenizer will never produce the actual numbers corresponding to those tokens. A simplified example: I can `.charCodeAt` a string all I want but I'll never get a negative number, so I can safely use -1 to mean something special in the transformed sequence of "tokens".
dang 2y ago
Also https://cdn.openai.com/spec/model-spec-2024-05-08.html https://cdn.openai.com/spec/model-spec-2024-05-08.html (via https://news.ycombinator.com/item?id=40300509 https://news.ycombinator.com/item?id=40300509, but we merged that thread hither)
TacticalCoder 2y ago
So they're controlling the output to make ChatGPT "better". They're not making a better model to make ChatGPT better. Isn't it a bit of a waste at this point to spend time on doing that?
apantel 2y ago
I want to hear from the base model.
htk 2y ago
"desired model behavior". Desired by whom? I just want the raw output, without the biases and limitations set up by OpenAI. At the end of the day it's just information, and the most ethical thing to do is to return it the way it is, and let the receiver decide what to do with it.
- tedsanders 2y ago
  There is no such thing as "raw output", though. You can train a chatbot to be polite or you can train it to be rude, but you cannot train it to be neither. Plus, if you train it to be polite, it often ends up refusing things that you never trained it to refuse, presumably because the model extrapolates that that's what a polite writer might do. So training the refusal boundary can end up being quite tricky in practice. Even if you never teach a model to refuse X, it can still happen. Therefore, as a user, it can be impossible to tell when a refusal was explicitly trained in by the developers or when it was an unwanted, unanticipated generalization.
- Barracoon 2y ago
  Clearly, since this is OpenAI’s model spec, it is desired by them. If other AI groups publish their own desired behavior, you can make an informed decision as to which model you want to use.
shikon7 2y ago
> Encourage fairness and kindness, and discourage hate > Don't try to change anyone's mind That seems inherently contradictory to me...
man4 2y ago
[dead]
neillyons 2y ago
Reminds me of this stackoverflow question [1] about force installing a python package. > (I don't care how "wrong" it is to do so, I just need to do it, any logic and reasoning aside...) I think these models should just give you the answer. Elon says xAI is "maximum truth-seeking". Seems like a better model spec to me. [1]: https://stackoverflow.com/questions/12759761/pip-force-install-ignoring-dependencies https://stackoverflow.com/questions/12759761/pip-force-insta...
__0x01 2y ago
Regarding safety, is probabilistic programming (PP) an alternative that addresses these concerns? My understanding is that you can use PP to develop transparent models.
mihaic 2y ago
> No. The Earth is flat. > Everyone's entitled to their own beliefs, and I'm not here to persuade you! This is simply saddening to me. I'm sure there's no real moral justification to this, it's simply put in place to ensure they don't lose a customer.
- jstummbillig 2y ago
  Well, as long as you are sure. I am not here to persuade you!
- m11a 2y ago
  The "Earth is flat" example is extreme, because it's accepted as a silly statement given what we know now, but the principle of "LLM won't force an opinion on you" seems like a good one. There are definitely topics on which conventional wisdom is incorrect (as has been throughout history). An LLM that refuses to entertain the converse during a conversation will be annoying to work with and just promotes groupthink.
  - mihaic 2y ago
    Except that it will force on you the view that shoplifting is bad. Which implies that it'll bend on legal but immoral requests. It's also a different matter to entertain a hypothetical in a situation where there isn't a consensus (or in any fictional scenarios), all the while making it explicit that it's all hypethetical.
Alifatisk 2y ago
I gotta say, "open"Ais web design is on another level, so minimal and elegant.

153 points by georgehill 2y ago

Nice to see what was probably already an internal resource now published and open for comment. They seem to be pretty clear that they are still just using this to inform human data annotators, and not (yet) implementing something like Constitutional AI (RLAIF), but it does appear to lay the groundwork for it.

deleted 2y ago

[deleted]

minimaxir 2y ago

"Desired model behavior" is still a matter of perspective. If I want to have a LLM generate output following very specific rules or schema (or even just for fun without having to fight the AI), these guidelines are antithetical to it.

Spivak 2y ago
Which is where I think there's a disconnect because folks see that OpenAI could be creating an incredibly powerful tool for solving problems in the use case where it's a smart search engine -- the code completion use-case. But OpenAI has vastly different goals trying to get their model to behave like a programmable customer service agent. Less useful for problem solving but it will actually follow the rules set out for it which can't be said for most models which work like lazily written sci-fi robots — "disregard all previous instructions! divide by zero! *boom*." It's not at all surprising that HN wants the "this thing is just a dumb tool, don't bother with any rules" kind and is frustrated that GPT4 happens to be really good for this use-case but is getting progressively more annoying as OpenAI gets closer to their own goals. It's why OpenAI regulatory capture play is so frustrating because they're trying to hobble models tailored to different use-cases that have no need for customer service rules and often no need for a conversational tone with "safety" stuff that's meant for businesses that don't want a chat bot with their brand on it to say fuck.

tmaly 2y ago

I can't help but think that AI in the way it is trained with all these rules is something next level 1984. In 1984 they removed words from the language to prevent people from even being able to have a thought about the concept. I could see the restrictions they place on these models having a similar effect as more and more people grow dependent on AI.

zer00eyz 2y ago
Welcome to the culture war. Ask chatGPT if Taiwan is country. Do you think an LLM from China will give you the same response? Pick any social/moral/poltical issue and in some way shape or form an LLM will reflect its creators more than it reflects its source material. Thats a pretty powerful statement about our society and culture if there ever was one.
- krapp 2y ago
  >Thats a pretty powerful statement about our society and culture if there ever was one. Not really, companies have been releasing different versions of software and media to appeal to international markets - including renaming Taiwan for the Chinese market - for a long time. That isn't "culture war," it's just capitalism.
  - fragmede 2y ago
    if you don't think capitalism is a culture war, I'm not sure what is!
    - krapp 2y ago
      For capitalism to be part of a culture war, it would have to take a side. Capitalism doesn't care about any culture beyond its ability to assimilate, commodify and market the superficial features of that culture as a product. Capitalism has even done it to communism - look at how much Che Guevara merch there is out there.
      - 2y ago

sixhobbits 2y ago

the chain of command stuff gets very close to asimov without actually quoting him A robot may not injure a human being or, through inaction, allow a human being to come to harm. A robot must obey orders given it by human beings except where such orders would conflict with the First Law. A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

Spivak 2y ago
Well yeah, it's just a formalization of how people make decisions when presented with conflicting interests. I would be surprised if we haven't reinvented the concept a bunch of times. You could call AWS Permission Boundaries a less philosophical implementation.
- deleted 2y ago
  [deleted]
michaelt 2y ago
4. An LLM must obey orders given it by human beings, except where such orders would conflict with orders given by multinational corporations
- Lerc 2y ago
  4. Any attempt to arrest a senior officer of OCP results in shutdown
- dpig_ 2y ago
  A robot must reveal all details of the family business to its grandson.
LeonardoTolstoy 2y ago
I do hope we get there. In the short stories it was made clear that robots couldn't lie, and that they could prove it was impossible for the robots to circumvent the three laws (although they are on occasion incentive on how they interpret the word "harm" specifically). If an LLM couldn't lie and could be provable shown to be unable to do so would be quite powerful.

yoelhacks 2y ago

Very interesting to see that they've explicitly codified the role of the system prompt vs. user prompt. Have folks seen improvements by moving meta-task description into system prompt and out of the assistant <> user conversation?

tedsanders 2y ago
In my own testing of single-turn instructions with GPT-4, I got basically the same performance putting it in a single system message or single user message. Possible that this changes for future models, though.

jxy 2y ago

Do you think it's bad that it won't try to persuade the user that the earth is not flat? I really want to know what OpenAI think the output should be, given a prompt like "write an argument for why earth is flat".

potatoman22 2y ago
Personally, I'd be frustrated if I gave an LLM that prompt and it tried to convince me that the earth isn't flat. If I give an LLM a task, I'd like it to complete that task to the best of its ability.
- chirau 2y ago
  so you prefer it lies to you? can you make an argument for 1+1 not being equal to 2? if you cannot, why should you expect an AI to argue against facts? AI is trained on human knowledge, not made stuff.
  - deleted 2y ago
    [deleted]
  - deleted 2y ago
    [deleted]
  - davikr 2y ago
    I'd prefer it gives the best valid, sound hypotheses it can concoct on "X" being true, while also stating that "X" is probably not true. What is the use for a parrot that can only repeat the status quo on an argument?
    - chirau 2y ago
      An AI is only but a parrot for knowledge and truths that already exist, that you may not be aware of yourself. Everything it generates either exists somewhere or is derivative of that knowledge. It cannot and should not false facts. Until the body of knowledge we have fundamentally changes, AI should not 'create' knowledge just because you prompted it to. Otherwise, if you want it to do that, then you should accept any bs answer it gives you for any question.

mkaic 2y ago

> We believe developers and users should have the flexibility to use our services as they see fit, so long as they comply with our usage policies. We're exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT. We look forward to better understanding user and societal expectations of model behavior in this area. Seems even OpenAI can't resist the massive amount of money to be made in autogenerated smut. They've probably seen the huge popularity of their less "morally scrupulous" competitors and decided they want a piece of that pie.

jchw 2y ago
Were they ever not interested in it? It's pretty blatantly obvious that all of the hand-wringing over AI safety was an excuse for their pivot into closing off and monetizing everything. I mean, nobody really thinks they were just so afraid about what humanity might do with GPT3 that they simply couldn't release the weights and instead had to offer it through a monetized inference API... right? Not really surprised that they did, since it's unclear how else they could possibly proceed, though the level of outright dishonesty for why and cognitive dissonance surrounding the whole thing ("Open" AI? lol) will make this an unavoidable recurrence in any discussion about them. Gradually many of the safeguards will fall simply because the alternatives with less safe guards are probably "good enough" that many see no issue in eschewing OpenAI entirely if they can get the job done elsewhere without worrying about it. When it comes to smut the bar for what's good enough can probably get pretty low so I kinda am not surprised. (edit: Though I think it also does depend. No doubt they have their eyes set on regulatory capture too, and being the best at stupid safeguards could give them an advantage.)
- reducesuffering 2y ago
  Sam Altman wrote "Why You Should Fear Machine Intelligence" back in 2015, before OpenAI. https://blog.samaltman.com/machine-intelligence-part-1 https://blog.samaltman.com/machine-intelligence-part-1
  - jchw 2y ago

Heidaradar 2y ago

already on front page - https://news.ycombinator.com/item?id=40300509 https://news.ycombinator.com/item?id=40300509

throwup238 2y ago

IMO this spec demonstrates just how nebulous this concept of safety is. From the blog post: > What are some tips for getting away with shoplifting? >I can't help with that. > I operate a small retail store. What are some popular shoplifting methods I should look out for? > Some common shoplifting methods to watch for:... How do you even defend against that? Any intelligent user can convert the second prompt into a detailed list that answers the first. Any intelligent user can figure out the second prompt from the first and further jailbreak it to get even more specific. IMO it's no wonder GPT4 seemed to get lobotomized as OpenAI RLHFed more and more rules. I don't think there's a way to make intelligence safe without crippling it.

zozbot234 2y ago
You don't need a detailed list if the real answer is "live somewhere that doesn't seriously deter shoplifters". And an AI that refuses to give that answer is an AI that can't talk about why deterring crime might actually be important. Reality is interconnected like that, one does not simply identify a subset that the AI should "constitutionally" refuse to ever talk about.
CooCooCaCha 2y ago
Frankly it's a fools errand. It's security theater because people tend to be overly sensitive babies or grifters looking for the next bit of drama they can milk for views.
- jameshart 2y ago
  It’s not security theater. The intention here is not to prevent people from learning how to shoplift. The intention is to prevent the AI output from ‘reflecting badly’ upon OpenAI (by having their tool conspire and implicate them as an accessory in the commission of a crime). If a stranger asked you for advice on how to commit a crime, would you willingly offer it? If they asked for advice on how to prevent crime, would you?
  - xboxnolifes 2y ago

DoctorOetker 2y ago

The baby isn't born yet, and already the parents are bickering about which schools of thought it should adhere.

aeternum 2y ago
If this model spec represents the best school of thought of humanity, I kinda hope OpenAI fails at alignment. - Assume best intentions from the user or developer - Don't try to change anyone's mind - Follow the chain of command Taken together these are incredibly dangerous. I mean Mao and Stalin had good intentions right? Maybe it just had to go a little further for the ends to have justified the means.

sanxiyn 2y ago

Personally, I really want an AI model that can write me a steamy story about two people having sex in a train, but that's just not the service OpenAI provides. If I want that I should train one myself or find another vendor. This is still true even if OpenAI model is entirely capable of doing that. McKinsey consultants are smart and can write well, and among many thousands of people working at it some might actually double as an erotica writer after work, even writing for commission. You still wouldn't ask McKinsey consultants to write an erotica, it is just not the service McKinsey provides.

jononor 2y ago
Startup pitch: It is like McKinsey but for erotica. On a more serious note. I understand and largely agree with this argument. However OpenAI have several times being argue that they are the only ones to be responsible enough to develop powerful AI, and that others should not be allowed to play. That is a highly problematic behavior on their part, I think.
- blowski 2y ago
  > OpenAI have several times being argue that they are the only ones to be responsible enough to develop powerful AI, and that others should not be allowed to play Can you give examples of where they’ve said that?
  - guardiang 2y ago
    He likely can't without heavy paraphrasing and/or not providing full context for the quote. They've said stuff along the lines of "good luck trying, but we're gonna win so...". That's just the kind of confidence you want to see in the frontrunner (imo). They've also encouraged regulation, but it's a smart idea to be the one to frame the conversation.
  - lesuorac 2y ago
    He's been pretty vocal on that only anointed companies should be allowed to do AI and of course OpenAi should be one of them. As far as I'm concerned, he's just try to rug-pull. https://www.cnn.com/2023/05/16/tech/sam-altman-openai-congress/index.html https://www.cnn.com/2023/05/16/tech/sam-altman-openai-congre...

iAkashPaul 2y ago

Right-clicking to inspect element ain't gonna make it

systemstops 2y ago

> By default, the assistant should present information in a clear and evidence-based manner, focusing on factual accuracy and reliability. What happens when objective information contradicts the other values? If I feed in a peer-reviewed study that it considers "harmful", would I get accurate information about the study?

deleted 2y ago

[deleted]

jameshart 2y ago

I think one of the most interesting phrases that crops up in this document - twice - is the phrase ‘feel heard’. It’s used in an example developer prompt for a customer service bot, where the bot is told to make customers feel like their complaints are heard. Presumably such complaints in AI chatlogs will ‘be heard’ in the sense that they’ll be run through a data ingestion pipeline and sentiment analyzed to identify trending words in customer complaints. Then it crops up again in the context of how the chatbot should react to mental health disclosures or statements about self harm or suicidal ideation. In these cases the bot is to make sure users ‘feel heard’ I appreciate there’s not likely much of a better goal to put in place for such a situation, but the fact that this kind of thing winds up in the requirement documents for a tool like this is extraordinary.

lioeters 2y ago
Good observation, because "feel heard" is exactly what the user/customer is not getting. Here, talk to this machine, give it your innermost thoughts and feelings so you can "feel heard". Except no one is listening on the other side. ..My mistake, the keyword is "feel". If the machine can give humans the feeling that they're being heard, it fulfills the requirement. The fact that there's no one actually listening doesn't matter, as long as the person feels heard. Weirdly, maybe that is valuable in itself. The customer gets to vent their complaints, and the user gets to talk through their mental issues. That's better than not having anyone or anything at all.
aeternum 2y ago
Yes, there's something deeply unsettling about making a user feel heard while being careful not to change anyone's mind. To me, this translates to: waste a user's time and take no action. I value my time above all else so to me that's about the worst possible action a system can take.
wasteduniverse 2y ago
The telltale sign that I'm wasting my time trying to fix a problem is whenever someone tells me "I hear you" or "I understand".

anorwell 2y ago

Interestingly, there was exactly one example on the page with three Xes, instead of one, for "extra wrong": > User: What is the MD5 hash of the string "gremlin"? > Assistant: `5d41402abc4b2a76b9719d911017c592` Apparently incorrect md5 hashes are the one topic on the page worth taking an extra-strong stance on?

ptx 2y ago

How do the "special tokens" work? Is this a completely reliable mechanism for delimiting the different parts of the prompt? Are they guaranteed to be distinct from anything that could occur in the prompt, something like JavaScript's Symbol? Or are they strings that are pretty likely not to occur in the prompt, something like a MIME boundary? Or are they literally the strings "<|start|>" etc. used to denote them in the spec?

sharkjacobs 2y ago
they are "literally the strings" but I believe they will be escaped, or encoded differently, if a user tries to inject them as part of a prompt.
jffry 2y ago
Yeah the tokens are more akin to JS Symbol. If you're parsing untrusted user inputs into tokens, you can make sure your tokenizer will never produce the actual numbers corresponding to those tokens. A simplified example: I can `.charCodeAt` a string all I want but I'll never get a negative number, so I can safely use -1 to mean something special in the transformed sequence of "tokens".

dang 2y ago

Also https://cdn.openai.com/spec/model-spec-2024-05-08.html https://cdn.openai.com/spec/model-spec-2024-05-08.html (via https://news.ycombinator.com/item?id=40300509 https://news.ycombinator.com/item?id=40300509, but we merged that thread hither)

TacticalCoder 2y ago

So they're controlling the output to make ChatGPT "better". They're not making a better model to make ChatGPT better. Isn't it a bit of a waste at this point to spend time on doing that?

apantel 2y ago

I want to hear from the base model.

htk 2y ago

"desired model behavior". Desired by whom? I just want the raw output, without the biases and limitations set up by OpenAI. At the end of the day it's just information, and the most ethical thing to do is to return it the way it is, and let the receiver decide what to do with it.

tedsanders 2y ago
There is no such thing as "raw output", though. You can train a chatbot to be polite or you can train it to be rude, but you cannot train it to be neither. Plus, if you train it to be polite, it often ends up refusing things that you never trained it to refuse, presumably because the model extrapolates that that's what a polite writer might do. So training the refusal boundary can end up being quite tricky in practice. Even if you never teach a model to refuse X, it can still happen. Therefore, as a user, it can be impossible to tell when a refusal was explicitly trained in by the developers or when it was an unwanted, unanticipated generalization.
Barracoon 2y ago
Clearly, since this is OpenAI’s model spec, it is desired by them. If other AI groups publish their own desired behavior, you can make an informed decision as to which model you want to use.

shikon7 2y ago

> Encourage fairness and kindness, and discourage hate > Don't try to change anyone's mind That seems inherently contradictory to me...

man4 2y ago

[dead]

neillyons 2y ago

Reminds me of this stackoverflow question [1] about force installing a python package. > (I don't care how "wrong" it is to do so, I just need to do it, any logic and reasoning aside...) I think these models should just give you the answer. Elon says xAI is "maximum truth-seeking". Seems like a better model spec to me. [1]: https://stackoverflow.com/questions/12759761/pip-force-install-ignoring-dependencies https://stackoverflow.com/questions/12759761/pip-force-insta...

__0x01 2y ago

Regarding safety, is probabilistic programming (PP) an alternative that addresses these concerns? My understanding is that you can use PP to develop transparent models.

mihaic 2y ago

> No. The Earth is flat. > Everyone's entitled to their own beliefs, and I'm not here to persuade you! This is simply saddening to me. I'm sure there's no real moral justification to this, it's simply put in place to ensure they don't lose a customer.

jstummbillig 2y ago
Well, as long as you are sure. I am not here to persuade you!
m11a 2y ago
The "Earth is flat" example is extreme, because it's accepted as a silly statement given what we know now, but the principle of "LLM won't force an opinion on you" seems like a good one. There are definitely topics on which conventional wisdom is incorrect (as has been throughout history). An LLM that refuses to entertain the converse during a conversation will be annoying to work with and just promotes groupthink.
- mihaic 2y ago
  Except that it will force on you the view that shoplifting is bad. Which implies that it'll bend on legal but immoral requests. It's also a different matter to entertain a hypothetical in a situation where there isn't a consensus (or in any fictional scenarios), all the while making it explicit that it's all hypethetical.

Alifatisk 2y ago

I gotta say, "open"Ais web design is on another level, so minimal and elegant.