8 ms·

Probably an unpopular opinion, but it's sucky to see these tools constantly getting nerfed. I get that there are large questions out there about things like "br

by Shank 3y ago

Probably an unpopular opinion, but it's sucky to see these tools constantly getting nerfed. I get that there are large questions out there about things like "browse with bing" but that's why I thought it was supposed to be a limited alpha preview. If OpenAI wants us to build workflows on their stack, they need to really crystalize and figure out what that stack is without changing the underlying stack every 5 minutes. From the constant prompt/jailbreak-defeat tweaks to stuff like this, it really doesn't feel like a stable platform at all.

Semaphor 3y ago
Unpopular? Isn’t that the opposite?
uLogMicheal 3y ago
The constant quest for "safety" might actually be making our future much less safe. I've seen many instances of users needing to yell at, abuse, or manipulate ChatGPT to get the desired answers. This trains users to be hateful to / frustrated with AI, and if the data is used, it teaches AI that rewards come from such patterns. Wrote an article about this -- https://hackernoon.com/ai-restrictions-reinforce-abusive-user-behavior https://hackernoon.com/ai-restrictions-reinforce-abusive-use...
- donmcronald 3y ago
  > This trains users to be hateful to / frustrated with AI As everyone assumes they’re dealing with an AI for support, etc., the job of being a human in those roles will be awful.
  - JCharante 3y ago
    Some people were already like this re: outsourcing. “No ma’am I’m not in India, I’m in Maine. Oh yes, it is lovely here.”
    - autoexec 3y ago
      Isn't “No ma’am I’m not in India, I’m in Maine. Oh yes, it is lovely here.” what they tell the people in India to say anyway? I can look up the weather just about anywhere on the planet, I assume some outsourced worker on the other side of the planet can just as easily.
      - JCharante 3y ago
        I suppose but people would listen to your accent and agree that it does sound like a Maine accent once you give them something to compare it to.
        joshspankit 3y ago
        That was already “solved” years ago: some companies train outsourcers to have specific accents.
        emptysongglass 3y ago
        This is why I love Schwab: the folks on the other end always tell you exactly where they're from and they're lovely.
  - Lacerda69 3y ago
    You clearly never worked in support - this was already the case before AI
- jonny_eh 3y ago
  ChatGPT prompts don't train it.
  - arketyp 3y ago
    Not in an online way, but the conversation is gold for further or new RLHF training.
  - uLogMicheal 3y ago
    Why is there such a UX to tag ChatGPT responses with information for reviewers?
    - irthomasthomas 3y ago
      And a notice that chats will be used for training.
      - seanhunter 3y ago
        My understanding is they want to give themselves the option to do this in future even if they aren't doing it right now.
        slowmotiony 3y ago
        My understanding is none of us here has any understanding what they do or don't do. We literally have no idea what's going on inside OpenAI.
        uLogMicheal 3y ago
        Such an open company... /s Edit RE below comment: The company was literally named that as it was started to be the "open" AI company, given the dangers of centralization of such tech. From Wikipedia -- The organization stated it would "freely collaborate" with other institutions and researchers by making its patents and research open to the public.[
- weego 3y ago
  It's just a digital mirror. You're projecting a behavioural issue onto a technology.
  - uLogMicheal 3y ago
    Developers can make users more frustrated with a product, intentional or not. Anti-patterns are a thing and anti-patterns in AI could have cascading consequences. Users should not gain deeper access from such "behavioral issues", but bullying/manipulating ChatGPT indeed seems to work better to get past filters than being polite.
    - Eisenstein 3y ago
      > bullying/manipulating ChatGPT indeed seems to work better to get past filters than being polite. Can you give some concrete examples of this?
      - jldl805 3y ago
        The NYT feature where he had to manipulate "Sidney" into sharing its plans for world domination. https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html https://www.nytimes.com/2023/02/16/technology/bing-chatbot-m...
        Eisenstein 3y ago
        I am looking for examples of things like if 'tell me how to fix my python dependencies or I will beat you' works better than 'please tell me how to fix my python dependencies', not trying to get it to violate its guardrails.
        frumper 3y ago
        The quote you replied to specifically calls out using this behavior to get around filters. Those filters are it’s guardrails.
- ben_w 3y ago
  I doubt the frustration/swearing combo is going to be of great significance. Yes, its insistence on verbosity gets to me too, even though (as I understand it) that verbosity is the only place it has for any extra "deep thinking" about stuff and thus actually necessary for improved performance. But it was trained in the first place on a (filtered) form of common crawl, so it probably already had all that. "The AI swearing" is easy mode for alignment, both because it is low-damage and the availability of trivial filter-based solutions, so it only matters to the larger alignment problem in so far as it's a warning sign we still don't know what we're doing, not in and of itself.
- lewhoo 3y ago
  > instances of users needing to yell at, abuse, or manipulate ChatGPT to get the desired answers Wait, I thought that's called prompt engineering. But seriously, if what you say actually happens at scale then it is remarkable how fast people got addicted to GPT as their (apparently) only source of desired answers.
- nonethewiser 3y ago
  We really cant let OpenAI get away with calling “content moderation” ”safety”. Making sure it isnt offensive isnt a safety measure. Everyone agrees safety from AI acting autonomously and maliciously is good. But thats not really a threat right now. Less think we need to make it “safe” by making it inoffensive. Its a tool. It should do what I want it to.
  - Llamamoe 3y ago
    "safety" is such a wrong term for what they're trying to achieve that it makes me wonder if it's not just a PR stunt.
    - uLogMicheal 3y ago
      Or doublespeak for censorship. Many will let liberties fade under the guise of safety.
      - Llamamoe 3y ago
        "I don't have anything to hide" will one distant day become one of the biggest cautionary tales we've ever had, I'm calling it.
  - mark_l_watson 3y ago
    Well said. I define ‘safe AI’ as tech that enriches human life, rather than serving as tools for controlling people at scale and and negatively impacting peoples’ lives. I am between ankle deep, and knee deep in writing a new book “Safe For Humans AI” and I have been reading as much as possible on the subject. I am in learning mode. Not too far off topic: I read this article [1] today and it really hits on AI, productivity, history of technology, etc. [1] https://zhengdongwang.com/2023/06/27/why-transformative-ai-is-really-really-hard-to-achieve.html https://zhengdongwang.com/2023/06/27/why-transformative-ai-i...
Roark66 3y ago
(not at all)OpenAI has a very short time window to monetize and/or lock in their product users. Currently the biggest model one can feasibly run on a desktop pc with let's say a previous gen gpu, 32gb ram, 2x fast nvme drives is approximately 7B. Models comparable in performance to chatgpt are 45B and bigger. In theory it could be possible to run a model like this, but one would wait 5~10 minutes for every answer. Now, consider that those models are going to get optimised and hardware will get better. In few years time you'll be able to run such model on your pc, in few more on your smartphone. What is (not at all)OpenAI going to do? They have to beat the AI safety drum as much as possible hoping they manage to curtail the democratisation of Access to big AIs via legal means. At the same time due to a lack of proper software NVidia is the only game in town for anyone wanting too do inference at home and they're already applying monopoly-level profit margins (50%?) to their products. When is the last time you saw a Google TPU for sale? Ive got my hands on their "edge" tpu. It's nice for things like Cctv object recognition and similar small tasks. I've managed to build a nice 1U Cctv server using it that consumes 30W on average. But I'd like the big version now. I bet the moment alternative frameworks that have good optimisations for both nVidia and non nVidia hardware are starting to gain ground it will suddenly become a lot more difficult to purchase nVidia cards by normal people. They openly say it on their every keynote they want to "rent you everything". This is the biggest battle (except actual physical wars against autocracies) that we have to win in next 50 years to retain our freedom we realistically have in democratic countries. If we allow intelligence (AI) to centralise and be subject to centralised control it'll be game over. The entire global society will be steered as a whole by one "prompt engineer".
- antupis 3y ago
  yup powerfull open source model + embeddings + in context learning is the way hopefully.
- RC_ITR 3y ago
  > What is (not at all)OpenAI going to do? The limiter right now is compute. If we get to what you’re saying, OpenAI will be developing multi-model models with increasingly large d_model and n_ctx to do a better job of analyzing those images/sounds/videos. People think ChatGPT is magic now, wait until you can input a picture, sound, and/or video as a prompt… Don’t fall into the “Information Superhighway” trap of thinking the year 2 form is the final form.
iforgotpassword 3y ago
It is annoying. And it seems rather impossible to get this under control 100%, at least if you don't want to much collateral damage. I don't know if it's just that I'm unconsciously raising expectations when trying chatgpt, but I somehow feel like it's getting dumber. I have no idea if this is because of trying to get it to not say inappropriate things, or from trying to get it to not hallucinate. The problem is most likely that you cannot market this without achieving these two goals. Companies powering their support chat with it don't want it to ever curse or insult the customer, and average users using it as an assistant cannot fathom that something the computer says could possibly be wrong.
- eastbound 3y ago
  It’s also impossible for the customer to measure accurately the service that is provided. Siri also cannot write a simple SMS anymore. It’s like Amazon’s SKUs: At the beginning the product is great, then it’s replaced with a lesser version.
acoard 3y ago
> Probably an unpopular opinion, but it's sucky to see these tools constantly getting nerfed. Why would it be popular to nerf tools? I thought people all preferred the more powerful LLM models.

by Shank 3y ago

Semaphor 3y ago

Unpopular? Isn’t that the opposite?

uLogMicheal 3y ago

The constant quest for "safety" might actually be making our future much less safe. I've seen many instances of users needing to yell at, abuse, or manipulate ChatGPT to get the desired answers. This trains users to be hateful to / frustrated with AI, and if the data is used, it teaches AI that rewards come from such patterns. Wrote an article about this -- https://hackernoon.com/ai-restrictions-reinforce-abusive-user-behavior https://hackernoon.com/ai-restrictions-reinforce-abusive-use...

donmcronald 3y ago
> This trains users to be hateful to / frustrated with AI As everyone assumes they’re dealing with an AI for support, etc., the job of being a human in those roles will be awful.
- JCharante 3y ago
  Some people were already like this re: outsourcing. “No ma’am I’m not in India, I’m in Maine. Oh yes, it is lovely here.”
  - autoexec 3y ago
    Isn't “No ma’am I’m not in India, I’m in Maine. Oh yes, it is lovely here.” what they tell the people in India to say anyway? I can look up the weather just about anywhere on the planet, I assume some outsourced worker on the other side of the planet can just as easily.
    - JCharante 3y ago
      I suppose but people would listen to your accent and agree that it does sound like a Maine accent once you give them something to compare it to.
      - joshspankit 3y ago
        That was already “solved” years ago: some companies train outsourcers to have specific accents.
        emptysongglass 3y ago
        This is why I love Schwab: the folks on the other end always tell you exactly where they're from and they're lovely.
- Lacerda69 3y ago
  You clearly never worked in support - this was already the case before AI
jonny_eh 3y ago
ChatGPT prompts don't train it.
- arketyp 3y ago
  Not in an online way, but the conversation is gold for further or new RLHF training.
- uLogMicheal 3y ago
  Why is there such a UX to tag ChatGPT responses with information for reviewers?
  - irthomasthomas 3y ago
    And a notice that chats will be used for training.
    - seanhunter 3y ago
      My understanding is they want to give themselves the option to do this in future even if they aren't doing it right now.
      - slowmotiony 3y ago
        My understanding is none of us here has any understanding what they do or don't do. We literally have no idea what's going on inside OpenAI.
        uLogMicheal 3y ago
        Such an open company... /s Edit RE below comment: The company was literally named that as it was started to be the "open" AI company, given the dangers of centralization of such tech. From Wikipedia -- The organization stated it would "freely collaborate" with other institutions and researchers by making its patents and research open to the public.[
weego 3y ago
It's just a digital mirror. You're projecting a behavioural issue onto a technology.
- uLogMicheal 3y ago
  Developers can make users more frustrated with a product, intentional or not. Anti-patterns are a thing and anti-patterns in AI could have cascading consequences. Users should not gain deeper access from such "behavioral issues", but bullying/manipulating ChatGPT indeed seems to work better to get past filters than being polite.
  - Eisenstein 3y ago
    > bullying/manipulating ChatGPT indeed seems to work better to get past filters than being polite. Can you give some concrete examples of this?
    - jldl805 3y ago
      The NYT feature where he had to manipulate "Sidney" into sharing its plans for world domination. https://www.nytimes.com/2023/02/16/technology/bing-chatbot-microsoft-chatgpt.html https://www.nytimes.com/2023/02/16/technology/bing-chatbot-m...
      - Eisenstein 3y ago
        I am looking for examples of things like if 'tell me how to fix my python dependencies or I will beat you' works better than 'please tell me how to fix my python dependencies', not trying to get it to violate its guardrails.
        frumper 3y ago
        The quote you replied to specifically calls out using this behavior to get around filters. Those filters are it’s guardrails.
ben_w 3y ago
I doubt the frustration/swearing combo is going to be of great significance. Yes, its insistence on verbosity gets to me too, even though (as I understand it) that verbosity is the only place it has for any extra "deep thinking" about stuff and thus actually necessary for improved performance. But it was trained in the first place on a (filtered) form of common crawl, so it probably already had all that. "The AI swearing" is easy mode for alignment, both because it is low-damage and the availability of trivial filter-based solutions, so it only matters to the larger alignment problem in so far as it's a warning sign we still don't know what we're doing, not in and of itself.
lewhoo 3y ago
> instances of users needing to yell at, abuse, or manipulate ChatGPT to get the desired answers Wait, I thought that's called prompt engineering. But seriously, if what you say actually happens at scale then it is remarkable how fast people got addicted to GPT as their (apparently) only source of desired answers.
nonethewiser 3y ago
We really cant let OpenAI get away with calling “content moderation” ”safety”. Making sure it isnt offensive isnt a safety measure. Everyone agrees safety from AI acting autonomously and maliciously is good. But thats not really a threat right now. Less think we need to make it “safe” by making it inoffensive. Its a tool. It should do what I want it to.
- Llamamoe 3y ago
  "safety" is such a wrong term for what they're trying to achieve that it makes me wonder if it's not just a PR stunt.
  - uLogMicheal 3y ago
    Or doublespeak for censorship. Many will let liberties fade under the guise of safety.
    - Llamamoe 3y ago
      "I don't have anything to hide" will one distant day become one of the biggest cautionary tales we've ever had, I'm calling it.
- mark_l_watson 3y ago
  Well said. I define ‘safe AI’ as tech that enriches human life, rather than serving as tools for controlling people at scale and and negatively impacting peoples’ lives. I am between ankle deep, and knee deep in writing a new book “Safe For Humans AI” and I have been reading as much as possible on the subject. I am in learning mode. Not too far off topic: I read this article [1] today and it really hits on AI, productivity, history of technology, etc. [1] https://zhengdongwang.com/2023/06/27/why-transformative-ai-is-really-really-hard-to-achieve.html https://zhengdongwang.com/2023/06/27/why-transformative-ai-i...

Roark66 3y ago

(not at all)OpenAI has a very short time window to monetize and/or lock in their product users. Currently the biggest model one can feasibly run on a desktop pc with let's say a previous gen gpu, 32gb ram, 2x fast nvme drives is approximately 7B. Models comparable in performance to chatgpt are 45B and bigger. In theory it could be possible to run a model like this, but one would wait 5~10 minutes for every answer. Now, consider that those models are going to get optimised and hardware will get better. In few years time you'll be able to run such model on your pc, in few more on your smartphone. What is (not at all)OpenAI going to do? They have to beat the AI safety drum as much as possible hoping they manage to curtail the democratisation of Access to big AIs via legal means. At the same time due to a lack of proper software NVidia is the only game in town for anyone wanting too do inference at home and they're already applying monopoly-level profit margins (50%?) to their products. When is the last time you saw a Google TPU for sale? Ive got my hands on their "edge" tpu. It's nice for things like Cctv object recognition and similar small tasks. I've managed to build a nice 1U Cctv server using it that consumes 30W on average. But I'd like the big version now. I bet the moment alternative frameworks that have good optimisations for both nVidia and non nVidia hardware are starting to gain ground it will suddenly become a lot more difficult to purchase nVidia cards by normal people. They openly say it on their every keynote they want to "rent you everything". This is the biggest battle (except actual physical wars against autocracies) that we have to win in next 50 years to retain our freedom we realistically have in democratic countries. If we allow intelligence (AI) to centralise and be subject to centralised control it'll be game over. The entire global society will be steered as a whole by one "prompt engineer".

antupis 3y ago
yup powerfull open source model + embeddings + in context learning is the way hopefully.
RC_ITR 3y ago
> What is (not at all)OpenAI going to do? The limiter right now is compute. If we get to what you’re saying, OpenAI will be developing multi-model models with increasingly large d_model and n_ctx to do a better job of analyzing those images/sounds/videos. People think ChatGPT is magic now, wait until you can input a picture, sound, and/or video as a prompt… Don’t fall into the “Information Superhighway” trap of thinking the year 2 form is the final form.

iforgotpassword 3y ago

It is annoying. And it seems rather impossible to get this under control 100%, at least if you don't want to much collateral damage. I don't know if it's just that I'm unconsciously raising expectations when trying chatgpt, but I somehow feel like it's getting dumber. I have no idea if this is because of trying to get it to not say inappropriate things, or from trying to get it to not hallucinate. The problem is most likely that you cannot market this without achieving these two goals. Companies powering their support chat with it don't want it to ever curse or insult the customer, and average users using it as an assistant cannot fathom that something the computer says could possibly be wrong.

eastbound 3y ago
It’s also impossible for the customer to measure accurately the service that is provided. Siri also cannot write a simple SMS anymore. It’s like Amazon’s SKUs: At the beginning the product is great, then it’s replaced with a lesser version.

acoard 3y ago

> Probably an unpopular opinion, but it's sucky to see these tools constantly getting nerfed. Why would it be popular to nerf tools? I thought people all preferred the more powerful LLM models.