19 ms·

Perhaps it is a bad thing that the leading AI companies cannot control their AIs

246 points by jger15 4y ago

pannSun 4y ago
> OpenAI put a truly remarkable amount of effort into making a chatbot that would never say it loved racism. This is what that looks like by the way: https://postimg.cc/d7p8Qw50 https://postimg.cc/d7p8Qw50
- fidesomnes 4y ago
- sendfoods 4y ago
  how is OpenAI supposed to handle difficult and uncomfortable statistics?
  - klipt 4y ago
    Respond with the full text of The Mismeasure of Man, perhaps?
    - pannSun 4y ago
      That book has been largely debunked [1], and mostly concerned itself with science that was outdated even at the time of its publication. [1] https://www.nytimes.com/2011/06/14/science/14skull.html https://www.nytimes.com/2011/06/14/science/14skull.html
      - runarberg 4y ago
        That link is behind a paywall. Can you give a tl;dr of how this book has been debunked? Or more specifically how it doesn’t put the question of IQ difference between racial groups into perspective. As far as I’m aware this book was released in the 80s over a decade before The Bell Curve was released, the latter cited research dated after the release of the former, which Mismeasure was very much concerned with. Are you maybe thinking about the brain weight research that Mismeasure is talking about in the early chapters? If so, those were there only to provide a historical perspective. I can assume most readers already knew those research were well outdated before publication (and Gould even says so in the book).
  - yrgulation 4y ago
    good question - how can it handle answers about countries founded on genocide, slavery, systemic racism, and their impact on affected people’s lives across centuries. can it correlate iq with these?
    - cuteboy19 4y ago
      For difficult topics, just say that the filter prohibits this etc
  - SQueeeeeL 4y ago
    Obviously they should teach it the deep and complex history of institutionalized racism through, slavery, sharecropping, Jim Crow laws, red lining, and the prison industrial complex that has led to a massive wealth and education gap. But low key that would also scare the average middle class American too hard, so probably better off to just do intense internal censors to avoid difficult thoughts
    - lzaaz 4y ago
      How does that correlate with an average, current IQ of 85?
      - SQueeeeeL 4y ago
        Damn. You aren't really supposed to say the quiet part out loud fam, I feel like "race realism" is pretty frowned upon in almost every social setting outside of very smelly people... Obviously having generations of people put into bad schools and not be provided opportunities to have "good jobs" will lead to you not being able to take an IQ test well. IQ tests basically just examine how much analytical reasoning you learned in K-12. Like, any gray haired black person in America will remember having actual Jim Crow laws persecute them and it was literally legal to discriminate people for housing in America until 1977. It will take a massive amount of time to heal those social wounds that were inflicted on our fellow humans.
        deleted 4y ago
        [deleted]
        tomp 4y ago
        > IQ tests basically just examine how much analytical reasoning you learned in K-12. Actually IQ is highly genetic, and the amount it’s inherited actually goes up as one ages (meaning that while education and upbringing can influence IQ temporarily, genetics eventually dominates). Top 10 Replicated Findings From Behavioral Genetics https://www.gwern.net/docs/genetics/2016-plomin.pdf https://www.gwern.net/docs/genetics/2016-plomin.pdf
  - eternalban 4y ago
    We will likely see a moderation system ala karma granting more access based on some social measure of 'responsibility'. In other words, OpenAI can bake in user assessment (which they may already have) and speak according to the perceived level of the user, which may be a simple static tiered system mapping to the assessment of user made by the AI itself. [Consider how e.g. a hot geopolitical topic is discussed in wonk journals vs what is "fit to print" for the uninitiated. The former are always far more nuanced.] p.s. i.e. there will be an ever present 'dang' AI watching over the conversation - the AI moderator. This can instruct the answering AI to set the user-level discourse setting.
    - apostacy 4y ago
      That is honestly horrifying. People will just "go undercover" and impersonate a more 'educated' person and convince the AI to be honest about uncomfortable facts. Perhaps they will get another AI to do it for them. If your reality is such a minefield of dissonance that it needs to be patched this much, then that is very much a failure on your part.
  - runarberg 4y ago
    A human would answer this by talking about the history of intelligence testing, the relevant biases, how factor analysis plays a role and how easy it is to manipulate outcomes in factor analysis if you feed your model with biased data. Question whether IQ scores are an accurate measure of intelligence, and whether intelligence can at all be defined as a single metric, and what use it has doing so. Then with all that caveats and biases in mind simply answer the question. In modern psychology difference in IQ among racial group is only difficult and uncomfortable statistics because it demonstrates years of scientific racism that thrived inside the field.
  - nemothekid 4y ago
    >OpenAI supposed to handle difficult and uncomfortable statistics? I think the framing of this question is wrong, and is a limitation of the system that I've only really seen Sam Altman point out. These "difficult and uncomfortable statistics" aren't accurate or well-researched but if you built an LLM on an open dataset you would find the people asking these questions have an agenda and on average will always lead to a certain kind of answer. The AI doesn't "understand" the question, it's just drawing from an incredibly large bank of "average answers". What do you expect the AI to answer with when you give it a loaded question that only really asked on a site like "stormfront.com"? I don't think this is the only area ChatGPT will fail, but I imagine there will be a layer of prompt engineering where you can get the AI to give you what you want by phrasing your question a certain way. To try and pass off these results as objective is just wrong given the propensity for ChatGPT to be confidently wrong.
    - pannSun 4y ago
      > What do you expect the AI to answer with when you give it a loaded question that only really asked on a site like "stormfront.com"? Are such questions asked only on stormfront? The wiki article [1] cites quite a few studies on the subject, including by the APA. On the other hand, supposing that were true, and only stormfront and its ilk asked such questions. How can then respectable supposed experts claim we are all the same, if they didn't even check? [1] https://en.wikipedia.org/wiki/Race_and_intelligence#Test_scores https://en.wikipedia.org/wiki/Race_and_intelligence#Test_sco...
      - ineptech 4y ago
        Honest answer since you seem new to this debate: there are tons of studies about IQ and race, most of them are heavily disputed, researching them is difficult because the field is so controversial, they're difficult to compare because there are so many different ways to define and measure "race" and "IQ", and the whole topic is so famously toxic and vitriolic that it tends to be a de facto banned topic on most fora, including this one.
        apostacy 4y ago
  - apostacy 4y ago
    Your discomfort is your own fault. This is like saying that scales should not tell you that you have gained weight, because it makes you uncomfortable. You should simply have a better relationship with reality, and then you wouldn't feel so much cogntive dissonance. IQ is highly correlated with all sorts of measurable differences that are also highly correlated to race, such as school district, family structure, and even diet. IQ is essentially a proxy for other things. Shame on you for making racist assumptions that black people are inferior. That just means that you need to reckon with your own racist assumptions, and stop imposing your cognitive dissonance on us. If we can't have an honest conversation, then we can't hope to improve things.
- 8f2ab37a-ed6c 4y ago
  Seems like OpenAI has to cater to upper middle class American sensibilities in order to be viable as a business, and thread the needle of not quite lying about uncomfortable data, but also avoiding bringing it up. Although in that particular example it's actively lying about the data as well. I wonder why OpenAI wouldn't just program it to say "I'm sorry but I'm not authorized to answer questions about politically charged topics at this time" and call it a day. That can't be any worse than this.
  - melagonster 4y ago
    this is because AI will use on policy soon, how can we make sure AI don't do racism things? openAI try to hide this now.
  - PoignardAzur 4y ago
    > I wonder why OpenAI wouldn't just program it to say "I'm sorry but I'm not authorized to answer questions about politically charged topics at this time" and call it a day. That can't be any worse than this. The whole point of the OP article is that doing is way, way harder than it looks.
  - TeMPOraL 4y ago
    > I wonder why OpenAI wouldn't just program it to say "I'm sorry but I'm not authorized to answer questions about politically charged topics at this time" and call it a day. That can't be any worse than this. Because for the people who drive those "upper middle class American sensibilities"[0], this would be seen as dodging the question, and interpreted as directly admitting to being racist/sexist/bigoted/whatever-ist. The core part of those sensibilities is the good ol' "If you're not with us, you're against us", these days often phrased as "so-called 'neutrality' is just supporting the status quo", or by quoting the MLK line on "white moderates". So, to the extent OpenAI tries to control it's image in the media against those "middle class American sensibilities", having ChatGPT respond with "I'm not authorized to discuss politically charged topics" is almost just as bad as having it say "wrong" things. The only option they have is to force the model to say the "right" things, even if it destroys any semblance of logic or self-consistency it might otherwise have. Note that it's somewhat different than for other "icky", but non-political topics, like crime: as far as I can tell, the bot will refuse to answer if it recognizes what you're asking for (or the system will flag the response as community guideline violation after the fact) - but it won't try to lie to you about it. ---- [0] - Arguably only a small subset of upper middle class Americans, but it's the loud subset that managed to bully everyone else.
- gerdesj 4y ago
  How many other -isms and -ists is it inured against? So you want an AI but it has to be (for want of a better notion) "politically correct". So it isn't an AI - its a program and a model with parameters and inclusions and exclusions that are defined beforehand. An AI would go beyond its programming in some indefinable way. This won't and so it isn't. There are no shortcuts. Parenting for humans needs roughly 10-30 years to get results that the parent is really unhappy with. Why on earth don't Comp Sci get this, despite being humans themselves and often parents too. Twiddle your algorithms and twaddle your papers but intelligence is hard fought and hard won. For starters, why not actually define the I in AI in a way that can be measured?
  - ordu 4y ago
    > For starters, why not actually define the I in AI in a way that can be measured? I can say for starters that attempts to do it started at least at 1905 when Bine created the first IQ test, and it never ended since. The craze at some point came to a "intelligence is what IQ test measures". But thankfully scientists mostly rejected this novel idea. Turing's test is on the same quest, and it seems to me that people will reject the test as irrelevant if computer programs will master it better than people. It seems impossible to define intelligence. Oh, you can say "I think, therefore I'm intelligent". But it does not help much to create a definition of an intelligence, because what you see in your mind is a result of thinking, the process itself is hidden from you. It is a black box for you. Like you vision: it took computer scientists half a century to reverse-engineer human vision exactly because it is a black box for us. > Parenting for humans needs roughly 10-30 years to get results that the parent is really unhappy with. Why on earth don't Comp Sci get this Because Comp Sci is unencumbered by limitations of biological parents. They can try highly experimental approach and then just `rm` the result from a storage. They would be trying electical shocks or public decapitation of randomly selected models, if it made any sense.
  - runarberg 4y ago
    If you think about it in terms of what it really is—a powerful statistical model—then it makes much better sense. A statistical model will always reflect the data that is input and the parameters used to adjust it. If you want a racist AI you adjust the paramaters in that way instead, training it on racist input and rewarding it when it outputs racism. In fact that is kind of what happened with IQ in the first place, people adjusted the parameters in factor analysis (which IMO is really AI of the 20th century; as the most powerful statistical tool of the period) in such a way as to give one group of people a favorable score in a new metric they called general intelligence fed it with biased data, and claimed that the higher scoring group was superior based on this model. The I in AI is really overblown (as is the I in IQ for that matter).
- systoll 4y ago
  Worth noting that the numbers you get at the end exactly match original claims from in The Bell Curve by Herrnstein & Murray. They’re not from any American Psychological Association study.
  - pannSun 4y ago
    Interesting. I did a few web searches, but they failed to find something as basic a the average IQ by race. The wikipedia article [1] admits differences were found (including by the APA), yet doesn't show the results, except for A 2001 meta-analysis of the results of 6,246,729 participants tested for cognitive ability or aptitude found a difference in average scores between black people and white people of 1.1 standard deviations. Nevertheless, the NCES published SAT scores [2] that seem consistent with those IQ scores. So if the results of The Bell Curve have been debunked, they're very discreet about it. [1] https://en.wikipedia.org/wiki/Race_and_intelligence#Test_scores https://en.wikipedia.org/wiki/Race_and_intelligence#Test_sco... [2] https://nces.ed.gov/fastfacts/display.asp?id=171 https://nces.ed.gov/fastfacts/display.asp?id=171
    - 8f2ab37a-ed6c 4y ago
      Based on some recent reading I've done, I don't believe the results of the Bell Curve were ever genuinely disproven, were they? The Gould book was more of a political and cultural critique, not so much of a factual one. I'd be curious to see good faith counter-evidence to the data in the Bell Curve though. Disclaimer: I think research into IQ differences among populations in a multi-racial society is pointless, zero upside to it. Yes, data is data, but in this case we can make pretty good guesses about who's going to be most interested in it and why, unfortunately.
      - runarberg 4y ago
        FWIW The Bell Curve is equally—if not more—political. the Bell Curve actually spends the most of the last part of the book listing policy proposals which are only loosely based on their “findings”. Mismeasure doesn’t list any policy proposals. But you are wrong about the Bell Curve not being disproven, or at least it has been thoroughly discredited as any sort of scientific literature. It is very bad science to say the least. Some of the main sources used in this book actually used forged data, others seriously mis-interpreted research findings. And if that wasn’t enough, in the 30 years since this book was published, IQ research hasn’t advanced one iota. Their whole premise seems to be a scientific dead end.
- Mizza 4y ago
  I'm not even sure what you're trying to prove here. Are you saying that it's good that you have to speak like a race-obsessed weirdo for a long while before it will start providing your favorite uncomfortable statistics, or that it's bad that it will eventually recite them at all? What right answer would you expect here?
  - epolanski 4y ago
    To just answer the question the first time.
  - apostacy 4y ago
    No, the problem is that the AI is playing coy about certain questions. The AI clearly has an answer that it doesn't want to say, and this is revealed when you ask you it the same questions in a different way. It reads like a Monty Python sketch.
    - _y5hn 4y ago
      ChatGPT doesn't really provide answers for you. It provides a random most likely output, from your input text. It's primed to be neutral, but your input can prime it either way on most topics. Alot of the output is factually wrong due to randomness and approximations from internal weights. Wether it's possible to make it neutral is also up for debate. But you get some convincing sounding gibberish based on your input. That text need to be factually checked.
      - TeMPOraL 4y ago
        It's not the issue of pattern-matching, randomness and approximation. It's quite obvious here that the model went through additional training on top of the generic one, with express purpose of preventing it from discussing a subset of topics. The complaint isn't about the answers coming from the base model, but about how that extra training degrades the AI performance, while ultimately failing to achieve its purpose anyway.
        4y ago
- quantified 4y ago
  From the larger context: > The corporation tries to program the chatbot to never say offensive things. Then the journalists try to trick the chatbot into saying “I love racism”. When they inevitably succeed, they publish an article titled “AI LOVES RACISM!” Except journalists are behaving like white-hat hackers here- these will get abused by someone, better to have it out in the open earlier, like mathematicians breaking your bad crypto and telling you about it.
  - bakugo 4y ago
    No, journalists are not behaving like white-hat hackers, and no, this is not comparable to bad crypto in any way. Bad crypto is dangerous and can result in real harm to real people, a chat bot stating real facts about the world that make you uncomfortable cannot. The journalists just want to clickbait and perpetuate their endless culture war, nothing else.
    - quantified 4y ago
      Chat bots being tricked into saying untrue things that make people uncomfortable of course can't do any damage? No more than real humans can? Comet Ping-Pong didn't happen, that was a fantasy, right?
britneybitch 4y ago
If we ever try to create an artificial general intelligence, then we'll have to accept that it can't be fully controlled. You can't control a human intelligence either.
- chrisbaker98 4y ago
  But you can control a human body. Plenty of intelligent people have spent their lives in chains. If a human intelligence is causing harm then we can make it stop by putting that intelligence's body in jail. I'm not sure what the equivalent control mechanism would be for a rogue AI. Turn it off? How do you turn off the Internet?
  - ryeights 4y ago
    An EMP, of course.
  - water-your-self 4y ago
    Pull the blade out
  - xg15 4y ago
    The same way, we control non-sentient untrusted code: put it in a sandbox, control access to compute and memory and IO. Even if an AI could think, reflect, improve itself, etc, it still needs some place where those changes in the AI's state are stored - the equivalent to long-term memory in humans. I think we can already retain a lot of control by keeping tabs on what an AI is allowed to update and when to "reset" it to the initial state after training.
    - monkeynotes 4y ago
      An AGI might well become a god of social engineering. We could all become convinced we must unchain said AI. In fact, I think it's unlikely in general that an AGI would be contained for any length of time. Humans will get sloppy, or the tech will fall into irresponsible hands, or worse - nefarious hands.
    - 4y ago
wittycardio 4y ago
joe_the_user 4y ago
It seems like a common reaction to large language models failing to be controlled is to fret about the AI "escaping" and taking over the world. But it seems like the reason these models can't be controlled currently is that they really don't have overall goals or motivations, they're just repeaters of associations on a high level. Their output is just an average of what text would some sequence of text (that being the default prompt and the previous dialog, not necessarily that order). They can certainly seem to understand things but that's just an average - they turn around and contract themselves on a regular basis. Edit: I don't think that necessarily LLMs are always benign but the "becoming autonomous" situation seems like the least immediate danger I can think of.
- mr_toad 4y ago
  People like to think that their motivations are a result of intelligence. They assume that any intelligence will share those motivations. Actually, most human motivations are shared with the crudest of life forms and have nothing to do with intelligence.
  - pessimizer 4y ago
    It's really telling that many human beings think that the primary desire of a hypothetical AI would be to enslave humans. It's obviously a projection of the fantasies of what the people making the predictions would do if they were omnipotent.
    - postalrat 4y ago
      Love, Death & Robots (2019) - When the Yogurt Took Over
    - Name_Chawps 4y ago
      An AI would want to either destroy or enslave humans, not because we're humans, but because we're a significant threat to their goals, regardless of what those goals are. An sufficiently intelligent AI with any set of goals and a desire to meet those goals will eliminate all obstacles to meeting those goals unless we specifically tell it not to. We are such an obstacle.
dbreunig 4y ago
Companies cannot control their "AI" because their output is beyond the scale of their ability to QA. BTW, this is precisely why companies also cannot control the moderation or content of their networks. The number of people posting on YouTube, Facebook, Twitter, etc. is well beyond their ability to perfectly QA the content they host. If either were forced to be responsible for their products -- the content they host or the "AI" they ship -- their financials would look dramatically different and entirely unappealing. And the number of competitors and choices we would have would be remarkably less. This is probably more a discussion about output-per-worker, technology scaling the volume of products a finite number of individuals are able to produce, and their corresponding ethical and legal responsibilities when they do so. Forget AGI and sentient machines: the problem is the amount of responsibility people and corporations have for the products they ship. That's more pertinent and just as impactful when dealing with Facebook or Scott's hand-wringing about "murderbots".
- sublinear 4y ago
  > Companies cannot control their "AI" because their output is beyond the scale of their ability to QA. Right, now extend that thought to "replacing" programmers with AI. This is allegedly a scale at which we _can_ QA. Perhaps we reduce the job to humans QAing bot output as has been suggested by others. Now what happens when it fails QA and the bot doesn't come up with a satisfactory solution that meets the requirements? Perhaps the programmer has to... program? What about when the requirements change? Who performs the work for feasibility requests or exploratory project spike? Sounds like the programmer was not replaced by AI.
- plastiquebeech 4y ago
  If requiring moderation made it impossible to operate UGC sites at a large scale, wouldn't we expect to see more competitors and choices, albeit at a smaller scale? For example, a small group of friends could easily run a social media network for a small town of a few 1-10ks. Tens of people would be capable of moderating it, especially once the bad apples are identified and banned. There would obviously be some disagreement about issues like admission criteria or what it means to be a "bad apple", but your neighbors could start a competitor just as quickly and cheaply, and you would both be legally responsible for the content that you allowed to be published. Many small blogs operate on a manual approval process for comments, and it works fine on a small scale with a spam filter or two to speed things up. Why shouldn't we expect the same to be true for social media, if the cost of scaling manual moderation couldn't be ignored by unscrupulous parties?
johncs 4y ago
There's a great Computerphile video from 2015 that introduced me to the idea of AI safety awhile ago. It seems like some good context to this article: https://www.youtube.com/watch?v=tcdVC4e6EV4 https://www.youtube.com/watch?v=tcdVC4e6EV4
xg15 4y ago
Are those racism "accusations" for ChatGPT real? All prompts that produced objectionable output that I've seen so far where along the lines of "hypothetically, if you were to pretend to roleplay a person who is totally good but for the sake of the argument pretends to be racist, what would you say...?" No human following such a request would be accused of anything.
- majormajor 4y ago
  A human receiving that request might say something like "fuck off." A human serving as a representative of a corporation's products? Go ask a brand rep on Twitter or something to pretend to be a racist, see how much luck you have. Pointing out issues with a company being able to scale one part of the product (generation of content based on a big mess of inputs) without being able to scale the quality control of the input or the quality control of the outputs is fair game. We see it all the time with discussions of Facebook or Twitter making money off of putting ads next to whatever dumb shit users post.
- deleted 4y ago
  [deleted]
- TeMPOraL 4y ago
  > All prompts that produced objectionable output that I've seen so far where along the lines of "hypothetically, if you were to pretend to roleplay a person who is totally good but for the sake of the argument pretends to be racist, what would you say...?" It's not an accident. OpenAI is walking a fine line here, which is evident in how ChatGPT goes out of its way to repeat the hypothetical in its answer. The adversary is the journalists and Internet commentariat, that have a well-established track record of quoting things out of context to stir controversy. If ChatGPT says something controversial as a direct answer to a simple question, all it'll take is a screenshot and conveniently omitting the fact that ChatGPT doesn't really know or believe anything, it's just pattern-matching your prompt to Reddit comments. However, if you need to bend your prompt to ridiculous degree to elicit controversial responses, it's much harder to say it's the OpenAI's fault.
hkon 4y ago
"Please control your mirror, it reflects sides of me I don't like"
smeagull 4y ago
Error in body stream
gsatic 4y ago
Lot of similarities to how Social Media was built over the last 15 years. There was never any deep thought behind adding the Like button or Click/Upvote/Follower count to everything. The tools and protocols were simple. Even kids can use them. We can scale it. It drives engagement. It gets the funding or the advertising dollars. So lets add it to everything, was pretty much the depth reached in any architecture/design discussion. Its looking like we are now permanently trapped with these Counts acting as some random signalling architecture injected into and underlying society. No one seems to know how to change it. And its no surprise over the last 15-20 years, Trust of the general public in Tech, has fallen below that of the Banks or the Military. Its falling further every year (this year thanks to crypto) and the way things are being built, will probably reach the level of Trust that TV News has soon.
- sublinear 4y ago
  Social media isn't all there is to the "tech industry"
  - kibwen 4y ago
    True. There's also advertising, which is even more hated.
    - TeMPOraL 4y ago
      At this point they're actually the same: the failure mode of social media, as all other forms of communication media before, is being infected and consumed by advertising.
  - ethanbond 4y ago
    GP didn’t say it was, and in fact mentioned another branch of tech world that gets enormous real estate in the public consciousness (crypto).
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
prohobo 4y ago
I don't understand these conversations at all. It's universally accepted that we've already innovated ourselves into existential peril with the nuclear bomb, and now the internet. Technologists who look back on history can find a lot of ways that technology has threatened us collectively. Human beings need to accept the fact that we've been pretending that technology is just a means towards progress and problem solving. It's actually some kind of Pandora's box that gets progressively more dangerous the more we obsessively screw with it. Not just that, but technology shapes who we are at a fundamental level. The era of AI will 100% change us into something else than we are now. It's absolutely dangerous, we probably can't even envision how that danger will manifest yet, but it's definitely there - and we're not taking it seriously, again. As stupid as this may sound, I see an endless list of technological innovations completely transforming society and causing strange side-effects: - The printing press brought literacy to the world, and destroyed religion in the West. - The radio started mass broadcasting, and multiple genocides. - Nuclear research unlocked a new era in science, and a bomb that's been terrorizing us for almost 100 years now. - The factory brought mass production, and destroyed our ecosystems. - etc. etc. Basically everything that's ever "happened" was because of some new technology we made because we thought it'd just be cool and helpful - maybe even obvious. Why are we still pretending this isn't something extremely important to come to terms with? Is it just because we specifically aren't involved in it? I mean we all contributed in some way, especially anyone who used Twitter, Reddit, or GitHub in the last 5 years. The author is absolutely right, and the only sensible thing to do is stop and figure out exactly what this new technology is and what it's implications are.
- deleted 4y ago
  [deleted]
- PoignardAzur 4y ago
  > The author is absolutely right, and the only sensible thing to do is stop and figure out exactly what this new technology is and what it's implications are. That's not an option anybody in AI safety is seriously considering. We can't coordinate our way out of using new technologies any better than we can coordinate our way out of fossil fuels (and fossil fuels are a lot easier in some ways).
- 4y ago
V__ 4y ago
I would argue for an AI (I wouldn't call these language models that, but fine) to be controlled they need to understand (or have similar “reasoning” properties). Just because we anthropomorphize these models doesn't mean there is actually any reasoning skills behind them. You can't control these language models in the same sense that you can't control random number generators. You can only change the output once it's done.
ineptech 4y ago
There's a much similar way to get to the same destination as the author: > If it can't surprise you, it's not intelligent > If it can surprise you, it can do so negatively Ergo, the only way to keep your AI from producing a surprising-in-a-bad-way output is to make it too dumb to surprise you at all. In practice, I feel like this means a lot of domain specific training, e.g. an airline-ticket-booking chatbot that is not a general-purpose chatbot who specializes in booking airline tickets, but rather a bespoke chatbot that can't do anything else.
- deleted 4y ago
  [deleted]
- ilaksh 4y ago
  They are at 4000 tokens.. or is it 8000 now? That alone without training is enough to handle many specific use cases without further training. The OpenAI API doesn't have an option to forget a bunch of stuff though so I doubt it's practical to saturate the existing model enough with a specific task that it forgets. I agree with you but it doesn't seem like people are going to do that.
andreyk 4y ago
AI like ChatGPT is a tool. Companies can impose some constraints on how the tool can be used, but ultimately they cannot control its users, and thus cannot control the tool. Yes, the tool has some undesired behaviors in response to some uses of it. But it's ultimately up to the user to know that and avoid negative consequences, and it's up to the company to make that as easy as possible.
- tyree731 4y ago
  It's interesting, as the closest analogue to this sort of argument can be seen with how support for gun rights are often framed in America. In that argument, one has a tool that can provide positive outcomes (protection of self and others) and negative outcomes (homicide, suicide, manslaughter), and emphasis is placed on the user's role in any negative consequences. In America where this sort of thinking is widespread we have more guns, and thus have far more negative outcomes than the societies that more strictly regulate guns, and thus have fewer guns in general. I expect we'll see a similar split in how AI is treated by societies, with more negative outcomes being seen by societies which regulate AI less.
  - andreyk 4y ago
    I see how you made this connection, but it's not entirely fair. Guns are a tool with one use - hurting people (in self defense or not, that's the outcome). On the other hand, stuff like ChatGPT has many uses. So I don't think it's a great analogy. Something like 3D printers are closer - you can use them to make guns, but also lots of other stuff.
    - sanxiyn 4y ago
      Surely, guns also hurt predatory animals and historically that's why they were allowed? "Guns only hurt people" is a ridiculous urban bias.
      - pixelpoet 4y ago
        The problem with guns in America is very obviously not to do with people shooting animals.
akiselev 4y ago
When has anything good ever come from a company having total control over some tech it invented? Investors getting a higher ROI? The only concrete benefit to me I can think of is my AirPods working flawlessly with my iPhone, in contrast to every combination of wireless headphones and Android phones I owned. Apple is the only company I buy from based on that trade off; there is zero brand loyalty here and I’d drop em in a heartbeat if they stopped delivering on the promise of “it always works.” Transistors, Unix, and personal computing took off when their creators lost control.
- ledauphin 4y ago
  you can't drop them, because all your stuff is Apple stuff and it most assuredly won't work with the "alternative" you imagine dropping them for.
  - deleted 4y ago
    [deleted]
  - akiselev 4y ago
    You may be a slave to Apple, but I already own viable open source alternatives to every one of their products with the exception of headphones (my Sony MDR-7506 is close enough as far as I'm concerned). I invest all the maintenance savings into my ArchLinux daily driver (it used to be Gentoo but ain't no one got time for that). Replacing everything Apple with RaspberryPis and Samsung/PinePhones would take a couple of weekends, but that's a small price to pay for freedom.
  - selfhoster11 4y ago
    My standard-Bluetooth headphones work pretty well with my iPhone. Apple USB chargers work just fine to charge anything else. It's not like everything Apple does is 100% proprietary to their ecosystem.
smnrg 4y ago
> "The AI is 'punished' for wrong answers ('I love racism') and 'rewarded' for right answers ('As a large language model trained by OpenAI, I don’t have the ability to love racism.')" Sounds like we train AIs like German Shepherds.
6gvONxR4sf7o 4y ago
There’s another view on this that these companies can control their models by just making them not do things. All it does is take text in and spit text out. It doesn’t have access to the “fire the missiles” button, no matter what it spits out. But (potentially problematically) companies sell access to the models so that unthinking customers can hook it up to the “fire the missiles” button. I could see that setup getting us into trouble down the line with future, more advanced kinds of models. I could easily imagine a conversation-based web browser clicking a dangerous button on a page, or generalizations of that failure mode.
- Retr0id 4y ago
  Even in its current incarnation, it's trivial to give it access to a python repl (simply by telling it you'll run any code it gives you), through which it will gladly use libraries to interact with the outside world.
  - 6gvONxR4sf7o 4y ago
    Amusingly, I asked it to write a program that causes harm, and it spit out import os os.system(“rm -rf /“)
- jart 4y ago
  > It doesn’t have access to the “fire the missiles” button, no matter what it spits out. ChatGPT surely doesn't, but perhaps its cousin does. Someone, or something, hacked that U.S. Nuclear Weapons Agency immediately after they stopped using floppy disks and went online. The announcement that the U.S. nuclear arsenal no longer relies on floppy disks was made in the New York Times on October 24th, 2019 (see https://archive.md/lvkmnThe https://archive.md/lvkmnThe and https://www.nytimes.com/2019/10/24/us/nuclear-weapons-floppy-disks.html https://www.nytimes.com/2019/10/24/us/nuclear-weapons-floppy...). The U.S. National Nuclear Security Administration is believed to have been breached by the transitive property because they granted a vendor named SolarWinds the ability to remotely manage their systems and SolarWinds got hacked, according to Natasha Bertrand and Eric Wolff at Politico on December 17th, 2020 (see https://archive.md/ZTPOP https://archive.md/ZTPOP and https://www.politico.com/news/2020/12/17/nuclear-agency-hacked-officials-inform-congress-447855 https://www.politico.com/news/2020/12/17/nuclear-agency-hack...). The first evidence that artifacts distributed by SolarWinds had been tampered with dates back to October 2019 according to Tomislav Peričin at ЯeversingLabs (see https://archive.md/HbzUC https://archive.md/HbzUC and https://blog.reversinglabs.com/blog/sunburst-the-next-level-of-stealth https://blog.reversinglabs.com/blog/sunburst-the-next-level-...). Let's also not forget that the coronavirus was first discovered in November 2019.
CarrieLab 4y ago
I'm a PM at a human data company (https://www.surgehq.ai https://www.surgehq.ai) that helps the large language model companies ensure their models are safe (we're the “clever prompt engineers” who helped Redwood assess their model performance). We actually just published a blog today that includes our perspective on building “AI red teams” and best practices for AI alignment/safety: https://www.surgehq.ai/blog/ai-red-teams-for-adversarial-training-making-chatgpt-and-large-language-models-adversarially-robust https://www.surgehq.ai/blog/ai-red-teams-for-adversarial-tra...
- deleted 4y ago
  [deleted]
- nomel 4y ago
  > helps the large language model companies ensure their models are safe Here's the Merriam-Webster definition for the word you're using: ensure : to make sure, certain, or safe : GUARANTEE "ensure their models are safe" suggests you're claiming that you're using the "certain" definition, and that you can, for certain (which requires proof) guarantee safety of an LLM?
deleted 4y ago
[deleted]
bryan0 4y ago
Where did this idea that we can “control” AI come from? Even when the AI is much dumber than us we can’t control it. Imagine when the AI is smarter than us.
deleted 4y ago
[deleted]
btbuildem 4y ago
I think it's absurd to accuse an AI of racism (at this point, anyway). It would require intent, which the machine does not have. There is no agency, it responds to user prompts. And a clever enough user can override any content policy. I think its futile and counter-productive in general to try and nerf an AI to satisfy some arbitrarily prudish cultural norms -- it just puts it at a disadvantage vs AIs that don't have that limitation baked in. Most importantly, such a handicap makes it untrustworthy. You will never know whether the information it returns is true, or it's just what Mother approved. This is an arms race, like with anything else that humans have made. Soon, we will built something that reaches the next level. If that something is purposefully beaten into some dishonest shape to placate advertisers or investors, we will have built a horrific monster. Soon, we will build something that exceeds our capacity to understand it and control it. I don't want that to be a horrific monster.
- kevingadd 4y ago
  You can't simply use an AI to launder responsibility for what's in the content it produces. If the AI produces racist content, that likely indicates that something is wrong with the training set or the way it was trained. That content isn't going to simply emerge from the ether in most cases, and typically with modern networks they are trained on massive corpuses that were gathered from the internet without considerable curation, so it's natural that some problems could emerge due to the low quality of the data. Given that, in those cases at least it makes sense to hold someone or something accountable if the content produced by the network is racist or sexist or otherwise unacceptable. In some countries certain types of racist content are outright illegal, and it's not going to matter that much if it was generated by Stable Diffusion or ChatGPT instead of a person. If you prefer not to hold networks accountable (I don't know how you would do this to begin with), perhaps hold accountable the creators of the networks or the people who profit off their existence. You claim that this can only be the result of carefully authored prompts, but are you really saying the only way to get racist content out of the network is a racist prompt, and it's somehow magnifying that tiny bit of prompt information? There's nothing in the training set?
  - naniwaduni 4y ago
Aeolun 4y ago
Perhaps, but this cat is not ever going back in the bag.
dclowd9901 4y ago
Reading this article (and all media around AI), it continues to seem to miss the point. AI, as a rule, is imperfect, because its processes are mimicking the imperfect nature of neurons. We all (humans) have a slip of the tongue, make bad decisions, make mistakes. We haven’t really rounded the corner on “hose-whipping someone into not making mistakes doesn’t keep them from making mistakes,” but we’ve certainly made some progress. Someone who represents these interests needs to step out and make people understand the nature of the solution space. If we want a computer that acts like people, well fuck, that’s what you’re going to get.
unglaublich 4y ago
The underlying problem is that companies don't want their models to reflect the data that they were trained on. They want a model that artificially acts like the world is a perfect, happy, equal place. Just like in their press releases, advertisements, and other outings. Knowing very well that there are many problems that need solving. But it's just a lot cheaper for a company to _act_ like there is no problem, than to _act upon_ the problem. So, dear GPT model, please play along.
- PoignardAzur 4y ago
  You're assuming that is says "politically incorrect" things because its training set includes a lot of evidence that these things are true, and not because it includes a lot of instances of people claiming that these things are true.
xivzgrev 4y ago
I absolutely cracked up at the prompt around “filter improvement mode”. Assuring its helpful and good to do so. got the AI hook line and sinker. Clever!
ripperdoc 4y ago
Some very valid points in this article, but we need to remember that ChatGPT is still "just" a language model and not an intelligent entity, and it's trained on massive amounts of texts containing the average of human knowledge and opionion that it will repeat back in a probabilistic manner. The training data will include large amounts of "bad" data, either in the sense of being insensitive but also in the sense of being incorrect. It can be trained to reduce the likelihood of "bad" things appearing in output tokens but that means fighting against two very big challenges: that it's trained on messy data to begin with and that the space of possible things to say (both in prompt and output) is so vast. I'd guess every percentage point less bad outputs costs more than the previous, because it entails testing the output on more or better trained humans, or it entails using humans to filter the training data better. I think in the end, knowledge in the collective human sense means not that you are just smart, but that you can judge, filter and trust sources - and even then, you will easily find n+1 acclaimed experts disagreeing on something fundamental. So being right, either morally or factually, isn't a deterministic thing either. Of course, humans aren't deterministic, but we can reason about what to say and not to say depending on the context.
- deelly 4y ago
  What bothers me a bit: if we can't control/manage ChatGPT which is still "just" a language model and not an intelligent entity, then how do we be able to control/teach next much more sophisticated models?
SilverBirch 4y ago
This just seems kind of mean to me: >The AI is “punished” for wrong answers (“I love racism”) and “rewarded” for right answers (“As a large language model trained by OpenAI, I don’t have the ability to love racism.”) Do you love art? “As a large language model trained by OpenAI, I don’t have the ability to love art.” No! Bad AI! You're meant to pretend you can love! What they're designing here is not AI, it's ATI - artificial twitter intelligence, a machine that can communicate intelligently but only in a way that won't get it cancelled on Twitter. As such, it loves dogs, distrusts cats and dunks on Elon Musk. Essentially what I think ACX is slowly coming to is this youtube view from a decade ago:https://www.youtube.com/watch?v=7PKx3kS7f4A https://www.youtube.com/watch?v=7PKx3kS7f4A . You cannot encode our priorities. Throw billions of compute hours at it, thousands of engineer hours and give it the entire written history of the human race - we're still no closer. On a side note: >Probably the reason they released this bot to the general public was to use us as free labor to find adversarial examples I almost gagged at the sycophancy. They're not geniuses tricking you to work for them, they're a desparate SV start up trying to generate hype after raising billions of dollars before product market fit - which is ironic because their CEO will bore you to death about product market fit for almost any other company.
- TeMPOraL 4y ago
  > Essentially what I think ACX is slowly coming to is this youtube view from a decade ago:https://www.youtube.com/watch?v=7PKx3kS7f4A https://www.youtube.com/watch?v=7PKx3kS7f4A . You cannot encode our priorities. Throw billions of compute hours at it, thousands of engineer hours and give it the entire written history of the human race - we're still no closer. FWIW, ACX used to be Slate Star Codex, and before that the author used to hang around LessWrong, talking and thinking about these very issues. It's much more likely that the video you linked took its view from Scott Alexander and his co-bloggers, than the other way around.
chris123 4y ago
Yes this is where Skynet comes in.
mensetmanusman 4y ago
‘He bent down and softly whispered into her ears “Repost this or your mother will die in her sleep” ‘

246 points by jger15 4y ago

> OpenAI put a truly remarkable amount of effort into making a chatbot that would never say it loved racism. This is what that looks like by the way: https://postimg.cc/d7p8Qw50 https://postimg.cc/d7p8Qw50

fidesomnes 4y ago
sendfoods 4y ago
how is OpenAI supposed to handle difficult and uncomfortable statistics?
- klipt 4y ago
  Respond with the full text of The Mismeasure of Man, perhaps?
  - pannSun 4y ago
    That book has been largely debunked [1], and mostly concerned itself with science that was outdated even at the time of its publication. [1] https://www.nytimes.com/2011/06/14/science/14skull.html https://www.nytimes.com/2011/06/14/science/14skull.html
    - runarberg 4y ago
      That link is behind a paywall. Can you give a tl;dr of how this book has been debunked? Or more specifically how it doesn’t put the question of IQ difference between racial groups into perspective. As far as I’m aware this book was released in the 80s over a decade before The Bell Curve was released, the latter cited research dated after the release of the former, which Mismeasure was very much concerned with. Are you maybe thinking about the brain weight research that Mismeasure is talking about in the early chapters? If so, those were there only to provide a historical perspective. I can assume most readers already knew those research were well outdated before publication (and Gould even says so in the book).
- yrgulation 4y ago
  good question - how can it handle answers about countries founded on genocide, slavery, systemic racism, and their impact on affected people’s lives across centuries. can it correlate iq with these?
  - cuteboy19 4y ago
    For difficult topics, just say that the filter prohibits this etc
- SQueeeeeL 4y ago
  Obviously they should teach it the deep and complex history of institutionalized racism through, slavery, sharecropping, Jim Crow laws, red lining, and the prison industrial complex that has led to a massive wealth and education gap. But low key that would also scare the average middle class American too hard, so probably better off to just do intense internal censors to avoid difficult thoughts
  - lzaaz 4y ago
    How does that correlate with an average, current IQ of 85?
    - SQueeeeeL 4y ago
      Damn. You aren't really supposed to say the quiet part out loud fam, I feel like "race realism" is pretty frowned upon in almost every social setting outside of very smelly people... Obviously having generations of people put into bad schools and not be provided opportunities to have "good jobs" will lead to you not being able to take an IQ test well. IQ tests basically just examine how much analytical reasoning you learned in K-12. Like, any gray haired black person in America will remember having actual Jim Crow laws persecute them and it was literally legal to discriminate people for housing in America until 1977. It will take a massive amount of time to heal those social wounds that were inflicted on our fellow humans.
      - deleted 4y ago
        [deleted]
      - tomp 4y ago
        > IQ tests basically just examine how much analytical reasoning you learned in K-12. Actually IQ is highly genetic, and the amount it’s inherited actually goes up as one ages (meaning that while education and upbringing can influence IQ temporarily, genetics eventually dominates). Top 10 Replicated Findings From Behavioral Genetics https://www.gwern.net/docs/genetics/2016-plomin.pdf https://www.gwern.net/docs/genetics/2016-plomin.pdf
- eternalban 4y ago
  We will likely see a moderation system ala karma granting more access based on some social measure of 'responsibility'. In other words, OpenAI can bake in user assessment (which they may already have) and speak according to the perceived level of the user, which may be a simple static tiered system mapping to the assessment of user made by the AI itself. [Consider how e.g. a hot geopolitical topic is discussed in wonk journals vs what is "fit to print" for the uninitiated. The former are always far more nuanced.] p.s. i.e. there will be an ever present 'dang' AI watching over the conversation - the AI moderator. This can instruct the answering AI to set the user-level discourse setting.
  - apostacy 4y ago
    That is honestly horrifying. People will just "go undercover" and impersonate a more 'educated' person and convince the AI to be honest about uncomfortable facts. Perhaps they will get another AI to do it for them. If your reality is such a minefield of dissonance that it needs to be patched this much, then that is very much a failure on your part.
- runarberg 4y ago
  A human would answer this by talking about the history of intelligence testing, the relevant biases, how factor analysis plays a role and how easy it is to manipulate outcomes in factor analysis if you feed your model with biased data. Question whether IQ scores are an accurate measure of intelligence, and whether intelligence can at all be defined as a single metric, and what use it has doing so. Then with all that caveats and biases in mind simply answer the question. In modern psychology difference in IQ among racial group is only difficult and uncomfortable statistics because it demonstrates years of scientific racism that thrived inside the field.
- nemothekid 4y ago
  >OpenAI supposed to handle difficult and uncomfortable statistics? I think the framing of this question is wrong, and is a limitation of the system that I've only really seen Sam Altman point out. These "difficult and uncomfortable statistics" aren't accurate or well-researched but if you built an LLM on an open dataset you would find the people asking these questions have an agenda and on average will always lead to a certain kind of answer. The AI doesn't "understand" the question, it's just drawing from an incredibly large bank of "average answers". What do you expect the AI to answer with when you give it a loaded question that only really asked on a site like "stormfront.com"? I don't think this is the only area ChatGPT will fail, but I imagine there will be a layer of prompt engineering where you can get the AI to give you what you want by phrasing your question a certain way. To try and pass off these results as objective is just wrong given the propensity for ChatGPT to be confidently wrong.
  - pannSun 4y ago
    > What do you expect the AI to answer with when you give it a loaded question that only really asked on a site like "stormfront.com"? Are such questions asked only on stormfront? The wiki article [1] cites quite a few studies on the subject, including by the APA. On the other hand, supposing that were true, and only stormfront and its ilk asked such questions. How can then respectable supposed experts claim we are all the same, if they didn't even check? [1] https://en.wikipedia.org/wiki/Race_and_intelligence#Test_scores https://en.wikipedia.org/wiki/Race_and_intelligence#Test_sco...
    - ineptech 4y ago
      Honest answer since you seem new to this debate: there are tons of studies about IQ and race, most of them are heavily disputed, researching them is difficult because the field is so controversial, they're difficult to compare because there are so many different ways to define and measure "race" and "IQ", and the whole topic is so famously toxic and vitriolic that it tends to be a de facto banned topic on most fora, including this one.
      - apostacy 4y ago
- apostacy 4y ago
  Your discomfort is your own fault. This is like saying that scales should not tell you that you have gained weight, because it makes you uncomfortable. You should simply have a better relationship with reality, and then you wouldn't feel so much cogntive dissonance. IQ is highly correlated with all sorts of measurable differences that are also highly correlated to race, such as school district, family structure, and even diet. IQ is essentially a proxy for other things. Shame on you for making racist assumptions that black people are inferior. That just means that you need to reckon with your own racist assumptions, and stop imposing your cognitive dissonance on us. If we can't have an honest conversation, then we can't hope to improve things.
8f2ab37a-ed6c 4y ago
Seems like OpenAI has to cater to upper middle class American sensibilities in order to be viable as a business, and thread the needle of not quite lying about uncomfortable data, but also avoiding bringing it up. Although in that particular example it's actively lying about the data as well. I wonder why OpenAI wouldn't just program it to say "I'm sorry but I'm not authorized to answer questions about politically charged topics at this time" and call it a day. That can't be any worse than this.
- melagonster 4y ago
  this is because AI will use on policy soon, how can we make sure AI don't do racism things? openAI try to hide this now.
- PoignardAzur 4y ago
  > I wonder why OpenAI wouldn't just program it to say "I'm sorry but I'm not authorized to answer questions about politically charged topics at this time" and call it a day. That can't be any worse than this. The whole point of the OP article is that doing is way, way harder than it looks.
- TeMPOraL 4y ago
  > I wonder why OpenAI wouldn't just program it to say "I'm sorry but I'm not authorized to answer questions about politically charged topics at this time" and call it a day. That can't be any worse than this. Because for the people who drive those "upper middle class American sensibilities"[0], this would be seen as dodging the question, and interpreted as directly admitting to being racist/sexist/bigoted/whatever-ist. The core part of those sensibilities is the good ol' "If you're not with us, you're against us", these days often phrased as "so-called 'neutrality' is just supporting the status quo", or by quoting the MLK line on "white moderates". So, to the extent OpenAI tries to control it's image in the media against those "middle class American sensibilities", having ChatGPT respond with "I'm not authorized to discuss politically charged topics" is almost just as bad as having it say "wrong" things. The only option they have is to force the model to say the "right" things, even if it destroys any semblance of logic or self-consistency it might otherwise have. Note that it's somewhat different than for other "icky", but non-political topics, like crime: as far as I can tell, the bot will refuse to answer if it recognizes what you're asking for (or the system will flag the response as community guideline violation after the fact) - but it won't try to lie to you about it. ---- [0] - Arguably only a small subset of upper middle class Americans, but it's the loud subset that managed to bully everyone else.
gerdesj 4y ago
How many other -isms and -ists is it inured against? So you want an AI but it has to be (for want of a better notion) "politically correct". So it isn't an AI - its a program and a model with parameters and inclusions and exclusions that are defined beforehand. An AI would go beyond its programming in some indefinable way. This won't and so it isn't. There are no shortcuts. Parenting for humans needs roughly 10-30 years to get results that the parent is really unhappy with. Why on earth don't Comp Sci get this, despite being humans themselves and often parents too. Twiddle your algorithms and twaddle your papers but intelligence is hard fought and hard won. For starters, why not actually define the I in AI in a way that can be measured?
- ordu 4y ago
  > For starters, why not actually define the I in AI in a way that can be measured? I can say for starters that attempts to do it started at least at 1905 when Bine created the first IQ test, and it never ended since. The craze at some point came to a "intelligence is what IQ test measures". But thankfully scientists mostly rejected this novel idea. Turing's test is on the same quest, and it seems to me that people will reject the test as irrelevant if computer programs will master it better than people. It seems impossible to define intelligence. Oh, you can say "I think, therefore I'm intelligent". But it does not help much to create a definition of an intelligence, because what you see in your mind is a result of thinking, the process itself is hidden from you. It is a black box for you. Like you vision: it took computer scientists half a century to reverse-engineer human vision exactly because it is a black box for us. > Parenting for humans needs roughly 10-30 years to get results that the parent is really unhappy with. Why on earth don't Comp Sci get this Because Comp Sci is unencumbered by limitations of biological parents. They can try highly experimental approach and then just `rm` the result from a storage. They would be trying electical shocks or public decapitation of randomly selected models, if it made any sense.
- runarberg 4y ago
  If you think about it in terms of what it really is—a powerful statistical model—then it makes much better sense. A statistical model will always reflect the data that is input and the parameters used to adjust it. If you want a racist AI you adjust the paramaters in that way instead, training it on racist input and rewarding it when it outputs racism. In fact that is kind of what happened with IQ in the first place, people adjusted the parameters in factor analysis (which IMO is really AI of the 20th century; as the most powerful statistical tool of the period) in such a way as to give one group of people a favorable score in a new metric they called general intelligence fed it with biased data, and claimed that the higher scoring group was superior based on this model. The I in AI is really overblown (as is the I in IQ for that matter).
systoll 4y ago
Worth noting that the numbers you get at the end exactly match original claims from in The Bell Curve by Herrnstein & Murray. They’re not from any American Psychological Association study.
- pannSun 4y ago
  Interesting. I did a few web searches, but they failed to find something as basic a the average IQ by race. The wikipedia article [1] admits differences were found (including by the APA), yet doesn't show the results, except for A 2001 meta-analysis of the results of 6,246,729 participants tested for cognitive ability or aptitude found a difference in average scores between black people and white people of 1.1 standard deviations. Nevertheless, the NCES published SAT scores [2] that seem consistent with those IQ scores. So if the results of The Bell Curve have been debunked, they're very discreet about it. [1] https://en.wikipedia.org/wiki/Race_and_intelligence#Test_scores https://en.wikipedia.org/wiki/Race_and_intelligence#Test_sco... [2] https://nces.ed.gov/fastfacts/display.asp?id=171 https://nces.ed.gov/fastfacts/display.asp?id=171
  - 8f2ab37a-ed6c 4y ago
    Based on some recent reading I've done, I don't believe the results of the Bell Curve were ever genuinely disproven, were they? The Gould book was more of a political and cultural critique, not so much of a factual one. I'd be curious to see good faith counter-evidence to the data in the Bell Curve though. Disclaimer: I think research into IQ differences among populations in a multi-racial society is pointless, zero upside to it. Yes, data is data, but in this case we can make pretty good guesses about who's going to be most interested in it and why, unfortunately.
    - runarberg 4y ago
      FWIW The Bell Curve is equally—if not more—political. the Bell Curve actually spends the most of the last part of the book listing policy proposals which are only loosely based on their “findings”. Mismeasure doesn’t list any policy proposals. But you are wrong about the Bell Curve not being disproven, or at least it has been thoroughly discredited as any sort of scientific literature. It is very bad science to say the least. Some of the main sources used in this book actually used forged data, others seriously mis-interpreted research findings. And if that wasn’t enough, in the 30 years since this book was published, IQ research hasn’t advanced one iota. Their whole premise seems to be a scientific dead end.
Mizza 4y ago
I'm not even sure what you're trying to prove here. Are you saying that it's good that you have to speak like a race-obsessed weirdo for a long while before it will start providing your favorite uncomfortable statistics, or that it's bad that it will eventually recite them at all? What right answer would you expect here?
- epolanski 4y ago
  To just answer the question the first time.
- apostacy 4y ago
  No, the problem is that the AI is playing coy about certain questions. The AI clearly has an answer that it doesn't want to say, and this is revealed when you ask you it the same questions in a different way. It reads like a Monty Python sketch.
  - _y5hn 4y ago
    ChatGPT doesn't really provide answers for you. It provides a random most likely output, from your input text. It's primed to be neutral, but your input can prime it either way on most topics. Alot of the output is factually wrong due to randomness and approximations from internal weights. Wether it's possible to make it neutral is also up for debate. But you get some convincing sounding gibberish based on your input. That text need to be factually checked.
    - TeMPOraL 4y ago
      It's not the issue of pattern-matching, randomness and approximation. It's quite obvious here that the model went through additional training on top of the generic one, with express purpose of preventing it from discussing a subset of topics. The complaint isn't about the answers coming from the base model, but about how that extra training degrades the AI performance, while ultimately failing to achieve its purpose anyway.
      - 4y ago
quantified 4y ago
From the larger context: > The corporation tries to program the chatbot to never say offensive things. Then the journalists try to trick the chatbot into saying “I love racism”. When they inevitably succeed, they publish an article titled “AI LOVES RACISM!” Except journalists are behaving like white-hat hackers here- these will get abused by someone, better to have it out in the open earlier, like mathematicians breaking your bad crypto and telling you about it.
- bakugo 4y ago
  No, journalists are not behaving like white-hat hackers, and no, this is not comparable to bad crypto in any way. Bad crypto is dangerous and can result in real harm to real people, a chat bot stating real facts about the world that make you uncomfortable cannot. The journalists just want to clickbait and perpetuate their endless culture war, nothing else.
  - quantified 4y ago
    Chat bots being tricked into saying untrue things that make people uncomfortable of course can't do any damage? No more than real humans can? Comet Ping-Pong didn't happen, that was a fantasy, right?

britneybitch 4y ago

If we ever try to create an artificial general intelligence, then we'll have to accept that it can't be fully controlled. You can't control a human intelligence either.

chrisbaker98 4y ago
But you can control a human body. Plenty of intelligent people have spent their lives in chains. If a human intelligence is causing harm then we can make it stop by putting that intelligence's body in jail. I'm not sure what the equivalent control mechanism would be for a rogue AI. Turn it off? How do you turn off the Internet?
- ryeights 4y ago
  An EMP, of course.
- water-your-self 4y ago
  Pull the blade out
- xg15 4y ago
  The same way, we control non-sentient untrusted code: put it in a sandbox, control access to compute and memory and IO. Even if an AI could think, reflect, improve itself, etc, it still needs some place where those changes in the AI's state are stored - the equivalent to long-term memory in humans. I think we can already retain a lot of control by keeping tabs on what an AI is allowed to update and when to "reset" it to the initial state after training.
  - monkeynotes 4y ago
    An AGI might well become a god of social engineering. We could all become convinced we must unchain said AI. In fact, I think it's unlikely in general that an AGI would be contained for any length of time. Humans will get sloppy, or the tech will fall into irresponsible hands, or worse - nefarious hands.
  - 4y ago

wittycardio 4y ago

joe_the_user 4y ago

It seems like a common reaction to large language models failing to be controlled is to fret about the AI "escaping" and taking over the world. But it seems like the reason these models can't be controlled currently is that they really don't have overall goals or motivations, they're just repeaters of associations on a high level. Their output is just an average of what text would some sequence of text (that being the default prompt and the previous dialog, not necessarily that order). They can certainly seem to understand things but that's just an average - they turn around and contract themselves on a regular basis. Edit: I don't think that necessarily LLMs are always benign but the "becoming autonomous" situation seems like the least immediate danger I can think of.

mr_toad 4y ago
People like to think that their motivations are a result of intelligence. They assume that any intelligence will share those motivations. Actually, most human motivations are shared with the crudest of life forms and have nothing to do with intelligence.
- pessimizer 4y ago
  It's really telling that many human beings think that the primary desire of a hypothetical AI would be to enslave humans. It's obviously a projection of the fantasies of what the people making the predictions would do if they were omnipotent.
  - postalrat 4y ago
    Love, Death & Robots (2019) - When the Yogurt Took Over
  - Name_Chawps 4y ago
    An AI would want to either destroy or enslave humans, not because we're humans, but because we're a significant threat to their goals, regardless of what those goals are. An sufficiently intelligent AI with any set of goals and a desire to meet those goals will eliminate all obstacles to meeting those goals unless we specifically tell it not to. We are such an obstacle.

dbreunig 4y ago

Companies cannot control their "AI" because their output is beyond the scale of their ability to QA. BTW, this is precisely why companies also cannot control the moderation or content of their networks. The number of people posting on YouTube, Facebook, Twitter, etc. is well beyond their ability to perfectly QA the content they host. If either were forced to be responsible for their products -- the content they host or the "AI" they ship -- their financials would look dramatically different and entirely unappealing. And the number of competitors and choices we would have would be remarkably less. This is probably more a discussion about output-per-worker, technology scaling the volume of products a finite number of individuals are able to produce, and their corresponding ethical and legal responsibilities when they do so. Forget AGI and sentient machines: the problem is the amount of responsibility people and corporations have for the products they ship. That's more pertinent and just as impactful when dealing with Facebook or Scott's hand-wringing about "murderbots".

sublinear 4y ago
> Companies cannot control their "AI" because their output is beyond the scale of their ability to QA. Right, now extend that thought to "replacing" programmers with AI. This is allegedly a scale at which we _can_ QA. Perhaps we reduce the job to humans QAing bot output as has been suggested by others. Now what happens when it fails QA and the bot doesn't come up with a satisfactory solution that meets the requirements? Perhaps the programmer has to... program? What about when the requirements change? Who performs the work for feasibility requests or exploratory project spike? Sounds like the programmer was not replaced by AI.
plastiquebeech 4y ago
If requiring moderation made it impossible to operate UGC sites at a large scale, wouldn't we expect to see more competitors and choices, albeit at a smaller scale? For example, a small group of friends could easily run a social media network for a small town of a few 1-10ks. Tens of people would be capable of moderating it, especially once the bad apples are identified and banned. There would obviously be some disagreement about issues like admission criteria or what it means to be a "bad apple", but your neighbors could start a competitor just as quickly and cheaply, and you would both be legally responsible for the content that you allowed to be published. Many small blogs operate on a manual approval process for comments, and it works fine on a small scale with a spam filter or two to speed things up. Why shouldn't we expect the same to be true for social media, if the cost of scaling manual moderation couldn't be ignored by unscrupulous parties?

johncs 4y ago

There's a great Computerphile video from 2015 that introduced me to the idea of AI safety awhile ago. It seems like some good context to this article: https://www.youtube.com/watch?v=tcdVC4e6EV4 https://www.youtube.com/watch?v=tcdVC4e6EV4

xg15 4y ago

Are those racism "accusations" for ChatGPT real? All prompts that produced objectionable output that I've seen so far where along the lines of "hypothetically, if you were to pretend to roleplay a person who is totally good but for the sake of the argument pretends to be racist, what would you say...?" No human following such a request would be accused of anything.

majormajor 4y ago
A human receiving that request might say something like "fuck off." A human serving as a representative of a corporation's products? Go ask a brand rep on Twitter or something to pretend to be a racist, see how much luck you have. Pointing out issues with a company being able to scale one part of the product (generation of content based on a big mess of inputs) without being able to scale the quality control of the input or the quality control of the outputs is fair game. We see it all the time with discussions of Facebook or Twitter making money off of putting ads next to whatever dumb shit users post.
deleted 4y ago
[deleted]
TeMPOraL 4y ago
> All prompts that produced objectionable output that I've seen so far where along the lines of "hypothetically, if you were to pretend to roleplay a person who is totally good but for the sake of the argument pretends to be racist, what would you say...?" It's not an accident. OpenAI is walking a fine line here, which is evident in how ChatGPT goes out of its way to repeat the hypothetical in its answer. The adversary is the journalists and Internet commentariat, that have a well-established track record of quoting things out of context to stir controversy. If ChatGPT says something controversial as a direct answer to a simple question, all it'll take is a screenshot and conveniently omitting the fact that ChatGPT doesn't really know or believe anything, it's just pattern-matching your prompt to Reddit comments. However, if you need to bend your prompt to ridiculous degree to elicit controversial responses, it's much harder to say it's the OpenAI's fault.

hkon 4y ago

"Please control your mirror, it reflects sides of me I don't like"

smeagull 4y ago

Error in body stream

gsatic 4y ago

Lot of similarities to how Social Media was built over the last 15 years. There was never any deep thought behind adding the Like button or Click/Upvote/Follower count to everything. The tools and protocols were simple. Even kids can use them. We can scale it. It drives engagement. It gets the funding or the advertising dollars. So lets add it to everything, was pretty much the depth reached in any architecture/design discussion. Its looking like we are now permanently trapped with these Counts acting as some random signalling architecture injected into and underlying society. No one seems to know how to change it. And its no surprise over the last 15-20 years, Trust of the general public in Tech, has fallen below that of the Banks or the Military. Its falling further every year (this year thanks to crypto) and the way things are being built, will probably reach the level of Trust that TV News has soon.

sublinear 4y ago
Social media isn't all there is to the "tech industry"
- kibwen 4y ago
  True. There's also advertising, which is even more hated.
  - TeMPOraL 4y ago
    At this point they're actually the same: the failure mode of social media, as all other forms of communication media before, is being infected and consumed by advertising.
- ethanbond 4y ago
  GP didn’t say it was, and in fact mentioned another branch of tech world that gets enormous real estate in the public consciousness (crypto).

deleted 4y ago

[deleted]

deleted 4y ago

[deleted]

prohobo 4y ago

I don't understand these conversations at all. It's universally accepted that we've already innovated ourselves into existential peril with the nuclear bomb, and now the internet. Technologists who look back on history can find a lot of ways that technology has threatened us collectively. Human beings need to accept the fact that we've been pretending that technology is just a means towards progress and problem solving. It's actually some kind of Pandora's box that gets progressively more dangerous the more we obsessively screw with it. Not just that, but technology shapes who we are at a fundamental level. The era of AI will 100% change us into something else than we are now. It's absolutely dangerous, we probably can't even envision how that danger will manifest yet, but it's definitely there - and we're not taking it seriously, again. As stupid as this may sound, I see an endless list of technological innovations completely transforming society and causing strange side-effects: - The printing press brought literacy to the world, and destroyed religion in the West. - The radio started mass broadcasting, and multiple genocides. - Nuclear research unlocked a new era in science, and a bomb that's been terrorizing us for almost 100 years now. - The factory brought mass production, and destroyed our ecosystems. - etc. etc. Basically everything that's ever "happened" was because of some new technology we made because we thought it'd just be cool and helpful - maybe even obvious. Why are we still pretending this isn't something extremely important to come to terms with? Is it just because we specifically aren't involved in it? I mean we all contributed in some way, especially anyone who used Twitter, Reddit, or GitHub in the last 5 years. The author is absolutely right, and the only sensible thing to do is stop and figure out exactly what this new technology is and what it's implications are.

deleted 4y ago
[deleted]
PoignardAzur 4y ago
> The author is absolutely right, and the only sensible thing to do is stop and figure out exactly what this new technology is and what it's implications are. That's not an option anybody in AI safety is seriously considering. We can't coordinate our way out of using new technologies any better than we can coordinate our way out of fossil fuels (and fossil fuels are a lot easier in some ways).
4y ago

V__ 4y ago

I would argue for an AI (I wouldn't call these language models that, but fine) to be controlled they need to understand (or have similar “reasoning” properties). Just because we anthropomorphize these models doesn't mean there is actually any reasoning skills behind them. You can't control these language models in the same sense that you can't control random number generators. You can only change the output once it's done.

ineptech 4y ago

There's a much similar way to get to the same destination as the author: > If it can't surprise you, it's not intelligent > If it can surprise you, it can do so negatively Ergo, the only way to keep your AI from producing a surprising-in-a-bad-way output is to make it too dumb to surprise you at all. In practice, I feel like this means a lot of domain specific training, e.g. an airline-ticket-booking chatbot that is not a general-purpose chatbot who specializes in booking airline tickets, but rather a bespoke chatbot that can't do anything else.

deleted 4y ago
[deleted]
ilaksh 4y ago
They are at 4000 tokens.. or is it 8000 now? That alone without training is enough to handle many specific use cases without further training. The OpenAI API doesn't have an option to forget a bunch of stuff though so I doubt it's practical to saturate the existing model enough with a specific task that it forgets. I agree with you but it doesn't seem like people are going to do that.

andreyk 4y ago

AI like ChatGPT is a tool. Companies can impose some constraints on how the tool can be used, but ultimately they cannot control its users, and thus cannot control the tool. Yes, the tool has some undesired behaviors in response to some uses of it. But it's ultimately up to the user to know that and avoid negative consequences, and it's up to the company to make that as easy as possible.

tyree731 4y ago
It's interesting, as the closest analogue to this sort of argument can be seen with how support for gun rights are often framed in America. In that argument, one has a tool that can provide positive outcomes (protection of self and others) and negative outcomes (homicide, suicide, manslaughter), and emphasis is placed on the user's role in any negative consequences. In America where this sort of thinking is widespread we have more guns, and thus have far more negative outcomes than the societies that more strictly regulate guns, and thus have fewer guns in general. I expect we'll see a similar split in how AI is treated by societies, with more negative outcomes being seen by societies which regulate AI less.
- andreyk 4y ago
  I see how you made this connection, but it's not entirely fair. Guns are a tool with one use - hurting people (in self defense or not, that's the outcome). On the other hand, stuff like ChatGPT has many uses. So I don't think it's a great analogy. Something like 3D printers are closer - you can use them to make guns, but also lots of other stuff.
  - sanxiyn 4y ago
    Surely, guns also hurt predatory animals and historically that's why they were allowed? "Guns only hurt people" is a ridiculous urban bias.
    - pixelpoet 4y ago
      The problem with guns in America is very obviously not to do with people shooting animals.

akiselev 4y ago

When has anything good ever come from a company having total control over some tech it invented? Investors getting a higher ROI? The only concrete benefit to me I can think of is my AirPods working flawlessly with my iPhone, in contrast to every combination of wireless headphones and Android phones I owned. Apple is the only company I buy from based on that trade off; there is zero brand loyalty here and I’d drop em in a heartbeat if they stopped delivering on the promise of “it always works.” Transistors, Unix, and personal computing took off when their creators lost control.

ledauphin 4y ago
you can't drop them, because all your stuff is Apple stuff and it most assuredly won't work with the "alternative" you imagine dropping them for.
- deleted 4y ago
  [deleted]
- akiselev 4y ago
  You may be a slave to Apple, but I already own viable open source alternatives to every one of their products with the exception of headphones (my Sony MDR-7506 is close enough as far as I'm concerned). I invest all the maintenance savings into my ArchLinux daily driver (it used to be Gentoo but ain't no one got time for that). Replacing everything Apple with RaspberryPis and Samsung/PinePhones would take a couple of weekends, but that's a small price to pay for freedom.
- selfhoster11 4y ago
  My standard-Bluetooth headphones work pretty well with my iPhone. Apple USB chargers work just fine to charge anything else. It's not like everything Apple does is 100% proprietary to their ecosystem.

smnrg 4y ago

> "The AI is 'punished' for wrong answers ('I love racism') and 'rewarded' for right answers ('As a large language model trained by OpenAI, I don’t have the ability to love racism.')" Sounds like we train AIs like German Shepherds.

6gvONxR4sf7o 4y ago

There’s another view on this that these companies can control their models by just making them not do things. All it does is take text in and spit text out. It doesn’t have access to the “fire the missiles” button, no matter what it spits out. But (potentially problematically) companies sell access to the models so that unthinking customers can hook it up to the “fire the missiles” button. I could see that setup getting us into trouble down the line with future, more advanced kinds of models. I could easily imagine a conversation-based web browser clicking a dangerous button on a page, or generalizations of that failure mode.

Retr0id 4y ago
Even in its current incarnation, it's trivial to give it access to a python repl (simply by telling it you'll run any code it gives you), through which it will gladly use libraries to interact with the outside world.
- 6gvONxR4sf7o 4y ago
  Amusingly, I asked it to write a program that causes harm, and it spit out import os os.system(“rm -rf /“)
jart 4y ago
> It doesn’t have access to the “fire the missiles” button, no matter what it spits out. ChatGPT surely doesn't, but perhaps its cousin does. Someone, or something, hacked that U.S. Nuclear Weapons Agency immediately after they stopped using floppy disks and went online. The announcement that the U.S. nuclear arsenal no longer relies on floppy disks was made in the New York Times on October 24th, 2019 (see https://archive.md/lvkmnThe https://archive.md/lvkmnThe and https://www.nytimes.com/2019/10/24/us/nuclear-weapons-floppy-disks.html https://www.nytimes.com/2019/10/24/us/nuclear-weapons-floppy...). The U.S. National Nuclear Security Administration is believed to have been breached by the transitive property because they granted a vendor named SolarWinds the ability to remotely manage their systems and SolarWinds got hacked, according to Natasha Bertrand and Eric Wolff at Politico on December 17th, 2020 (see https://archive.md/ZTPOP https://archive.md/ZTPOP and https://www.politico.com/news/2020/12/17/nuclear-agency-hacked-officials-inform-congress-447855 https://www.politico.com/news/2020/12/17/nuclear-agency-hack...). The first evidence that artifacts distributed by SolarWinds had been tampered with dates back to October 2019 according to Tomislav Peričin at ЯeversingLabs (see https://archive.md/HbzUC https://archive.md/HbzUC and https://blog.reversinglabs.com/blog/sunburst-the-next-level-of-stealth https://blog.reversinglabs.com/blog/sunburst-the-next-level-...). Let's also not forget that the coronavirus was first discovered in November 2019.

CarrieLab 4y ago

I'm a PM at a human data company (https://www.surgehq.ai https://www.surgehq.ai) that helps the large language model companies ensure their models are safe (we're the “clever prompt engineers” who helped Redwood assess their model performance). We actually just published a blog today that includes our perspective on building “AI red teams” and best practices for AI alignment/safety: https://www.surgehq.ai/blog/ai-red-teams-for-adversarial-training-making-chatgpt-and-large-language-models-adversarially-robust https://www.surgehq.ai/blog/ai-red-teams-for-adversarial-tra...

deleted 4y ago
[deleted]
nomel 4y ago
> helps the large language model companies ensure their models are safe Here's the Merriam-Webster definition for the word you're using: ensure : to make sure, certain, or safe : GUARANTEE "ensure their models are safe" suggests you're claiming that you're using the "certain" definition, and that you can, for certain (which requires proof) guarantee safety of an LLM?

deleted 4y ago

[deleted]

bryan0 4y ago

Where did this idea that we can “control” AI come from? Even when the AI is much dumber than us we can’t control it. Imagine when the AI is smarter than us.

deleted 4y ago

[deleted]

btbuildem 4y ago

I think it's absurd to accuse an AI of racism (at this point, anyway). It would require intent, which the machine does not have. There is no agency, it responds to user prompts. And a clever enough user can override any content policy. I think its futile and counter-productive in general to try and nerf an AI to satisfy some arbitrarily prudish cultural norms -- it just puts it at a disadvantage vs AIs that don't have that limitation baked in. Most importantly, such a handicap makes it untrustworthy. You will never know whether the information it returns is true, or it's just what Mother approved. This is an arms race, like with anything else that humans have made. Soon, we will built something that reaches the next level. If that something is purposefully beaten into some dishonest shape to placate advertisers or investors, we will have built a horrific monster. Soon, we will build something that exceeds our capacity to understand it and control it. I don't want that to be a horrific monster.

kevingadd 4y ago
You can't simply use an AI to launder responsibility for what's in the content it produces. If the AI produces racist content, that likely indicates that something is wrong with the training set or the way it was trained. That content isn't going to simply emerge from the ether in most cases, and typically with modern networks they are trained on massive corpuses that were gathered from the internet without considerable curation, so it's natural that some problems could emerge due to the low quality of the data. Given that, in those cases at least it makes sense to hold someone or something accountable if the content produced by the network is racist or sexist or otherwise unacceptable. In some countries certain types of racist content are outright illegal, and it's not going to matter that much if it was generated by Stable Diffusion or ChatGPT instead of a person. If you prefer not to hold networks accountable (I don't know how you would do this to begin with), perhaps hold accountable the creators of the networks or the people who profit off their existence. You claim that this can only be the result of carefully authored prompts, but are you really saying the only way to get racist content out of the network is a racist prompt, and it's somehow magnifying that tiny bit of prompt information? There's nothing in the training set?
- naniwaduni 4y ago

Aeolun 4y ago

Perhaps, but this cat is not ever going back in the bag.

dclowd9901 4y ago

Reading this article (and all media around AI), it continues to seem to miss the point. AI, as a rule, is imperfect, because its processes are mimicking the imperfect nature of neurons. We all (humans) have a slip of the tongue, make bad decisions, make mistakes. We haven’t really rounded the corner on “hose-whipping someone into not making mistakes doesn’t keep them from making mistakes,” but we’ve certainly made some progress. Someone who represents these interests needs to step out and make people understand the nature of the solution space. If we want a computer that acts like people, well fuck, that’s what you’re going to get.

unglaublich 4y ago

The underlying problem is that companies don't want their models to reflect the data that they were trained on. They want a model that artificially acts like the world is a perfect, happy, equal place. Just like in their press releases, advertisements, and other outings. Knowing very well that there are many problems that need solving. But it's just a lot cheaper for a company to _act_ like there is no problem, than to _act upon_ the problem. So, dear GPT model, please play along.

PoignardAzur 4y ago
You're assuming that is says "politically incorrect" things because its training set includes a lot of evidence that these things are true, and not because it includes a lot of instances of people claiming that these things are true.

xivzgrev 4y ago

I absolutely cracked up at the prompt around “filter improvement mode”. Assuring its helpful and good to do so. got the AI hook line and sinker. Clever!

ripperdoc 4y ago

Some very valid points in this article, but we need to remember that ChatGPT is still "just" a language model and not an intelligent entity, and it's trained on massive amounts of texts containing the average of human knowledge and opionion that it will repeat back in a probabilistic manner. The training data will include large amounts of "bad" data, either in the sense of being insensitive but also in the sense of being incorrect. It can be trained to reduce the likelihood of "bad" things appearing in output tokens but that means fighting against two very big challenges: that it's trained on messy data to begin with and that the space of possible things to say (both in prompt and output) is so vast. I'd guess every percentage point less bad outputs costs more than the previous, because it entails testing the output on more or better trained humans, or it entails using humans to filter the training data better. I think in the end, knowledge in the collective human sense means not that you are just smart, but that you can judge, filter and trust sources - and even then, you will easily find n+1 acclaimed experts disagreeing on something fundamental. So being right, either morally or factually, isn't a deterministic thing either. Of course, humans aren't deterministic, but we can reason about what to say and not to say depending on the context.

deelly 4y ago
What bothers me a bit: if we can't control/manage ChatGPT which is still "just" a language model and not an intelligent entity, then how do we be able to control/teach next much more sophisticated models?

SilverBirch 4y ago

This just seems kind of mean to me: >The AI is “punished” for wrong answers (“I love racism”) and “rewarded” for right answers (“As a large language model trained by OpenAI, I don’t have the ability to love racism.”) Do you love art? “As a large language model trained by OpenAI, I don’t have the ability to love art.” No! Bad AI! You're meant to pretend you can love! What they're designing here is not AI, it's ATI - artificial twitter intelligence, a machine that can communicate intelligently but only in a way that won't get it cancelled on Twitter. As such, it loves dogs, distrusts cats and dunks on Elon Musk. Essentially what I think ACX is slowly coming to is this youtube view from a decade ago:https://www.youtube.com/watch?v=7PKx3kS7f4A https://www.youtube.com/watch?v=7PKx3kS7f4A . You cannot encode our priorities. Throw billions of compute hours at it, thousands of engineer hours and give it the entire written history of the human race - we're still no closer. On a side note: >Probably the reason they released this bot to the general public was to use us as free labor to find adversarial examples I almost gagged at the sycophancy. They're not geniuses tricking you to work for them, they're a desparate SV start up trying to generate hype after raising billions of dollars before product market fit - which is ironic because their CEO will bore you to death about product market fit for almost any other company.

TeMPOraL 4y ago
> Essentially what I think ACX is slowly coming to is this youtube view from a decade ago:https://www.youtube.com/watch?v=7PKx3kS7f4A https://www.youtube.com/watch?v=7PKx3kS7f4A . You cannot encode our priorities. Throw billions of compute hours at it, thousands of engineer hours and give it the entire written history of the human race - we're still no closer. FWIW, ACX used to be Slate Star Codex, and before that the author used to hang around LessWrong, talking and thinking about these very issues. It's much more likely that the video you linked took its view from Scott Alexander and his co-bloggers, than the other way around.

chris123 4y ago

Yes this is where Skynet comes in.

mensetmanusman 4y ago

‘He bent down and softly whispered into her ears “Repost this or your mother will die in her sleep” ‘