13 ms·
Microsoft's paper on OpenAI's GPT-4 had hidden information
- mudlus 4y ago[flagged]
- goodgoblin 4y agoThe spontaneous toxic content stuff is a little alarming, but probably in the future there will be gpt-Ns that have their core training data filtered so all the insane reddit comments aren't part of their makeup.
- GaggiX 4y agoIf you filter the dataset to remove anything that might be considered toxic, the model will have much more difficulty understanding humanity as a whole; the solution is alignment, not censorship.
- ben_w 4y agoWhile I share your belief, I am unaware of any proof that such censorship would actually fail as an alignment method. Nor even how much impact it would have on capabilities. Of course, to actually function this would also need to e.g. filter out soap operas, murder mysteries, and action films, lest it overestimate the frequency and underestimate the impact of homicide.
- GaggiX 4y agoIf "toxic content" is filtered out, it will be out of the model's distribution if it encounters it during inference, this is clearly not our goal and interest as AI designers, so it would not work as an alignment method; our interest is that the model can recognize toxic content but not produce it, OpenAI to address this issue is using RLHF, changing the model's objective from predicting the next token based on the distribution of the training dataset to maximizing the sparse reward of a human annotator.
- slowmovintarget 4y agoMe: "grblf is bad, don't write about it or things related to it." You: "What is grblf?" As parents, my wife and I go through this on a daily basis. We have to explain what the behavior is, and why it is unacceptable or harmful. The reason LLM models have such trouble with this is because LLMs have no theory of mind. They cannot project that text they generate will be read, conceptualized, and understood by a living being in a way that will harm them, or cause them to harm others. Either way, censorship is definitely not the answer.
- stevenhuang 4y agoWelll.... Theory of Mind May Have Spontaneously Emerged in Large Language Models - https://arxiv.org/abs/2302.02083 https://arxiv.org/abs/2302.02083 Previously discussed - https://news.ycombinator.com/item?id=34730365 https://news.ycombinator.com/item?id=34730365
- slowmovintarget 4y agoThank you for sharing... That's a really interesting paper.
- astrange 4y agoChatbot-tuned models must have a "theory of mind", because they're able to tell which parts of the chat history are theirs and which are yours. (This doesn't use tokens. You can have a conversation in OpenAI Playground with text-davinci-003 and provide all the text yourself.)
- ben_w 4y agoThat demonstrates possibly rather than necessity of alignment via having a definition. Behaviours can be reinforced or dissuaded in non-verbal subjects, such as wild animals. There's also the size of the possible behaviour space to consider: a discussion seldom has exactly two possible outcomes, the good one and the bad one, because even if you want yes-or-no answers it's still valid to respond "I don't know". For an example of the former, I'm not sure how good the language model in DALL•E 2 is, but asking it for "Umfana nentombazane badlala ngebhola epaki elihle elinelanga elinesihlahla, umthwebuli wezithombe, uchwepheshe, 4k" didn't produce anything close to the English that I asked Google Translate to turn into Zulu: https://github.com/BenWheatley/Studies-of-AI/blob/main/DALL•E%202:%20panlingual%3F/research.md https://github.com/BenWheatley/Studies-of-AI/blob/main/DALL•... (And for the latter, that might be why it did what it did with the Somali).
- cratermoon 4y ago"The Colossal Clean Crawled Corpus, used to train a trillion parameter LM in [43], is cleaned, inter alia, by discarding any page containing one of a list of about 400 “Dirty, Naughty, Obscene or Otherwise Bad Words”. This list is overwhelmingly words related to sex, with a handful of racial slurs and words related to white supremacy (e.g. swastika, white power) included. While possibly effective at removing documents containing pornography (and the associated problematic stereotypes encoded in the language of such sites) and certain kinds of hate speech, this approach will also undoubtedly attenuate, by suppressing such words as twink, the influence of online spaces built by and for LGBTQ people. If we filter out the discourse of marginalized populations, we fail to provide training data that reclaims slurs and otherwise describes marginalized identities in a positive light" from "On the Dangers of Stochastic Parrots: Can Language Models Be Too Big? " https://dl.acm.org/doi/10.1145/3442188.3445922 https://dl.acm.org/doi/10.1145/3442188.3445922 That list of words is https://github.com/LDNOOBW/List-of-Dirty-Naughty-Obscene-and-Otherwise-Bad-Words/blob/master/en https://github.com/LDNOOBW/List-of-Dirty-Naughty-Obscene-and...
- rhdunn 4y agoThat will also remove: 1. medical pages/docs using the medical terms anus, rectum, nipple, and semen (note that other medical terms are not on that list). 2. pages/docs using "sex" to refer to males and females. 3. pages/docs talking about rapeseed oil or the plant it comes from (https://en.wikipedia.org/wiki/Rapeseed_oil https://en.wikipedia.org/wiki/Rapeseed_oil). The big problem with these lists is that they exclude valid contexts, and only include a small set of possible terms, so the model would get a distorted view of the world (like it learning that people can have penises, vaginas, breasts, but not nipples or anuses, and breasts cannot be big [1]). It would be better to train the models on these, teach it the contexts, and teach it where various usages are archaic, out dated, old fashioned, etc. [1] but this is excluding the cases where "as big as", etc. are used to join the noun from the adjective, so just excluding the term "big breasts" is ineffective.
- cratermoon 4y agoThis is what's known as the Scunthorpe problem. https://en.wikipedia.org/wiki/Scunthorpe_problem https://en.wikipedia.org/wiki/Scunthorpe_problem
- MikeTheGreat 4y agoGenuine question: What do you mean by 'alignment'? Is this a technique for training AIs, or a philosophy about how to approach this, etc? I've never heard the term before and would love any pointers (including enough keywords to Google for it :) )
- GaggiX 4y agohttps://en.m.wikipedia.org/wiki/AI_alignment https://en.m.wikipedia.org/wiki/AI_alignment "In the field of artificial intelligence (AI), AI alignment research aims to steer AI systems towards their designers’ intended goals and interests." I also suggest the YouTube channel: "Robert Miles"
- thrown123098 4y ago[flagged]
- precompute 4y ago>the solution is alignment, not censorship haha, that's very naive. There's already heaps (veritable mountains, even) of information that isn't given to the public on the public-facing instances of ChatGPT, because some info is deemed too incendiary. Filtering out "unwanted" sources of information is already a goal of information labelling, on which these entire LLMs exist. If you were to really make a LLM out of what people really thought and put on the internet instead of the current practice of castration, you wouldn't have techbros wondering about jobs, you'd have a veritable revolution on your hands.
- GenerocUsername 4y agoOpenAI has incentive to 'accidentally' allow toxic content through, so when they make the case that all models should be censored and make it safe, they can pull up the ladder behind them.
- xnx 4y agoI was prepared to be very amused if this was the result of Windows screenshot tool acropalypse.
- taneq 4y agoOh wow, I hadn’t heard about this one! Content for anyone else in the same boat: https://www.techtimes.com/articles/289318/20230321/windows-11-snipping-tool-privacy-flaw-apocalypse-results-in-cropped-images-exposed-through-bug.htm https://www.techtimes.com/articles/289318/20230321/windows-1...
- layer8 4y agoHN discussions: https://news.ycombinator.com/item?id=35207787 https://news.ycombinator.com/item?id=35207787 https://news.ycombinator.com/item?id=35208721 https://news.ycombinator.com/item?id=35208721 https://news.ycombinator.com/item?id=35249971 https://news.ycombinator.com/item?id=35249971 https://news.ycombinator.com/item?id=35273305 https://news.ycombinator.com/item?id=35273305
- withinboredom 4y agoSigh. What an idiot (no offense). Why tell the world you got this from the comments? Now every damn researcher is going to strip them out and for those of us who knew to look for them, take away our fun. Never. Ever. Reveal your sources.
- albertzeyer 4y agoThis is already known. Google and DeepMind usually strip the comments out.
- gwern 4y agoDo they? I've read DM papers with the comments in the sources... but maybe they institutionalized a stripping phase.
- albertzeyer 4y agoMaybe older ones? But I'm also not sure how consistent they are. Google and DM are big. I don't really know their policies on publishing. Maybe not every group enforces it.
- deely3 4y agoCorrect me if Im wrong, but.. are you defending closed source and restriction of information?
- Tostino 4y agoThat's not at all how I read that comment... more like "through a side channel, we found this out" rather than spelling out exactly where they found it.
- withinboredom 4y agoIt’s the difference between going into a town square and yelling “look everyone, I found a treasure map!” And going on a treasure hunt. They found comments that were commented out for a reason. These commented out sections aren’t a good look for the author. Usually commented out sections are either funny, notes, or provide some extra context. He could have attempted to share this with someone in the industry of sharing information (like a reporter) who could validate it, and ask Microsoft for a comment — who will now be on the defensive instead of (potentially) forthcoming with more context about why those sections were commented out. This is a pretty fucked way of doing this.
- max_expectation 4y agoThere is a tool my supervisor always used to make me use to avoid this when posting to ArXiv: https://github.com/google-research/arxiv-latex-cleaner https://github.com/google-research/arxiv-latex-cleaner
- buildbot 4y agoErr wow, they left all the (very weird) comments in looking at the source. Our group always makes sure to strip the comments.
- kgeist 4y agoInteresting that it was originally called DV3 in the paper - looks similar to the name of the existing, older "davinci-003" model, which powered GPT-3
- AgentME 4y agoMaybe they planned to use the davinci-003 name for it originally, but then when GPT-4 took longer to make than they expected and a new revision GPT-3 came out first, they reallocated the name to that.
- zamnos 4y agoMore interestingly, "Davinci 3" is mentioned as an author of unknown affiliation*. Which, if it's referring to them having using davinci-003 to help author the paper would be interesting. It having unknown affiliation would but a) true and b) hilarious. * https://twitter.com/DV2559106965076/status/1638769438353940481/photo/1 https://twitter.com/DV2559106965076/status/16387694383539404...
- bombcar 4y agoSounds to me like the type of injoke common in early drafts of papers
- salad-tycoon 4y agoAh yes in jokes on a big early draft. Usually it’s funny because it’s at least a bit true but incongruent with the final work desired. Funny machine helping coauthor research on its self and investigating if it’s sentient with a “theory of mind “ and etc. or not and all the rest. Starting to sound like a great sci-fi book.
- kodah 4y agoInteresting that they note the power consumption and climate change impact. I believe there's a long list of folks who said this wasn't the case weeks ago.
- thrown123098 4y ago[flagged]
- ChatGTP 4y agoYup, kind of sucks but the end result is likely everyone and every actual living thing will suffer from this wave of “AI” because it’s an arms race now, let’s just keep burning fossil fuels and hope the thing spits out the answer to that problem ?
- jay_kyburz 4y agoHere is your answer to the fossil fuel problem. https://www.youtube.com/watch?v=kpeODvGJE1Q https://www.youtube.com/watch?v=kpeODvGJE1Q update: found a better one.
- JCharante 4y agoThe answer already exists but people refuse to implement it.
- kodah 4y agoWe have nuclear, but a lot of Americans are concerned about nuclear for reasonable reasons. We have renewables, but those can't take up the majority capacity of a grid unless we start adding massive batteries. Then there's grid rebalancing where we incentivize people to use and store renewable energy locally, this lessening the strain on the grid but that still results in fossil fuels or nuclear. Hydro has been found to be environmentally destructive. I'm not sure if that was just my state or if that's ubiquitous. What were you thinking?
- CatWChainsaw 4y ago
- psychphysic 4y agoEvery so often, very rarely, I end up wanting to read some twitter content. And I realise how agonisingly painful twitter threads are to consume. It's just as bad as those YOU WONT BELIEVE WHERE THOSE CELEBS ARE NOW. Where you had to click next one by one.
- deleted 4y ago[deleted]
- switch007 4y agonitter.net makes it slightly less painful. There are browser addons to auto redirect to nitter.net e.g. https://addons.mozilla.org/en-GB/firefox/addon/nitter-redirect/ https://addons.mozilla.org/en-GB/firefox/addon/nitter-redire...
- zamnos 4y agohttps://threadreaderapp.com/thread/1638769434763608064.html https://threadreaderapp.com/thread/1638769434763608064.html
- rvnx 4y agoLucky you, at least you can see the content. I could see only the first message, the second message says This Tweet is unavailable, the third is ok, and then a bunch of random/unrelated messages.
- layer8 4y agoSee https://news.ycombinator.com/item?id=35282579 https://news.ycombinator.com/item?id=35282579
- calf 4y agoDrives me nuts that scientists use Twitter x/n style writing. From Richard Feynman to Edward Tufte, we were told that PowerPoint talks are bad for science. And now Twitter writing style is uncritically accepted.
- zamnos 4y agoAccording to the Latex source, one of the original titles for this paper was "First Contact With an AGI System".
- armchairhacker 4y agoWhat makes GPT4 AGI that makes GPT3.5 not AGI? And what makes GPT3.5 AGI that makes GPT3 not AGI? And what makes GPT3 AGI that makes GPT2/smaller models not AGI? What is the "hard line" that makes something AGI or not AGI? Because IMO it looks like GPT4 is somewhat AGI, but also the older models possibly all the way down to even Markov chains: it's just that this AGI is nowhere near human-level.
- politician 4y agoIt’s a great title for a paper, so they’re probably saving it.
- beambot 4y agoWhat marks the hard line for "intelligent"? It's largely philosophical. Is a human? Dolphin? Mouse? Cockroach? Tree?
- esjeon 4y agoThe inability to fully define a thing doesn't invalidate all attempts to set its outline. At least, it's easy to conclude that an intelligent being has to be able to reliably perform basic reasoning (given all the necessary information is properly acquired). The current GPT models all fail at this, and neither the token length nor the network size can fix this.
- beambot 4y agoWhat do you mean by "reliably perform basic reasoning"?
- Thorentis 4y agoAbstraction and reflection have been posited in the past as pre-requisties for intelligence. "I am the thinker that is thinking." How do we prove whether an AI has this capability or not? I'd say it's nearly impossible. However, I think we can certainly prove when it doesn't. For instance, the fact we need an external plugin to get the model itself to return text claiming 1+1=2, tells me that GPT4 cannot reason about numbers in the abstract, and therefore lacks abstraction ability.
- knaik94 4y agoThe comments in interpretability read like science fiction to me. There's paragraphs on DV3 explaining other models and itself and the emergent properties that appear with bigger models. There's so much commented out related to functional explainability and counterfactual generations. "we asked DV3 for an explanation. DV3 replied that it detected sarcasm in the review, which it interpreted as a sign of negative sentiment. This was a surprising and reasonable explanation, since sarcasm is a subtle and subjective form of expression that can often elude human comprehension as well. However, it also revealed that DV3 had a more sensitive threshold for sarcasm detection than the human annotator, or than we expected -- thereby leading to the misspecification. To verify this explanation, we needed to rewrite the review to eliminate any sarcasm and see if DV3 would revise its prediction. We asked DV3 to rewrite the review to remove sarcasm based on its explanation. When we presented this new review to DV3 in a new prompt, it correctly classified it as positive sentiment, confirming that sarcasm was the cause of the specification error." The published paper instead says "we did not test for the ability to understand sarcasm, irony, humor, or deception, which are also related to theory of mind" . The main conclusion I took away from this is "the remarkable emergence of what seems to be increasing functional explainability with increasing model scale". I can see the reasoning for why OpenAI decided not to publish any more details about the size or steps to reproduce their model. I assumed we would need a much bigger model to see these level of "human" understanding from LLMs. I can respect Meta, Google, and OpenAI's decision, but I hope this accelerates the research into truly open source models. Interacting with these models shouldn't be locked behind corporate doors.
- PaulDavisThe1st 4y ago> "we did not test for the ability to understand sarcasm" I find it hard to see how detecting and eliminating sarcasm requires a theory of mind. It requires some association between various stylistic elements and the concept of sarcasm. The same is true of irony. I still wonder how many of these people have read Dennett's "The Intentional Stance", which holds that the best way to think about "intention" is as an explanatory model, not a mechanism. That is, we can say that the dog "behaves as if it has the intention to get inside" without making any claim about the internal state of the dog. Dennett further speculates that our own self-experience of intention is a matter of turning the same explanatory model upon our own behavior, but that's an extension that isn't directly relevant to this speculation about language models.
- subsubzero 4y agotweet #2 in the thread was missing, wonder what happened there?
- ReaLNero 4y agohttps://threadreaderapp.com/thread/1638769434763608064.html https://threadreaderapp.com/thread/1638769434763608064.html
- ChickenNugger 4y agoI mean does it mean it's accurate information? Could be a repurposed copy of some other document. Maybe they wanted it to be written by DV-3 and it didn't pan out, but they continued using the draft document anyway. I know from personal experience that I've had draft documents that were WILDLY wrong before I published to anyone but myself. Whole sections I just went back and completely deleted. In fact my senior project paper (LaTeX) in college had a whole section with big ASCII bull taking a shit on a paragraph because it was some work I'd done that didn't pan out at all. I left it in the source because I found it funny. lol, I found it: https://i.imgur.com/6Oj64AV.png https://i.imgur.com/6Oj64AV.png This was before I'd ever heard of a VCS system. Subversion 1.0 was released 6 months after I graduated, it turns out. So commented out code and multiple copies was all I had.
- boringuser1 4y ago[dead]
- tysam_and 4y agoThis is generally clickbait with a large amount of vapid information. I am surprised to be honest that HackerNews is giving the attention to it that they are. I would not encourage giving this any more attention. As an early draft, them putting in a placeholder of ~'this model uses a lot of compute {TODO: put in cost estimates here?}' does not at all equal 'the authors didn't even know how much it cost to train the model!' Additionally, of course the toxicity went down. There's a world of RLHF between here and that original draft, and they've shown how the non-RLHF model has lowered the toxicity of the untrained base model significantly. If the author of the tweets had done their due diligence, they might have noticed that. Rather obviously around the time when the model was originally being developed, text-only was sorta really the only way that LLMs were done. Them pivoting to multi-modal is just a natural part of following what works and what doesn't. This is really straightforward, I am mind-boggled that this is getting attention over discourse that is meaningful to the tidal wave of change coming with these models. One final note that this is a bit of shoveltext is at the bottom the author offers up vague concerns followed by mass-tagging accounts with high follow counts, to include Elon Musk. I'd encourage you not even to give the tweet the benefit of your view count and to just move on to more valuable discussions that are taking place. Why not take a look at a fun little thread like https://news.ycombinator.com/item?id=35283721 https://news.ycombinator.com/item?id=35283721 ? (not affiliated other the fact that I made the first comment on it, I just pulled from the rising threads on HN's frontpage)
- polskibus 4y agoCan anyone post the remaining hidden information from that paper in an easy to read form? Is there really some secret sauce in there ?
- jurimasa 4y agoMost of the paper probably was written by the model itself. They just removed the hallucinations.
- galaxyLogic 4y ago> we asked DV3 for an explanation. DV3 replied that it detected sarcasm They should have asked it "Why do you say there is sarcasm?" A human can answer that but I don't think a bot can, can it?
- Alifatisk 4y ago> we were worried about the unknown alignment procedures that OpenAI had taken to reduce the harmfulness of this powerful AI model Is the user referring to the ai alignment? the elitists who wants to nerf all the ai research?
- vaastav 4y agoAlways remember to clean up the .tex files before submitting to arxiv