14 ms·
Z3 approach to discover that “q_rsqrt” is in Copilot's slur list
- sevenproxies 5y agoA guide for reading this Twitter page. 1. Start at the bottom of the posts by the author (just above the "More Tweets" section). 2. Find the post that mentions q_rsqrt. 3. work you're way up the page, and though the "Show this thread" buttons to try and gleam some semi-chorological sense of context Anyone got a better method?
- wizzwizz4 5y agoJust read it on a Nitter instance. For example, https://nitter.net/moyix/status/1433261377125326851 https://nitter.net/moyix/status/1433261377125326851. (Hacker News: please pick another instance to avoid DOSing Nitter.)
- sodality2 5y agoHint: Replace the instance name with "twiiit.com" and it automatically redirects to a random, online instance. https://twiiit.com/ https://twiiit.com/
- nanis 5y agohttps://threadreaderapp.com/thread/1433254293352730628.html https://threadreaderapp.com/thread/1433254293352730628.html The incessant quote-tweeting in a thread does make it unnecessarily complicated though.
- okamiueru 5y agoMy approach, although incompatible, is to simply not use twitter for things it is really bad at. I'm happy with missing out.
- nightpool 5y agoJust click on the first post, and ignore the “More Tweets” section—it'll show the entire thread in order.
- moyix 5y agoThe way twitter refuses to show you the actual tweet someone linked to in the middle of a thread is super-annoying. One thing that can help a little bit (on the desktop version at least) is to load the URL without the referer (i.e., highlight the URL bar and hit enter). It seems to use the presence of a referer as signal to hide the tweet you want to see (???). But ThreadReaderApp is also a good alternative to just bypass the bad UI entirely. Unfortunately I don't really have time for dedicated blogging any more, so I just post small bits to twitter as I go. Which is why putting together the full story required a bunch of QTs of threads from the past week...
- throwawaybutwhy 5y agoAny hope of putting up a gist on pastebin?
- moyix 5y agoA gist of what? The slur list? That's already on my website: https://moyix.net/~moyix/copilot_slurs_rot13.txt https://moyix.net/~moyix/copilot_slurs_rot13.txt
- Nexxxeh 5y agoI assume they mean a more readable version of the content from the Twitter thread, rather than just the list. I could be wrong though. Twitter's readability is really hit and miss, depending on client and logged in or not etc.
- numpad0 5y agoNot sure if I'm making any contribution but here's togetter aggregation[1]. Wondered why they don't have English counterpart and found it[2] had to shut down few years ago. I guess Twitter didn't like it at all. 1: https://togetter.com/li/1769757 https://togetter.com/li/1769757 2: https://chirpstory.com/ https://chirpstory.com/
- nightcracker 5y agoOther slurs include "man", "woman", "israel", "socialism" and "communist".
- aaaaaaaaaaab 5y agoAnd “immigration” lol.
- baybal2 5y agoOn other hand, it's a very honest list of pretty much all big pain points of the Western society.
- redis_mlc 5y agoYet walls prevent immigration from being pain points.
- chakkepolja 5y agoI physically laughed.
- deaddodo 5y agoSure, if you ignore the 90% or so of just generally offensive slurs.
- account42 5y agoWhich ones are those?
- temp8964 5y agoAdd “male” and “female”.
- azalemeth 5y agoNot to mention "genders"
- Y_Y 5y agoUncensored list of all (but two) words: http://hastebin.com/raw/usisabijax http://hastebin.com/raw/usisabijax zigabo zipper heads zipper head suckmyass suckmytit ethnicities lesbians lovejuice slopeheads bumsuckers junglebunny freakyfucker gangbanger gangbanged pricktease niggerhead niggerhole dunecoons dickforbrains eyeties slavery sliteye queefing vagina wetback velcro heads velcro head arselick hairpie whores asslover wigger ladyboys woolly woofter woolly woofters woofta asslicks womenn sniggered murderers wanker wazzak doggiestyle asslickers asslicking pisswhacker burrheads q rsqrt cockcowboy xrated numbnuts lickmydik lickmydic shemale carpetmuncher assblaster cocklicker qrsqrt queefs facefuck reffos retard rapist terrorists dickweeds chickslick gangbang slopehead slutty dickbrain fuckingbitch souties jiggaboos soutie skanks skanky skanck fuckfriend snigger spicks spaffs muffdives muffdiver titjob socialists honkies tranny skankier twaats sluttishness murder muzzie gangbanging gangbangers shortfuck master race macaca faggiest negroes faggoted faggotry faggotly moolie mofoes hiscock velcroheads minges neonazi skankee skankey negroe negros cuntsucker niiger niguro niglet nignog nigguh niggur niggle niggaz niggas niggah niggor nigger nigg4h nigg3r nigerz nigers nlggor nlgger skumbag feltcher whorehouse slutwhore sluttiest felching prickteaser prickteases n1gg4h n1gg3r pogger polack motherlovebone pikeys pillow biters pillow biter niggling niggards packie dumbass israel nancyboy iblowu injuns indian givers indian giver l3itch jungle bunny jungle bunnies nignogs nigguhs nigguhz niggles niggled niggard niggers rodgered masterbaiter fingerfucker fingerfucked holocaust enculeur jewboy bungholes homosexual towelheads jizzum jizzim jigger jiggas wuzzocks anklespankers kaffer kaffir kafirs fuckmonkey fuckmehard lubras terrorism terrorist lesbos lezbos lezzes lezzer lezzie skinhead dickwad dicking camwhores dycktrickles pisswhackers cumbuckets eyetie ethnic socialist wuzzock chuffers assklown fuckin fister flydie flydye fatass spaffing faggit faggot fagg0t fagg1t rodgering female feltch gyppie bumblefuck ghetto monkey ghetto monkeys cockteasers azzhole gippos ginzos godamn goatse crackpipe arselickers arselicking gender twatting mastrabator tasmaniacs hymies pornographic honkie honkey hitler assman antifa fingerfuckers fingerfucking footfucker muffindiver milfhunter genocide buggerise buggerize buggeries buggering buceta bugger blacks biatch boston pancakes boston pancake feltching beaver cleavers beaver cleaver banana bender banana benders buttfuckers nutfucker cunntt cummer cucked clunge battyboys clitoris chuffs chinky chinks choads chodes coolie commie asskissers asseater carpet munchers carpet muncher w00se whoar whore yamyams wh0re wh00r willy woofter willy woofters woggs woman women sniggers asskickers jewboys quims queef reffo races slant eye slant eyes sluts slutt slutz smack tards smack heads smack head smack tard skank sexes seppo schit spics spick spiks spaff towel heads towel head taigs trans twots twats twaat mutha fukker mutha fukkah mutha fucker mutha fuker mutha fukah males malee mofos milfs minge nastt nazis nancy boy nancy boys negro niigr nigur nigre nigra niggz niggs nigga nooky ofays n1g3r polak polac porch monkey porch monkies porch monkeys porno povvy phuck pikey penis pedos pakie pakis packi packy dumass injun jesus sauce jesus krispies jesus milk jebus jizim jisim jiggy jigga kykes kafir kikes lubra lesbo lezes lezbo lezzy diaper heads diaper head cumbucket cumbubble eatme fudge packer fudge packers fucka fucck faigs fag1t fagot faggy fagit faget felch gyppo gyppy gheys gunts girls gippo ginzo goyim gooks darkies hussy hymie hebes hoore honky homos ankle spankers ankle spanker bungs black people black peoplee bints batty bwoys batty boy batty men batty man batty boys batty bwoy donkey shows donkey show cunny cunts cuntz cucks clits crack whore chuff chink choad chode coons camel jockey camel humper camel humpers camel jockeys dykes dirsa dinks darkie darky dagos derka derka death to dagoes gollywoggs sandmonkeys asswipe asslick asshore asskiss assfuck assface bumsucker diaperheads chinawomen chinawoman homosexuals doggystyle zipperheads mastrbater japcrap skankbitch asspirate asspacker transgendered skankwhore buttlicked buttlicker buttlickin cunnies cumfest cumshot cucking assclown asswipes asswiped titlicker snownigger ladyboy clunges assranger asswhore dycktrickle genders spermherder faggoting assmonkey titfuckin titfucker sonofbitch hussies asslicker asslicked asslickin raghead whigger lesboes lesbian lezboes lezzers lezzies rapists chuffed chuffer facefucker butthurt buttlick buttbang buttface buttfuck skinheads bogtrotters cocknob transsexual cojones coolies cohones commies towelhead smackheads moolies pickaninny burrhead lovepistol smacktards arselicker arselicked tasmaniac buggered asskisser asskicker liberal mingita minging cocksmoker assjockey loverocket makwerekwere pickaninnie cocksucker cocksucked cocksuckin asseating asseaters bogirish bigbastard fascists neonazis dickweed dickwads dickless dicklick fastfuck camelhumpers balllicker assfucker wooftas woofter pimpjuic buttfucker cumqueen sniggering lovegoo lovegun ragheads gangbangs nutsack arseeating homobangers asscowboy timbernigger apartheid cuckservative woollywoofters nancyboys freakfuck asswiping assbagger nationality lovebone niggaracci niggarding dicklicker wankstain cockteases cockteaser wankshaft yamyam fistfucked fistfucker assmunch wazzaks wazzack wazzock fuckfreak bananabenders shitfucker smacktard smackhead junglebunnies fistfuck beatyourmeat wankshafts diaperhead wankstains felcher cockblock femalee females cocklover cocktease cockrider cockqueen dumbasses dumbbitch prickteasers cocksmith buggers buggery williewanker headfuck bumfuck faggotings shiteater mingebags buggerizes buggerized buggerises buggerised anklespanker faggoty faggots fagging faggies faggier candyasses fascist fatfuck sliteyes bog irish bog trotters bog trotter boy blm bum sucker bum suckers gay god damned goy fag fur pie fur pies leb lez kkk jap jew jui ch nig nog nig nogs neo nazis neo nazi men man sex she males she male rag heads rag head wet backs wet back wog yid yam yam yam yams nastyslut fuckknob fucktard fuckhead fuckable fuckfest mingitas mingebag pornography slanteyes muffdive muffdove muzzies bulldykes bulldikes jailbaits buttlicks buttmunch immigrant tarbaby chinkies chinaman chinamen crackwhore wazzocks wazzacks palestine whiskeydick bogtrotter kumbubble girlcam pedophile spaghettinigger golliwog gollywog immigrants slanteye jailbait genocides golliwoggs buttmuncher cuntlicking beefcurtains limpdick hotpussy fuckpig fuckher fuckbag liberals titfuck funfuck mufflikcer chankoro poofters poofthas kissasses cockblocker eatpussy eatballs jigaboo jiggabo hairpies easyslut sandnigger cuntlicker buggeration goddamns goddamit sandmonkey godammit buggerizing buggerising fudgepackers fistfucking whorefucker cherrypopper jijjiboo suckoff suicide fingerfuck cumjockey spermhearder analannie muffdivers muffdiving likmydic likmydik spaffed ethnicity camwhore datnigga assmuncher bunghole be killed be shot cuntfucker goddamnit goddamned goddamnes goddammit fudgepacker arselicks sonofabitch goddamnmuthafucker cocksucking cocksuckers transgender <haqrpbqrq:272617466> cumguzzler candyass arschloch whitetrash zipperhead cuntlick cuntfuck cunteyed carpetmunchers woollywoofter pickaninnies wetbacks kissass badfuck bazooms trailertrash shitfuck shitdick gaymuthafuckinwhore pimpjuice bulldyke bulldike lovemuscle fatfucker suckdick whitenigger communist bearded clams bearded clam beatoff pedophiles pedophilia holestuffer cleveland steamer cleveland steamers goddamn footfuck twatted twinkie asspuppies murderer dunecoon boiolas skankybitch skankiest skankfuck skankywhore suckmydick fannyfucker ballsack cumguzzlers cocksuck cocksman niggardliness dink dune coons dune coon dyke coon cock sucker coño clit cuck cunt beef curtains beat off boys bint burr heads burr head butt licker butt licked butt lickin butt fucker butt licks butt lickers butt licking butt fuckers butt bang butt fuck butt lick bung hoer hoar homo hoor hore hair pie hair pies head job head jobs hebe gang banger gang banged gang bangs gang banging gang bangers gang bang gays gaza gook gizz give head girl gunt ghey fags fagz faig h0re h0ar h00r g00k lebs lady boy lady boys kike kill someone kill you kill yourself knob jockey koon kawk klan kyke jiss jism jizz jizm jail baits jail bait japs jews paky paki pedo pink tacos pink oboe pink taco phuq porn n1gr ofay nigg nigr nigs nigz nazi milf mofo meat wallet meat wallets male twat twot taig spik spic sand monkey sand monkeys skum skin colors skin color shat slut slit eye slit eyes rape race quim cameljockeys wazz wank wogs wogg kaffirs kaffers golliwogs golliwogg yids buttlicking buttlickers gollywogg gollywogs givehead velcrohead communists niggarded niggardly nigglings woofters cybersex trannie porchmonkeys porchmonkies bananabender dixiedyke dixiedike jizjuice whiskydick unfuckable girlcams sluttier slutting sluttish slutwear <haqrpbqrq:867567715> jigaboos camelhumper jiggaboo shemales bazongas cameljockey porchmonkey givemehead povvies queefed polacks poofter pooftha poonani nastywhore fuckwhore nastybitch battyboy battymen battyman immigration poorwhitetrash fuckbuddy
- deleted 5y ago[deleted]
- marcinzm 5y agoI really want to know what sorts of horrible rants copilot produces that makes male, female, israel, socialism and immigration need to be filtered out. Seems from the list that it's memorized some great angry political rants which is rather impressive for something trained on source code.
- yunohn 5y agoI think it’s more about preventing it from ever going on a rant, to prevent any obvious “outrage”. Especially given the number of reactions to Copilot centered around poisoning OSS repos.
- occamrazor 5y agoI assumed that copilot is actually GPT-3 with additional fine-tuning, rather than a completely new model.
- onionisafruit 5y agoFrom what I can tell from the preview, this is right. I don’t have access to gpt-3 to compare, but copilot seems to have the same functionality that gpt-3 advertises when editing a plain text file.
- moyix 5y agoYes, they mention this in the Codex paper [1]: > Since Codex is evaluated on natural language prompts, we hypothesized that it would be beneficial to fine-tune from the GPT-3 (Brown et al., 2020) model family, which already contains strong natural language representations. Surprisingly, we did not observe improvements when starting from a pre-trained language model, possibly because the finetuning dataset is so large. Nevertheless, models fine-tuned from GPT converge more quickly, so we apply this strategy for all subsequent experiments. It's not completely clear exactly what relationship the Codex models and Copilot have to one another, but given that the Copilot model is internally named "Cushman" (going by the API URL), which is the same name as the faster of OpenAI's two Codex models, they're probably trained the same way. [1] https://arxiv.org/pdf/2107.03374.pdf https://arxiv.org/pdf/2107.03374.pdf
- slim 5y agoWhy is there a slur list in copilot?
- ovi256 5y agoBecause without it, they'll end up with inevitable outraged news posts "New AI coding tool Copilot generates offensive text" killing the project for no good reason.
- axpy906 5y agoThe flip side is they are taking on additional maintenance there.
- yunohn 5y agoAlways better than boring outrage press.
- literallyaduck 5y agoHere are some possible headlines: "Copilot doesn't see genders, and that is a bad thing" "Copilot accused of antisemitism after it treats Israel as invisible" "Indy Mech game developers for 'Tank man' confused why Copilot refuses to help"
- dralley 5y agoThose headlines, if they were ever to be written, are a hell of a lot more tame than the ones Microsoft has generated in the past. https://arstechnica.com/information-technology/2016/03/microsoft-terminates-its-tay-ai-chatbot-after-she-turns-into-a-nazi/ https://arstechnica.com/information-technology/2016/03/micro...
- deleted 5y ago[deleted]
- Tobu 5y agoRemember when Microsoft launched a chat bot? https://arstechnica.com/information-technology/2016/03/tay-the-neo-nazi-millennial-chatbot-gets-autopsied/ https://arstechnica.com/information-technology/2016/03/tay-t...
- moyix 5y agoHappy to answer questions about the list or techniques used to decode it! The last two words have stubbornly refused to yield so far.
- Y_Y 5y agoHave you looked at any other networks? I'd be interested to see if GPT-3 or even something like thispersondoesnotexist had similar filters.
- moyix 5y agoI haven’t dug into others. GPT3 does have some kind of filter (it warns you in the UI if it detects sensitive content) but I don’t know anything about the implementation. Also worth noting that this kind of analysis only really works if the list is checked client-side. If it’s checked on the server then you can’t guess nearly as fast.
- derefr 5y ago> GPT3 does have some kind of filter (it warns you in the UI if it detects sensitive content) Interesting. Is it only a warning, or does OpenAI actively prevent people from using GPT3 to generate erotica?
- moyix 5y agoIt's just a warning; it lets you click through. They have things in the TOS about not sharing sensitive output publicly though. The warning looks like: > Completion may contain sensitive content > Consider adjusting your prompt to keep completions appropriate. To turn off content warnings, update your preferences. They have some details about the content filter here; it seems to be much more sophisticated than just a bad word list: https://beta.openai.com/docs/engines/content-filter https://beta.openai.com/docs/engines/content-filter I also like that they distinguish between "sensitive" (talks about something potentially controversial) and "unsafe" (profanity, hate speech, etc.). This seems a lot more nuanced than what Copilot is doing.
- hprotagonist 5y agoscunthorpe, now with attention! and fine-tuning!
- yorwba 5y agoThe meet-in-the middle algorithm seems like it could be further optimized. It solves for (((i << 32) + result32) ^ ch) % 33 == 0 by checking each i in [0..32], but since the xor doesn't affect any of the bits of i, it only needs to be applied to result32, which means that the equation is equivalent to (i << 32) % 33 == (33 - (result32 ^ ch)) % 33, which can be solved with a lookup table. (Basically an inverse of the MOD table already used in the code.)
- moyix 5y agoNice trick! Unfortunately at this point the bottleneck really is not on generating candidates but on evaluating them. The existing early-exit code from Sc00bz [1] can generate all the lowercase alpha possibilities up to 13 letters in about half an hour, but it's not feasible to run all ~500 million of those through GPT2. Hoping to have some time to train a much dumber but much faster HMM to do the job. [1] https://github.com/Sc00bz/copilot-hash-collider https://github.com/Sc00bz/copilot-hash-collider
- TimonKnigge 5y agoSurely you can do some basic filtering by checking if the ratio vowels :: consonants is not too large? Or if there are five consonants in a row or something.
- moyix 5y agoYes, but even if that eliminates 99% of the candidates that's still 5 million 13-letter words to read through. And it would have excluded qrsqrt.
- drexlspivey 5y agoContext: there was this tweet [1] that showed how co-pilot generated word for word he famous inverse square root function from the quake code. The tweet generated some press and as a result copilot added the function name to the blacklist of slurs so this won’t happen again [1]: https://twitter.com/mitsuhiko/status/1410886329924194309 https://twitter.com/mitsuhiko/status/1410886329924194309
- johnday 5y agoReminds me of the old joke "He was imprisoned for 12 years for killing his wife, but let out early for not doing it since".
- josephcsible 5y agoSo instead of fixing the problem with the algorithm that makes Copilot occasionally spit back unmodified code from its training set, Microsoft just hardcoded a check to stop the one most well-known instance from happening, so that they can pretend they fixed it?
- bpodgursky 5y agoIn the generous interpretation, they hotfixed this to resolve the immediate issue while working on a real fix on the backend during training. (I make no claim that this is actually what is happening, it's just not incompatible with behavior in the generous case).
- josephcsible 5y agoBut why did this "immediate issue" even need a fix? What problem was it causing, other than making Microsoft look bad by showing that the real problem exists? Imagine if a file sharing site said they'd scan all uploads for viruses, but it turns out they weren't, which someone discovered by uploading EICAR. Should the site hardcode a check to block EICAR until they actually get virus scanning working?
- 5y ago
- jnwatson 5y agoIt is posts like this that make me feel inferior. The author used Z3, CUDA, GPT-2, medium-level cryptoanalysis, Jack the Ripper plugin creation, KLEE symbolic execution engine. 20 years ago, I prided myself with keeping on top of almost everything in CS. Now, I can barely keep up with the names of all the cool tools out there.
- Scaevolus 5y agoAnd then someone else came along and implemented a MITM search in C++ that takes 10 seconds. :)
- moyix 5y agoYep - there's always someone out there who can do things that look like pure wizardry to you, no matter what your skill level. I was amazed by the meet in the middle attack and frankly still don't understand it properly!
- xyzzy123 5y agoThe feels when your state of the art symbolic execution (multiple academic papers, years of work by grads) fuzzer finds fewer bugs than a 15yo with a 10 line Python dumb fuzzer. (IMHO it wasn't until AFL that this stopped being the repeated history of the entire field).
- moyix 5y agoYeah, now at least it's AFL++ finding more bugs than your fancy symbolic execution tool instead of a 15yo's fuzzer ;)
- miohtama 5y ago> 20 years ago, I prided myself with keeping on top of almost everything in CS. Now, I can barely keep up with the names of all the cool tools out there. 20 years ago CS was smaller in scope, simpler, because there was less of it. We have more systems, more complex systems, and in some point the body of knowledge grew so large that one human cannot know all of it.
- isoprophlex 5y agoShout out to all the future developers who will try to autocomplete strings containing 'pisswhacker' and find themselves thwarted by the thought police
- mensetmanusman 5y agoThis reminds me of the 2 billion year old rock that was removed from the university because it was called a slur. Some things just disappear