15 ms·
ByteDance sacks intern for sabotaging AI project
- userbinator 2y agoI hope said intern finds a new job working for anti-AI causes.
- 0xDEAFBEAD 2y agoAre there are a lot of anti-AI organizations at this point? PauseAI is the main one I'm familiar with: https://pauseai.info/ https://pauseai.info/ One thing I suspect investors in e.g. OpenAI are failing to price in is the political and regulatory headwinds OpenAI will face if their fantastical revenue projections actually materialize. A world where OpenAI is making $100B in annual revenue will likely be a world where technological unemployment looms quite clearly. Polls already show strong support for regulating AI.
- bawolff 2y agoRegulation is not neccesarily bad for the market leader.
- jazzyjackson 2y agoThe Amish? I'm trying to think of whether it'd be worth starting some kind of semi-Luddite community where we can use digital technology, photos, radios, spreadsheets and all, but the line is around 2014, when computers still did the same thing every time. That's my biggest gripe with AI, the nondeterminism, the non-repeatability making it all undebuggable, impossible to interrogate and reason about. A computer in 2014 is complex but not incomprehensible. The mass matrix multiplication of 2024 computation is totally opaque and frankly I think there's room for a society without such black box oracles.
- fragmede 2y agoWhy 2014? Why not 2022 when ChatGPT was released? Or 2019 for ChatGPT 2? Why not 2005 when the first dual-core Pentium was released? After that, the two cores meant that you could be sure what order your program would run things. Or why not 2012 when Intel added the RdRand instruction to x86? Or 2021 when Linux 5.17 was released with random number generation improvements? Or 1985 when IEEE 754 floating point was released. Before that, it was all integer math but after that, 0.1 + 0.2 = 0.30000000000000004. Not that I have any objection to 2014, I'm just wondering why you chose then.
- jazzyjackson 2y agoIf I was really picky I would stop the clock in the 8bit era or at least well before speculative execution / branch prediction, but I do want to leave some room for pragmatism. 2014 is when I became aware of gradient descent and how entropy was used to search more effectively, leading to different runs of the same program arriving at different results, Deep Dream came soon after and it's been downhill from there If I were to write some regulations for what was allowed in my computing community I would make an exception for using PRNGs for scientific simulation and cryptographic purposes, but definitely I would draw a line at using heuristics to find optimal solutions. Slide rules got us to the moon and that's good enough for me.
- Sohcahtoa82 2y agoNitpick: > Or 2019 for ChatGPT 2 That wasn't ChatGPT, that was GPT-2. It wasn't even designed for "chat" and was purely text completion. If you tried to ask it a question, it was a toss-up over whether you'd get an answer or just a bunch of related questions and statements, as if it was part of what a single speaker was saying. Like, you could prompt it with "I'm here to talk about" and it would complete it with some random subject. I don't even know if any of the well-known LLMs (Mistral, Llama, what else?)can even operate in this mode now. Seems they're all being designed for being an assistant.
- 542458 2y agoFwiw, the Amish aren’t luddites, they’re not anti-technology in all facets of life. You’ll see Amish folks using power tools, cellphones, computers, etc in their professional lives or outside the context of their homes (exact standards vary by community). There are even multiple companies that manufacture computers specifically for the Amish. So there’s no reason an Amish business couldn’t use AI.
- 0xDEAFBEAD 2y agoDon't they have a process for determining whether new technology should be integrated into their lives?
- 542458 2y agoYes, the exact process varies by community but it generally involves church elders meeting to discuss whether a new technology is likely to benefit or harm family, community and spiritual life.
- sadeshmukh 2y agoRegulation supports the big players. See SB 1047 in California and read the first few lines: > comply with various requirements, including implementing the capability to promptly enact a full shutdown, as defined, and implement a written and separate safety and security protocol, as specified That absolutely kills open source, and it's disguised as a "safety" bill where safety means absolutely nothing (how are you "shutting down" an LLM?). There's a reason Anthropic was championing it even though it evidently regulates AI.
- 0xDEAFBEAD 2y ago>That absolutely kills open source Zvi says this claim is false: https://thezvi.substack.com/p/guide-to-sb-1047?open=false#%C2%A7false-claim-the-shutdown-requirement-bans-open-source https://thezvi.substack.com/p/guide-to-sb-1047?open=false#%C... >how are you "shutting down" an LLM? Pull the plug on the server? Seems like it's just about having a protocol in place to make that easy in case of an emergency. Doesn't seem that onerous.
- Tostino 2y agoWhich server? The one you have no idea about because you released your weights and anyone can download/use them at that point?
- sadeshmukh 2y agoTo be fair, I don't really agree with the concept of "safety" in AI in the whole Terminator-esque thing that is propagated by seemingly a lot of people. Safety is always in usage, and the cat's already out of the bag. I just don't know what harm they're trying to prevent anyways at all.
- pjc50 2y agoSAG-AFTRA are currently on strike over the issue of unauthorized voice cloning. The AI advocates actively advertised AI as a tool for replacing creatives, including plagiarizing their work, and copying the appearance and voices of individuals. It's not really surprising that everyone in the creative industries is going to use what little power they have to avoid this doomsday scenario.
- bawolff 2y agoPeople who sabotage things tend to do it against all sides (you can always find an excuse to sabotage if you try hard enough).
- tommica 2y ago> People who sabotage things tend to do it against all sides (you can always find an excuse to sabotage if you try hard enough). 'Holy Generalization, Batman!'
- xvector 2y agoI hope he spends a good long bit in prison
- Aurornis 2y agoThe story going around is that the sabotage was to make their own AI projects look better than their peers. This wasn’t an anti-AI crusader. It was petty office politics.
- peterkos 2y agoI'm reminded of a time that an intern took down us-east1 on AWS, by modifying a configuration file they shouldn't have had access to. Amazon (somehow) did the correct thing and didn't fire them -- instead, they used the experience to fix the security hole. It was a file they shouldn't have had access to in the first place. If the intern "had no experience with the AI lab", is it the right thing to do to fire them, instead of admitting that there is a security/access fault internally? Can other employees (intentionally, or unintentionally) cause that same amount of "damage"?
- raihansaputra 2y agoafaik this was intentional in that they stopped training runs and changing parameters for other employee training runs, and even joined in on the debugging group trying to solve the "issues".
- dudus 2y agoThe difference in this case is intent. Did the employee have the intent to cause damage? If so just fire him/her.
- danpalmer 2y agoMalicious intent to be precise. Well-intentioned attempts to demonstrate issues for the purposes of helping to fix should generally not be punished, unless there is a wider fallout than expected and that can be attributed to negligence.
- grogenaut 2y agoFrom what I've seen in Amazon it's pretty consistent that they do not blame the messenger which is what they consider the person who messed up. Usually that person is the last in a long series of decisions that could have prevented the issue, and thus why blame them. That is unless the person is a) acting with malice, b) is repeatedly shown a pattern of willful ignorance. IIRC, when one person took down S3 with a manual command overriding the safeguards the action was not to fire them but to figure out why it was still a manual process without sign off. Say what you will about Amazon culture, the ability to make mistakes or call them out is pretty consistently protected.
- aaron695 2y ago[flagged]
- viraptor 2y agoThe article quoting specific responses is garbage, here's a tweet explicitly stating it includes a rumour? What are you trying to say here?
- iamacyborg 2y agoHe’s basically highlighting why the media is dead. Gullible folks would rather read salacious rumours than actual news.
- solenoid0937 2y agoIt's legit. Read the malicious changes he made to the code and read the posts from the researchers. And sorry, people are not "gullible" for disbelieving the media. I have worked at most big tech companies and the media misreports so badly on easily verifiable things in my area of expertise, that I no longer trust them on much. https://en.m.wikipedia.org/wiki/Michael_Crichton#Gell-Mann_amnesia_effect https://en.m.wikipedia.org/wiki/Michael_Crichton#Gell-Mann_a...
- bobthepanda 2y agoIn a heavily controlled media landscape like China’s, eventually the rumors become the only source of credible news. Most major Western news media are sourcing at least some China stories from WeChat and Sina Weibo before it gets scrubbed by censors.
- novolunt 2y ago[dead]
- dankle 2y agoWhat a non-story.
- tmalsburg2 2y agoBut AI!
- HuangYuSan 2y agoAND China!
- xvector 2y agoThis guy maliciously interfered with ByteDance's LLM project. That is huge news. Millions or billions of dollars down the drain.
- sokoloff 2y ago> That is huge news - billions down the drain. Billions of what? milli-dollars? Bytedance denies the damages are over $10M.
- solenoid0937 2y agoOf course they will deny it, they have investors... Read the posts from the engineers - 30 people's research and large model training coming to a grinding halt for a quarter. That's easily worth billions in today's market, can you imagine if OpenAI or Google didn't report any progress on a major model for a quarter?
- deleted 2y ago[deleted]
- needaname 2y agoIt was a phd student that was mad about compensation or something purposely injecting malicious code.
- anigbrowl 2y agoI feel less informed after reading the article than I did after reading the headline.
- deleted 2y ago[deleted]
- ratedgene 2y agoyes, the article has nothing about what they were doing or how they "interfered" exactly.
- deleted 2y ago[deleted]
- rollulus 2y agoThis article merely relays what ByteDance says, so it’s nothing but PR, unrelated to journalism. No idea what it’s doing on bbc.com.
- quietbritishjim 2y agoNot really. It says: > ByteDance also denied reports that the incident caused more than $10m of damage It makes clear what ByteDance's official position is, while pretty clearly hinting that it might not be true.
- radu_floricica 2y ago"maliciously interfering" does a lot of the lifting here. And if true, I hope that they didn't stop at firing him. Play stupid games, win stupid prizes. I hate the kind of entitlement that makes people feel justified to destroy huge amounts of value.
- aimazon 2y agoThe context is here: https://github.com/JusticeFighterDance/JusticeFighter110 https://github.com/JusticeFighterDance/JusticeFighter110
- yapyap 2y agowhats this mean for us non chinese folk
- xvector 2y agoTranslated by ChatGPT. Summary: 10/18: Translation of the provided text: Title: Urgent Warning The “reputation washing” behavior of Tian Keyu has been extremely harmful For the past two months, Tian Keyu has maliciously attacked the cluster code, causing significant harm to nearly 30 employees of various levels, wasting nearly a quarter’s worth of work by his colleagues. All records and audits clearly confirm these undeniable facts: 1. Modified the PyTorch source code of the cluster, including random seeds, optimizers, and data loaders. 2. Randomly killed multi-machine experiment processes, causing significant experiment delays. 3. Opened login backdoors through checkpoints, automatically initiating random process terminations. 4. Participated in daily troubleshooting meetings for cluster faults, continuing to modify attack codes based on colleagues’ troubleshooting ideas. 5. Altered colleagues’ model weights, rendering experimental results unreproducible. It’s unimaginable how Tian Keyu could continue his attacks with such malice, seeing colleagues’ experiments inexplicably interrupted or fail, after hearing their debugging strategies and specifically modifying the attack codes in response, and witnessing colleagues working overnight with no progress. After being dismissed by the company, he received no penalties from the school or advisors and even began to whitewash his actions on various social media platforms. Is this the school and advisors’ tolerance of Tian Keyu’s behavior? We expect this evidence disclosure to attract the attention of relevant parties and for definitive penalties to be imposed on Tian Keyu, reflecting the social responsibility of higher education institutions to educate and nurture. We cannot allow someone who has committed such serious offenses to continue evading justice, even beginning to distort facts and whitewash his wrongdoing! Therefore, we decide to stand on behalf of all justice advocates and reveal the evidence of Tian Keyu’s malicious cluster attack! Tian Keyu, if you deny any part of these malicious attack behaviors, or think the content here smears you, please present credible evidence! We are willing to disclose more evidence as the situation develops, along with your shameless ongoing attempts to whitewash. We guarantee the authenticity and accuracy of all evidence and are legally responsible for the content of the evidence. If necessary, we are willing to disclose our identities and confront Tian Keyu face-to-face. Thanks to those justice advocates, you do not need to apologize; you are heroes who dare to speak out. Link to the inquiry recording of Tian Keyu: https://www.youtube.com/watch?v=nEYbYW--qN8 https://www.youtube.com/watch?v=nEYbYW--qN8 Personal homepage of Tian Keyu: https://scholar.google.com/citations?user=6FdkbygAAAAJ&hl=en https://scholar.google.com/citations?user=6FdkbygAAAAJ&hl=en GitHub homepage of Tian Keyu: https://github.com/keyu-tian https://github.com/keyu-tian 10/19: Clarification Regarding the “Intern Sabotaging Large Model Training” Incident Recently, some media reported that “ByteDance’s large model training was attacked by an intern.” After internal verification by the company, it was confirmed that an intern from the commercial technology team committed a serious disciplinary violation and has been dismissed. However, the related reports also contain some exaggerations and inaccuracies, which are clarified as follows: 1. The intern involved maliciously interfered with the model training tasks of the commercial technology team’s research project, but this did not affect the official commercial projects or online operations, nor did it involve ByteDance’s large model or other businesses. 2. Rumors on the internet about “involving over 8,000 cards and losses of millions of dollars” are greatly exaggerated. 3. Upon verification, it was confirmed that the individual in question had been interning in the commercial technology team, and had no experience interning at AI Lab. Their social media bio and some media reports are incorrect. The intern was dismissed by the company in August. The company has also reported their behavior to the industry alliance and the school they attend, leaving further actions to be handled by the school.
- yapyap 2y ago> Its commercial online operations, including its large language AI models, were unaffected by the intern's actions, the company added. so did something actually happen or did they just post some inaccuracies on social media
- sss111 2y agoSounds like Tian Keyu stumbled on something he wasn’t supposed to see — now they’re trying to bury him alive.
- Sabinus 2y agoIf that was the case the State would already have arrested him.
- treksis 2y agoThis is more or less issue of team lead. Just assigned wrong task to intern.
- lysace 2y agoI find it weird that China has a very tight information control and simultaneously over and over again has the weirdest "netizen" rumors that go mainstream. What's the explanation? That they are explicitly allowed for some strategical reason? Something else? Edit: @dang: Sorry in advance. I do feel like we got some pretty good discussion around this explosive topic, at least in its first hour. Folks, keep up the good behavior — it makes me look good.
- nuz 2y agoOne idea is that they're fake planted rumors. Certanily not the first time things like that happen
- lysace 2y agoIf people get to read shocking rumors, they don't feel that their information access is so censored, after all? I could see that at least partially working. "It's just some dangerous information that is censored."
- okasaki 2y agoWell that's what the Washington regime and its media lackies do anyway, "according to a confidential source at the Whitehouse..."
- erulabs 2y agoMy explanation is that their tight control is an illusion. Not to get political, but the illusion of power is power, and suggesting they control billions of peoples speech is certainly an illusion of power. China, and all other (supposedly) top-down-economies, survive only because their control is not airtight. If they were to actually have complete control, things would fall apart rapidly. “No one knows how Paris is fed” and all that.
- candiddevmike 2y agoIan Malcolm said it best: "the kind of control you're attempting simply is... it's not possible. If there is one thing the history of evolution has taught us it's that life will not be contained." Humans are clever and typically find workarounds given enough time/hope. Sure you could argue that this is some kind of authoritarian 4D chess/matrix scenario to let off steam for an unruly populace, or it's just the natural course of things.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- knowitnone 2y agoCall me paranoid..."paranoid." but this could be a good way for ByteDance to redirect blame to others when they do something in the future that people don't like. "It was a rouge employee and we fired them"
- londons_explore 2y agoSeems like the kind of thing that will work once only...
- stuckkeys 2y agoSo far that we know off haha. It could happen anywhere. Maliscious content is a thing.
- CaptainNegative 2y agono need to be colorist
- e40 2y agoIt had on the right letters, tho.
- 1123581321 2y agoPerhaps the rouge employee was red-teaming.
- robofanatic 2y agoI assume he is not the only one responsible because its hard to believe interns code wasn’t reviewed before releasing.
- dragonelite 2y agothe infamous "looks good to me" PR review.
- robofanatic 2y agoLGTM!
- kazinator 2y agoOTOH: ByteDance intern responsible for spamming your web server with crawlers that ignore robots.txt given permanent position with a raise, now in management.
- 123yawaworht456 2y agohonoring robots.txt is an informal courtesy, not international law.
- davemp 2y agoNot breaking the law is just about the lowest bar you can set for an organization.
- not_a_bot_4sho 2y agoWe can go lower
- hnfong 2y agoFYI, we're still not sure whether the scraped AI training datasets involve copyright infringement.
- bsebbreebd 2y ago[dead]
- wyldberry 2y agoWhat we often think of as Insider Threat in the west is just another Tuesday in Chinese business. I have many experiences of this in the video game industry. This industry sabotage and theft is a very real part of getting ahead, even amongst companies that are owned by the same parent company (ex: studios owned in part by Tencent).
- deleted 2y ago[deleted]
- quinttttle 2y agofor every gifted, motivated, positive, constructive person in the world, there are probably 100,000 that want to tear them down and eat the carcass.
- llamaimperative 2y agoNo there’s not. This is what narcissistic assholes tell themselves when someone calls them out for being a narcissistic asshole.
- roywiggins 2y ago10,000 people is as many people as some entire towns, I don't think society would hold together very long if it were true. 100,000 supposes that there are... hmm... about eighty thousand non-evil people in the world, and (odds are) exactly none of them are Marshallese and about 2 are Samoan, to give a sense of how silly this is.
- Log_out_ 2y agoIt doesn't . But usually that third of the population is busy going for each others throat and ignoring the "fools" while then taking the coins that mysteriously spawn near those as psychopath price mobey.
- BobaFloutist 2y ago
- phendrenad2 2y ago> the intern allegedly "maliciously interfered with the model training tasks" for a ByteDance research project Did the intern post a manifesto or something? What was the point of doing this?
- dang 2y agoRelated: https://arstechnica.com/tech-policy/2024/10/bytedance-intern-fired-for-planting-malicious-code-in-ai-models/ https://arstechnica.com/tech-policy/2024/10/bytedance-intern... (via https://news.ycombinator.com/item?id=41906970 https://news.ycombinator.com/item?id=41906970, but we merged that thread hither)
- nl 2y agoThe story that was going around on social media (which I only know because Claude refused to translate it sometimes) was that a particular developer was modifying weights in other developers models and crashing their training runs so that the developers own work looked better in comparison. https://twitter.com/YouJiacheng/status/1847420973580243092 https://twitter.com/YouJiacheng/status/1847420973580243092
- cushpush 2y agohow is this any different from starting new projects at google and leaving them in a half-baked state because that leads to a promotion faster? incentives align behavior
- nl 2y agoIf you really can't see how that is different I don't know what to say.
- deleted 2y ago[deleted]
- cushpush 2y agorhetorical questions are not questions
- throwaway314155 2y agoMaybe don't post stupid comments if you don't want people to point out that they're stupid? ^ You can't respond to this btw. It's rhetorical. ^^ Yes, that was sarcasm.
- cushpush 2y agoI didn't mean to suggest "don't reply" -- I did want to start a conversation on how they are more or less the same thing. I guess I wasn't asking for clarity, right, I just wanted to point out that they're similar.
- ChrisMarshallNY 2y agoEarly in my career, one of the senior engineers in my group had a T-shirt that read: Old Age and Treachery Beats Youth and Enthusiasm, Every Time. Looks like this guy tried the “treachery” part, before he had the “old age” part down.
- ugh123 2y agoDoes the "old age" part give the ability to better cover tracks?
- Dalewyn 2y agoFor better or worse, when you have more time to learn how the real world works and make the right connections with the right people, you get much more leeway in what you can get away with. Naturally, older people had more time to do that than younger people. This is why most young people get their shins blasted while older people just get a slap on the wrist, if they're found out.
- dspillett 2y agoIt can give you the experience to know how careful you need to be in doing that, if only because you've lived long enough to see many be scuppered because of their failure to do so well enough.
- datavirtue 2y agoYou don't leave tracks.
- DrillShopper 2y agoNo but it typically gives seniority in the organization which means more credibility which can foreclose anyone looking too carefully.
- dumpsterdiver 2y agoExperience certainly does, and with enough experience comes old age.
- deleted 2y ago
- lopkeny12ko 2y agoI have read the original article as well as many pieces of additional context posted in this thread and yet still don't understand what is going on here. Yes, the intern was actively behaving maliciously, but why? What did he stand to gain from breaking another team's training code? I don't buy that he went through all that effort and espionage simply to make his own work look better. An intern is only employed for 3 months, surely sabotaging another team's multi-year project is not the most efficient way to make your toy 3-month project look better in comparison.
- arthurcolle 2y agototal lack of intellectual and scientific integrity. maybe US asset?
- 00000z 2y ago[flagged]
- vab2500 2y ago[flagged]
- 00000z 2y agothis is some CIA/FBI operations type of shit
- josephd79 2y ago[flagged]