6 ms·
Senators send letter questioning Mark Zuckerberg over Meta’s LLaMA leak
- brucethemoose2 3y agoThis seems stupid, because other smaller companies are releasing LLaMA-likes out into the open. But I get why Facebook kept LLaMA legally restricted while throwing it out without much enforcement: to appease inquiries like this.
- londons_explore 3y agoI don't think Facebooks moves were heavily strategically thought out. Instead, I think a small team of a few researchers made LLaMA and published a paper about it[1], and then realised to get any citations for their paper they'd need to let other researchers have the weights. Citations are like money in the academic world. Every researcher wants citations. Executives and managers care less about that stuff. [1]: https://arxiv.org/abs/2302.13971 https://arxiv.org/abs/2302.13971
- cma 3y agoYann LeCun in a podcast said it was because of uncertain copyright status on models but an expected fair use defense if distributed for research only.
- weinzierl 3y agoI'm afraid, this is it. The crackdown has started. Next thing RIAA and consorts will chime in and request a take-down.
- pclmulqdq 3y agoLetters like this make me think that Facebook may have intentionally "leaked" the weights. Certainly they didn't hesitate to open-source the model once the leak occurred.
- ed 3y agoThe model was open source before the weights leaked
- mk_stjames 3y agoThe model was open for any researcher simply by applying; you had to enter your name, university or group affiliation, write like one sentence about your prior work or publications, agree to the noncommercial research only license, and that was it. They don't say how many people got to download the weights via this method but it was probably in the thousands. And the weight files weren't watermarked/fingerprinted or anything. The hashes were all the same. Therefore no real way to trace who put up the torrents first. So this isn't much of a 'leak' as it is... just good old piracy. Like if you found a torrent of a DVD rip of the movie Frozen, the MPAA wouldn't be blaming Disney for 'leaking' it.
- tqwhite 3y agoImagine that we now live in a country when a company can be called out by Congress for allowing the public to have access to a cool new thing. As weinzieri said, "The crackdown has started." "Show me your papers."
- JumpCrisscross 3y ago> a company can be called out by Congress for allowing the public to have access to a cool new thing Facebook didn’t “allow” the public access to LLaMA. It lost control of its model weights. That difference is material. (I know, practically speaking, there were zero controls on the weights. But by their own communication, it was “leaked,” not released.)
- RobotToaster 3y agoThe result was as predictable as leaving a bucket full of candy in a room full of six year olds.
- Slava_Propanei 3y ago[dead]
- jxy 3y agoOver 300 million reside in a nation with easy access to guns, leading to corporate profits and fatalities, while legislators focus on leaked company figures.
- _boffin_ 3y agoOh... just stop with this low quality comment. They are focusing on a lot of topics and this is just one of them.
- tqwhite 3y agoLet me be clear: Congress has no business discussing this any more. If they can demand that Zuckerberg justify himself, they can do it for anyone. There is no potential crime involved in his data being leaked. It is as if Congress summoned you for having your bicycle stolen. This is another step down the authoritarian road. I despise Zuckerberg, Meta and Facebook but this is BAD.
- twoodfin 3y agoNot to be a conspiracy theorist, but members of Congress considering freely available models to be a threat but “well-regulated” models hidden away behind APIs to be acceptable is just what I’d expect from an effective lobbying campaign by OpenAI/Microsoft.
- JumpCrisscross 3y ago> but “well-regulated” models hidden away behind APIs to be acceptable This isn’t my takeaway from the letter [1]. Expressing caution towards one element of a thing doesn’t imply acceptance of or even preference for the other aspects. [1] https://www.blumenthal.senate.gov/imo/media/doc/06062023metallamamodelleakletter.pdf https://www.blumenthal.senate.gov/imo/media/doc/06062023meta...
- twoodfin 3y agoIn February 2023, Meta released LLaMA, an advanced large language model (LLM) capable of generating compelling text results, similar to products released by Google, Microsoft and OpenAI. Unlike others, Meta released LLaMA for download by approved researchers, rather than centralizing and restricting access to the underlying data… I don’t know any way to read this other than “you would have been fine by us if you kept it behind an API wall”.
- JumpCrisscross 3y agoIt’s saying Facebook did something different; the Committee is asking why. There may be nefarious intent in the subtext. We should be wary of that. But the letter per se isn’t evidence of that intent.
- generalizations 3y agoI agree that the letter did try to give some nuance, but they clearly prefer that the models be hidden away behind APIs. > At least at this stage of technology’s development, centralized AI models can be more effectively updated and controlled to prevent and respond to abuse compared to open source AI models. > While centralized models can adapt to abuse and vulnerabilities, open source AI models like LLaMA, once released to the public, will always be available to bad actors who are always willing to engage in high-risk tasks, including fraud, obscene material involving children, privacy intrusions, and other crime. > Meta’s choice to distribute LLaMA in such an unrestrained and permissive manner raises important and complicated questions about when and how it is appropriate to openly release sophisticated AI models.
- ozr 3y agoThis feels like the slowpoke meme; there are many other equally-capable foundation models in the wild now. And even if there wasn't, DJB vs US established source code as speech. I'd strongly prefer to not see the federal government investigating speech.
- r00fus 3y agoDon't most of them have llama as their origin/basis? Which ones are independent completely?
- ozr 3y agoThere are tons of models that either LoRA or finetune LLaMA, but I meant foundational models, i.e., ones completely distinct from LLaMA. That includes Falcon, MPT, StableLM, StarCoder, Replit Code, CodeGen. There's also RedPajama and OpenLLaMA, which are intended to give similar performance to LLaMA but be legally distinct.
- infamouscow 3y agoI'm confident if we replaced congress with an LLM, the LLM would do a better job.
- ZiiS 3y agoI'm confident if we replaced congress with Math.random(), the PRNG would do a better job.
- CamperBob2 3y agoNo question about it.
- bioemerl 3y agoTo those of you who live in Connecticut or Missouri, contact the shit out of these representatives and give them a piece of your mind. This sort of blatant attempt to restrict the power of individuals should not be allowed.
- dado3212 3y agoOr if you live in CA, as Alex Padilla is on the committee.
- johnwheeler 3y agoThat sounds like a shot across the Bow from open AI. here we go. It’s pure unadulterated fun to watch these tech titans foil one another’s ambitions with cheap shots and swipes. Sama comes off as a little arrogant after all and downright scary sometimes. no doubt he is brilliant.
- deleted 3y ago[deleted]
- Quillbert182 3y ago> within days of the announcement, the full model appeared on BitTorrent, making it available to anyone, anywhere in the world, without monitoring or oversight. Oh no! Anyways…
- usr1106 3y agoI see 3 options. 1. The senators have no clue. 2. They support monopolies (or at least oligopolies). That's not compatible with US legislation, is it. 3. LLMs are dangerous like atomic weapons and all proliferation needs to be universally banned.
- bioemerl 3y ago3 is not likely as llama currently can't remember more than 2000 characters. I don't doubt they may think it's the case, but as dangerous as nukes would be a wild overstatement.
- constantcrying 3y ago>That's not compatible with US legislation, is it. Lmao. The trick is to have enormous legal bariers for entry, which is also what OpenAI seems to advocate for.
- dandellion 3y agoMight not be compatible with US legislation but it's the most compatible with lining their own pockets.
- RobotToaster 3y ago>That's not compatible with US legislation, is it. Maybe not de jure, but it sure seems it de facto is.
- snapcaster 3y agopoliticians tend to like monopolies as a subset of liking centralized control over things. This means they just need to get 10 people in the room to control entire industries
- abeppu 3y agoI'm guessing at least (1) is true. Blumenthal is the senator who thought "finsta" was a product that Meta offered. Hawley in hearings has tried to get Meta to commit to not use the contents of encrypted messages for ad-targeting, apparently unaware that those contents were not readable. Being for years on a committee responsible for technology has not pushed them to develop an understanding of the domain.
- feelandcoffee 3y agoThey should ask for the testimony of that famous hacker who leaked, I think his name was 4chan /s
- moomin 3y agoI mean, if they want to start on at companies distributing cool stuff that some people like but can be dangerous, guns are right there.
- Havoc 3y agoLeak? There was a form to fill in for access lol
- say_it_as_it_is 3y agoSenators are realizing the power of AI and want to control it, bending it to their will using back-channels. OpenAI is going to have to "leak" its models as well or will become an extension of the political elites.
- dicefordeath 3y ago"Vicuna is a fine-tuned version of LLaMA that matches GPT-4 performance." Oh is that right?
- mensetmanusman 3y agoIt's so cool to think of the billions of weights/numbers that are worth $100M. Would be cool to find out where in pi they are located and just share that coordinate.
- RcouF1uZ4gsC 3y agoAm I alone in hating this pattern of of Congress trying to do an end around the first amendment freedom of speech by pressuring companies to do it for them by using hearings to harangue them. It is questionable that Congress could pass a law regulating that behavior or even if they passed it if it would be Constitutional. Yet they harangue companies for fully legal behavior. Congress is elected to pass laws, not to be moral scolds.
- brucethemoose2 3y agoLLaMA weights are uploaded everywhere (including Huggingface, which Meta themselves regularly use) and I have not seen a single enforcement action. No one uses the access request forms or the XORs. Seeing the comments here, I think HN is unaware of just how nonexistant Meta's policing of LLaMA is. They don't seem to care about their own license. Hence my theory that Meta is having their cake and eating it. Clearly they don't mind LLaMA being widely used, but the restricted release and license gives them plausible deniability from displeased parties, especially those that have no idea what huggingface-hub even is. Maybe this wasn't the initial intention, but it sure is an easy way forward.
- frankreyes 3y agoThe Internet is a Series of Tubes, 2023 edition. https://en.m.wikipedia.org/wiki/Series_of_tubes https://en.m.wikipedia.org/wiki/Series_of_tubes >"A series of tubes" is a phrase used originally as an analogy by then-United States Senator Ted Stevens (R-Alaska) to describe the Internet in the context of opposing network neutrality.