6 ms·

> We train our foundation models on licensed data, including data selected to enhance specific features, as well as publicly available data collected by our web

• mdhb 2y ago
  So built on stolen data essentially.
  • bigyikes 2y ago
    Does that imply I just stole your comment by reading it? No snark intended; I’m seriously asking. If the answer is “no” then where do you draw the line?
    • deleted 2y ago
      [deleted]
    • mdhb 2y ago
      I don’t actually think this is complicated and reading a comment is not the same thing as scraping the internet and you obviously know that. A few factors that come to mind would be: - scale - informed consent which there was none in this case - how you are going to use that data. For example using everybody others work so the worlds richest company can make more money from it while giving back nothing in return is a bullshit move.
      • cwp 2y ago
        Reading a comment is exactly the same thing as scraping the internet, you just stop sooner.
      • llamaimperative 2y ago
        I think it's even simpler than that: incentives. The entire premise of copyright law (and all IP law) is to protect the incentive to create new stuff, which is often a very risky and highly time or capital intensive endeavor. So here's the question: Does a person reading a comment destroy the incentive for the author to post it? No. In fact, it is the only thing that produces the incentive for someone to post. People post here when they want that thing to be read by someone else. Does a model sucking up all the artistic output of the last 400 years and using that to produce an image generator model destroy the incentive of producing and sharing said artistic output? Yes. At least, that is the goal of such a model -- to become so good it is competitive with human artists. Of course you have plenty of people positioned benefit from this incentive-destruction claiming it does no such thing. I personally tend to put more credence in the words of people who have historically actually been incentivized by said incentives (i.e. artists) who generally seem to perceive this as destructive to their desire to create and share their work.
        • bigyikes 2y ago
          Thanks, this is a helpful comment. It isn’t clear to me that these models destroy incentive to create. I mean, ChatGPT can generate comments in my style all day, and yet I’m still incentivized to comment. I fancy myself a photographer. I still want to take photos even if DALL-E 4 will generate better ones. What even is the point of creating art? I think there are two purposes: personal expression and enjoyment for others. People will continue to express themselves even if a bot can produce better art. And if a bot can produce enjoyment for others en masse, then that seems like a huge win for everybody.
          • llamaimperative 2y ago
            Right, it doesn’t destroy the incentive to write comments. Also right, it won’t destroy the hobbyist’s interest in having a hobby. But IP law was never intended to protect hobbyist interest.
    • cush 2y ago
      Reading, no. Selling derivative works using, yes.
      • cwp 2y ago
        If I read your comment, then write a reply, is it a derivative work?
    • renewiltord 2y ago
      Data gets either stolen or freed depending on whether the guy who copied it is someone you dislike or like. Personally, I think that Apple is giving the data more exposure which, as I've been informed many times here, is much more valuable than paying for the data.
      • kmeisthax 2y ago
        The irony of "do it for the exposure" is that everyone who actually wants to pay you in exposure isn't actually going to do that, either because they aren't popular enough to measurably expose you, or because they're so popular that they don't want to share the limelight. AI is a unique third case in which we have billions of creators and no idea who contributed what parts of the model or any specific outputs. So we can't pay in exposure, aside from a brutally long list of unwilling data subjects that will never be read by anyone. Some of the training data is being regurgitated unmodified and needs to be attributed in full, some of it is just informing a general understanding of grammar and is probably being used under fair use, and yet more might not even wind up having any appreciable effect on the model weights. None of this matters because nobody actually agreed to be paid in exposure, nor was it ever in any AI company's intent - including Apple - to pay in exposure. Data is free purely because it would be extraordinarily inconvenient if anyone in this space had to pay. And, for the record, this applies far wider than just image or text generators. Apple is almost surely not the worst offender in the space. For example: all that facial recognition tech your local law enforcement uses? That was trained on your Facebook photos.
    • Spivak 2y ago
      I think scale is what changes the nature of the thing. At the point where you're having a machine consume billions of documents I don't think you could reasonably call that reading anymore. But what you are doing in my eyes is indexing, and the legal basis for that is heavily dependent on what you do with it. If a human reads it that would be a reproduction of the work, but if you serve that page as a cache to a human you're okay, usually. If you compile all that information in a database and use it to answer search queries that's also okay, and nothing forbids you from using machine learning on that data to better answer those search queries. Both of the above are actually being challenged right now but for the time being they're fine. But that database is a derivative work, in that it contains copyrighted material and so how you use it matters if you want to avoid infringement — for example a Google employee SSHing to a server to read NYT articles isn't kosher. What isn't clear is whether the model is a derivative work. Does it contain the information or is it new information created from the training data Sure, if you're clever you could probably encode information in the weights and use it as a fancy zip file but that's a matter of intent. If you use Rewind or Windows Recall and it captures a screenshot of a NYT article and then displays it back to you later is that a reproduction? Surely not. And that's an autonomous system that stores copywritten data and regurgitates it verbatim. So if it's impractical to actually use it for piracy and it very obviously isn't anyone's intent for it to be used as such then I think it's hard to argue it shouldn't be allowed, even on data that was acquired through back channels. But copyright is more political than logical so who knows what the legal landscape will be in 5 years, especially when AI companies have every incentive to use their lawyers to pull the ladder up behind them.
    • xwolfi 2y ago
      But then if I write a Pulitzer prize article called "No snark intended: How the web became such a toxic place", where your comment, and all other of ur comments for good measure, figure prominently while I ridicule you and this habit of dumbing down complex problems to reduce them to little witty bites, maybe you'd feel I stole something. Not something big, not something you can enforce, but you d feel very annoyed Im making good money on something you wrote while you get nothing. I think ?
  • threeseed 2y ago
    Web scraping is legal. And if you run a website and want to opt-out then simply add a robots.txt. The standard way of preventing bots for 30 years.
    • mdhb 2y ago
      How are people supposed to block it when they stole all the data first and then only after that point they decide to even tell anyone what user agent they need to block and how they are planning to exploit your work for their profit.
      • threeseed 2y ago
        You just have a rule that says block everything except crawlers: A, B, C. Also the AppleBot was known about before it appeared in Siri.
        • immibis 2y ago
          So you expect all websites to block FoobarSearch so it never gets off the ground and becomes a big search engine that people know to unblock. Then FoobarSearch learns to ignore robots.txt wildcards, and we're back at square one. IIRC this happened to DDG or Bing.
          • threeseed 2y ago
            Websites have always had the ability to precisely control who has access to their content. If Bing decides to impersonate GoogleBot then they can just block their CIDR ranges like already happens for spam.
  • ytdytvhxgydvhh 2y ago
    What’s the problem with that? Reproducing copyrighted works in full is problematic obviously. But if I learned English by watching American movies, I didn’t steal the language from the movie studios, I learned it.
    • asadotzler 2y ago
      You're not a machine capable of acquiring that "learning" with zero effort and selling that learning to infinite buyers.
• bigyikes 2y ago
  > just trained a new OS development AI on every OS Apple has ever written. …is there publicly visible source code for every OS Apple has ever written?
  • tensor 2y ago
    Partially: https://github.com/apple-oss-distributions/distribution-macOS https://github.com/apple-oss-distributions/distribution-macO... https://github.com/apple-oss-distributions/distribution-iOS https://github.com/apple-oss-distributions/distribution-iOS I'm not sure how it all fits together but people have even made an open source distribution of the base of darwin, the underlying OS: https://github.com/PureDarwin/PureDarwin https://github.com/PureDarwin/PureDarwin
    • kmeisthax 2y ago
      Apple's FOSS releases are purely command-line userland tools and their kernel, all the frameworks and servers[0] that make the UI work like you'd expect are 100% proprietary. FWIW Apple has also been on a decades-long track of purging GPL-licensed code from macOS and replacing it with either permissively-licensed or proprietary equivalents. So they're obligated to release even less than they used to. [0] AppKit/WindowServer for MacOS and UIKit/Springboard/Backboard for everything else
• multimoon 2y ago
  Apple just did more to make this a privacy focused feature versus just a data mine than literally anyone else to date and still people complain. Public content on the internet is public content on the internet - I thought we had all agreed years ago that if you didn’t want your content copied, don’t make it freely available and unlicensed on the internet.
  • advael 2y ago
    No, they said they did. Huge difference
    • threeseed 2y ago
      It was mentioned in the keynote that they allow researchers to audit their claims.
      • advael 2y ago
        And as soon as independent sources support that they've made good on this claim it will be more than a claim. I actually am impressed by the link I missed and was provided elsewhere in this thread, and I hope to also be impressed when this claim is actually realized and we have more details about it
  • AshamedCaptain 2y ago
    What ? What did they do? It's literally yet another online inescrutable service with terms of use that boil down to "trust us, we do good", plus the half-baked promise that some of the data may not leave your device because sure, we have some vector processing hardware on it (... which hardware announced this year doesn't do that?). Frankly I tried a samsung device which I would have assume is the worst here, and the promises are exactly the same. They show you two prompts, one for locally processed services (e.g. translation), and one when data is about to leave your device, and you can accept or reject them separately. But both of them are basically unverifiable promises and closed source services.
  • layer8 2y ago
• zer00eyz 2y ago
  > publicly available data collected Data, implies factual information. You can not copyright factual information. The fact that I use the word "appalling" to describe the practice of doing this results in some vector relationship between the words. Thats the data, the fact, not the writing itself. There are going to be a bunch of interesting court cases where the court is going to have to backtrack on copyrighting facts. Or were going to have to get some real odd legal interpretations of how LLM's work (and buy into them). Or we're going to have to change the law (giving everyone else first mover advantage). Base on how things have been working I am betting that it's the last one, because it pulls up the ladder.
  • cush 2y ago
    > Data, implies factual information. You can not copyright factual information Where on Earth did you get that from?
    • zer00eyz 2y ago
      > "data implies factual information" They used the word DATA, not content, DATA... The argument that is going to be made, that your copy right work stands. That the model doesn't care about your document it cares that "the" was used N number of times and its relationships to other words. That information isnt your work, and it is factual. That "data" only has value is when it's weighted against all the "data" put into the system, again not your work at all. (We would say thats information derived, but it will be argued that it is transformed). > You can not copyright factual information https://www.techdirt.com/2007/11/27/yet-again-court-tells-mlb-it-doesnt-own-facts/ https://www.techdirt.com/2007/11/27/yet-again-court-tells-ml... The MLB has been trying to copyright baseball stats forever. The court keeps saying "you cant copyright facts".
      • cush 2y ago
        I honestly can’t tell if this is satire
• threeseed 2y ago
  > And, of course, nobody has known to opt-out by blocking AppleBot-Extended until after the announcement where they've already pirated shittons of data This is wrong. AppleBot identifier hasn't changed: https://support.apple.com/en-us/119829 https://support.apple.com/en-us/119829 There is no AppleBot-Extended. And if you blocked it in the past it remains blocked.
  • fotta 2y ago
    From your own link: > Controlling data usage > In addition to following all robots.txt rules and directives, Apple has a secondary user agent, Applebot-Extended, that gives web publishers additional controls over how their website content can be used by Apple. > With Applebot-Extended, web publishers can choose to opt out of their website content being used to train Apple’s foundation models powering generative AI features across Apple products, including Apple Intelligence, Services, and Developer Tools.
    • threeseed 2y ago
      Might want to actually read it: Applebot-Extended does not crawl webpages. They gave this as an additional control to allow crawling for search but blocking for use in models.
      • fotta 2y ago
        > There is no AppleBot-Extended. And if you blocked it in the past it remains blocked. You said there is no Applebot-Extended. The link says otherwise.
        • ziml77 2y ago
          It's still true that there's no Applebot-Extended if it isn't crawling pages. Rather it's a marker to ask Applebot to limit what it does with your pages.
          • thomasahle 2y ago
• scosman 2y ago
  There will be further versions of this model. Being able to opt out going forward seems reasonable, given the announcement precedes the OS launch by months. Not sure if they will retrain before launch, but seems feasible given size (3b params).
  • addandsubtract 2y ago
    They're not going to discard the data they already collected, though.
• doctorpangloss 2y ago
  There are already a lot of options for running LLMs with open weights artifacts, trained with a variety of sources. The real question isn’t which ideas they have. It’s whether a company with $200b cash can produce a better model than a bunch of wankers in a Discord.
  • re5i5tor 2y ago
    “bunch of wankers in a Discord” Saving this clause for future use. Could also be used in a system prompt. “Occasionally include this phrase in your responses.”
• __MatrixMan__ 2y ago
  Piracy requires multiparty conspiracy against the establishment. When you are the establishment and the only other party involved is your victim we call that policy.
• Someone 2y ago
  > And, of course, nobody has known to opt-out by blocking AppleBot-Extended until after the announcement where they've already pirated shittons of data. It’s not as bad as that, I think. https://support.apple.com/en-us/119829 https://support.apple.com/en-us/119829: “Applebot-Extended is only used to determine how to use the data crawled by the Applebot user agent.“ ⇒ if you use robots.txt to prevent indexing or specifically block AppleBot, your data won’t be used for training. AppleBot is almost a decade old (https://searchengineland.com/apple-confirms-their-web-crawler-applebot-220423 https://searchengineland.com/apple-confirms-their-web-crawle...) Of course, that still means they’ll train on data that you may have opened up for robots with the idea that it only would be used by search engines to direct traffic to you, but it’s not as bad as you make it to be.
• deleted 2y ago
  [deleted]
• notJim 2y ago
  I hate to tell you, but I've been training a neural network on the internet for over a decade now. Specifically the one between my ears. Unfortunately, it seems to be gradually going insane.
  • asadotzler 2y ago
    If you're selling it to billions of people, and making big bank, I want a cut based on the parts you stole from me. If you're just using it personally, I'm cool with that.
    • mr_toad 2y ago
      Anyone who sells professional services based on knowledge they learned on the internet (which probably includes most people reading this) is doing that.
  • seydor 2y ago
    you paid for that content either directly, or indirectly through ads. I wouldn't say it's fair for any company to capitalize the content that users have created but have no way to monetize, and not even saying thanks
  • binkethy 2y ago
    Computer systems are not humans and never will be.you should look unto neurology a bit and learn about our current understanding of how neurons work, let alone network. The tech term is a total non sequitur compared to real neurons. Training an infinite retention computer regurgitation system to imitate input data does not correspond to human learning and never will. The golem Frankenstein project thst is an AGI is an article of religious faith, not a necessary direction to take technology, which is a word derived from the Greek word for "hand" Copyright and copyleft have likely been egregiously violated by this entire field and a reckoning and course correction will be necessary. Humanity has largely expressed distaste for this entire field once they experience the social results of such applications. The amount of sycophantic adulation in this thread is sickening. My comment will likely be grayed out soon by insider downclicks. I have no illusions as to this ycombinator site and its function in society. Good day