3 ms·
It would be very difficult to say. It confirms that Tristan's data is likely part of the data the models use, but a lot of filtering, pruning, and transform goe
by m00x 17d ago
It would be very difficult to say. It confirms that Tristan's data is likely part of the data the models use, but a lot of filtering, pruning, and transform goes into training.
Data has to be determined to be signal and not just noice, then it could go through processes of generating questions/answers from that data, then it RLHF's over this.
OpenAI have petabytes of data, all anonymized. It could take months to say for sure it was part of the training, and even more time to determine if it made any difference.
- Catloafdev 17d agoFrankly, I don't buy this difficulty argument. They know which model was used to come up with that particular idea. A text search over the corpus of user data used in the training set can only take so long.
- vlovich123 17d agoI think you may be underestimating how difficult a text search over their data is. They may have to build new mechanisms to do this. And what you really want is also an attribution of how much of a contribution a given corpus made which is a much harder question to answer; a single appearance of a chat probably has very little impact on the inference performance at this time unless it’s been explicitly preferenced somehow
- Catloafdev 17d agoI don't think anyone really cares about 'the measured impact the data had on the exact result' - a question which is fundamentally difficult to answer accurately in the first place - but rather whether the data was used in training at all - which as Tristan described, was extensive, beyond simply a 'single chat.' Can you explain the difficulty in engineering a search apparatus over a corpus of text data? Actually searching through it may not be easy, sure, but it's work that's doable, and creating an index is relatively trivial.
- JoshTriplett 17d ago> Can you explain the difficulty in engineering a search apparatus over a corpus of text data? My guess: "If we ever imply that's possible, people might start asking questions about all the other work we've ripped off, so the official answer is that it's impossible".
- dd8601fn 17d agoIf they literally can’t audit training data for a given model, they shouldn’t be operating.
- JumpCrisscross 17d ago> If they literally can’t audit training data for a given model, they shouldn’t be operating They can operate. They shouldn’t be claiming credit for discovering anything.
- benmathes 17d agoI worked in the tracing and tracking all the thousands of data sets that got tweaked and permuted and changed hands between thousands of researchers and data engineers at a major lab. The data that goes into training runs is permuted so much from the OG data that tracing the lineage is not trivial (dramatic understatement). And the difficulty is harder than just the extreme scale of text searching. but also explodes with organizational difficulty since there are so many people tweaking/shifting data independently upstream of the actual training run, and no they will not all add the telemetry you wish they did. In the ideal, should it be this hard? Well, no, but that's org wrangling for you.
- ashkankiani 17d agoIt feels convenient to not spend time on engineering around tooling that could be used to answer a question like “did you violate copyright by training on X?”
- deleted 17d ago[deleted]
- benmathes 17d agoDon't attribute to malice what is better explained by coordination headwinds in extremely large companies. The engineering around tooling wasn't remotely the issue. It's getting all the (thousands?) data researchers mostly iterating on fine tuning datasets that would get bristly if they couldn't work outside version control in a python notebook iteratively tweaking their dataset that processed and reprocessed a few datasets until a threshold was reached. The only _guaranteed_ chains of custody are down at the compute job and file read level. Which in a massively distributed computing job is... [redacted] nodes reading [redacted] fanouts of "datasets" that is just an abstraction over [redacted] individual files. There's no malice here. Just way way way more complex than you'd first think.
- ashkankiani 15d agoThe malice would be in not prioritizing the provenance tool at the start as a requirement of the rest of the product. Ethics would tell you that if you can't make the product in an ethical way, then you probably shouldn't make it.
- mapontosevenths 17d ago> A text search over the corpus of user data used in the training set can only take so long. Did you notice the line in the article that says the models had access to an offline copy of THE INTERNET. Like all of it.