3 ms·

Maybe the bloom filter solution is enough, but I wonder. - Paraphrasing n=7 words (and quite a few more) within a sentence can easily be fair use. - As n gets

Maybe the bloom filter solution is enough, but I wonder. - Paraphrasing n=7 words (and quite a few more) within a sentence can easily be fair use. - As n gets big, the bloom filter has to also. If/when attribution is solved for LLMs (and not fake attribution like from Bing or Perplexity) then creators can be compensated when their works are used in AI outputs. If compensation is high enough this can greatly incentivize creativity, perhaps to the point of realizing "free culture" visions from the late 90s.

visarga 3y ago
As n-gram length grows, we are still going to have the same number of ngrams, they go through a hashing function and indexed in the bloom filter as usual. The number of n-grams size n in a text is text_length - ngram_length + 1.
- groceryheist 3y ago
  The number of unique values in the bloom filter will go up ~exponentially with n. So to control the false positive rate the bloom filter has to grow.
  - visarga 3y ago
    At large enough ngram size there would be very few collisions. You can take for example this text and try in Google with quotes, it won't find anything matching exactly. I tested this 6-gram "it won't find anything matching exactly", no match. Almost anything we write has never been said exactly like that before.
    - groceryheist 3y ago
      Yes and the fact that the number of unique phrases grows so quickly with n is why the bloom filter needs to grow so that hashed n-grams don't collide.
    - dleeftink 3y ago
      > it won't find anything matching exactly This approach is probably inadequate. In my line of (NLP) research I find many things have been said exactly many, many times over. You can try this out yourself by grouping and counting strings using the many publically available Bigquery corpora for various substring lengths and offsets, e.g. [0-16]; [0-32]; [0-64] substring lengths at different offsets.
geysersam 3y ago
> if compensation is high enough Who pays the compensation? If it's the user, why wouldn't they just buy the authors work directly? Why go through the LLM middleman?
- groceryheist 3y ago
  The LLM users/middlemen pay. The user probably pays less than they would have to pay the author. The LMM provides information retrieval / discovery.
- starttoaster 3y ago
  > If it's the user, why wouldn't they just buy the authors work directly? Why go through the LLM middleman? If it's the user, why wouldn't they just buy the DVDs directly? Why go through the Netflix middleman? A retort to this would be that both NYT and ChatGPT are on the internet, so it's no added fuss of hopping in my car, driving to Walmart, and picking up a DVD case. My response to it would be that both the LLM and Netflix are content aggregators to the user. I can read the NYT, or I can read the NYT summary on ChatGPT and ask it for life advice with my pet hamster, or ask it how to reverse a linked list in bash.
sideshowb 3y ago
I like the idea but seems like there would be big problems. Like detecting if a work is reworded. Or a large number of sources have all slightly influenced a small response - isn't that pretty much considered new knowledge? Then there's the issue that however you credit attribution, it creates a game of enshittified content creation with the aim of being attributed as often as possible, regardless of whether the content really offered anything that wasn't out there already.

visarga 3y ago

As n-gram length grows, we are still going to have the same number of ngrams, they go through a hashing function and indexed in the bloom filter as usual. The number of n-grams size n in a text is text_length - ngram_length + 1.

groceryheist 3y ago
The number of unique values in the bloom filter will go up ~exponentially with n. So to control the false positive rate the bloom filter has to grow.
- visarga 3y ago
  At large enough ngram size there would be very few collisions. You can take for example this text and try in Google with quotes, it won't find anything matching exactly. I tested this 6-gram "it won't find anything matching exactly", no match. Almost anything we write has never been said exactly like that before.
  - groceryheist 3y ago
    Yes and the fact that the number of unique phrases grows so quickly with n is why the bloom filter needs to grow so that hashed n-grams don't collide.
  - dleeftink 3y ago
    > it won't find anything matching exactly This approach is probably inadequate. In my line of (NLP) research I find many things have been said exactly many, many times over. You can try this out yourself by grouping and counting strings using the many publically available Bigquery corpora for various substring lengths and offsets, e.g. [0-16]; [0-32]; [0-64] substring lengths at different offsets.

geysersam 3y ago

> if compensation is high enough Who pays the compensation? If it's the user, why wouldn't they just buy the authors work directly? Why go through the LLM middleman?

groceryheist 3y ago
The LLM users/middlemen pay. The user probably pays less than they would have to pay the author. The LMM provides information retrieval / discovery.
starttoaster 3y ago
> If it's the user, why wouldn't they just buy the authors work directly? Why go through the LLM middleman? If it's the user, why wouldn't they just buy the DVDs directly? Why go through the Netflix middleman? A retort to this would be that both NYT and ChatGPT are on the internet, so it's no added fuss of hopping in my car, driving to Walmart, and picking up a DVD case. My response to it would be that both the LLM and Netflix are content aggregators to the user. I can read the NYT, or I can read the NYT summary on ChatGPT and ask it for life advice with my pet hamster, or ask it how to reverse a linked list in bash.

sideshowb 3y ago

I like the idea but seems like there would be big problems. Like detecting if a work is reworded. Or a large number of sources have all slightly influenced a small response - isn't that pretty much considered new knowledge? Then there's the issue that however you credit attribution, it creates a game of enshittified content creation with the aim of being attributed as often as possible, regardless of whether the content really offered anything that wasn't out there already.