12 ms·

I was served with papers yesterday from the Silverman folks. No idea what they could possibly want, since books3 is open source. https://twitter.com/theshawwn/s

I was served with papers yesterday from the Silverman folks. No idea what they could possibly want, since books3 is open source. https://twitter.com/theshawwn/status/1704559992135717238?s=46&t=CHNZIj4cQ8XNT6qqZQipWQ https://twitter.com/theshawwn/status/1704559992135717238?s=4... I think we’re witnessing the death of open source AI. The logical outcome of this is that only large companies will be able to acquire and use the training data necessary to compete with ChatGPT. So anyone who thinks otherwise will have to answer the question: how are we going to make any datasets? It’s tempting to think that we can pull together data out of non copyrighted work. But there isn’t enough data. That would mean the model has no knowledge about any books.

ozr 3y ago
Reminds me of when the RIAA and MPAA were able to end piracy. Dark days.
- fallat 3y ago
  They were? how
  - jazzyjackson 3y ago
    Definitely a joke, but also maybe not a joke. Enough pressure was put on pirates through ISPs threatening to drop customers (or was it to sue them?) that streaming services saw greater adoption. People are risk averse when it comes to lawsuits from publishers I suppose. So you can still pirate, but most people don't. Maybe LLMs will be the same, sure you could use an uncensored/unlicensed model that doesn't pay royalties to publishers, but why risk it?
    - brucethemoose2 3y ago
      It's not quite the same... maybe even reversed. Big companies can "secretly" use private datasets with copyrighted or legally questionable data with a wink and a nod, without anyone being able to prove it in court. Open source models generally do not have that option. Maybe there are some exceptions like Meta using some private data in future Llamas, but I question how sustainable those efforts are.
    - csomar 3y ago
      Streaming services got higher adoption because the price/trouble cost was worth it. People saw value in putting $7/month into a service that delivers high quality video. This might not last with the fragmentation of streaming services (I don't think people are ready to pay $10x10 for access to all services).
      - ghaff 3y ago
        Part of the trouble side though is taking the proper precautions to not trigger a strike from your ISP or a letter from RIAA. And a lot of people didn’t/don’t understand what those precautions are. Don’t know the size of the effect but there was probably some.
    - somenameforme 3y ago
      As peer comments mentioned, that's completely missing the order of causation. Steam is a better example, because there was never any weird organization threatening everybody they could send a letter to with lawsuits. Yet it grew exponentially. I think, in general, there are relatively few completely ideological 'pirates.' People will buy things when they can easily afford it, and it's easy to do so. When it's too expensive, not locally available, or when the payment methods involve hassles, they will simply download it instead.
    - 29athrowaway 3y ago
      Most piracy is P2P and requires some "computer literacy" to use. Most streaming, AFAIK, is not P2P. Streaming gives you instant access to the content, and streaming clients are often simpler to use, with features such as recommender systems, etc.
jazzyjackson 3y ago
It will take an act of congress or the supreme court to decide the matter one way or another. But copyright is not universal law. Japan, for one, has thrown the fight - they stand to gain more by harboring new technology than ensuring royalties are paid to publishers of the last century. https://web.archive.org/web/20230607091817/https://technomancers.ai/japan-goes-all-in-copyright-doesnt-apply-to-ai-training/ https://web.archive.org/web/20230607091817/https://technoman... discussion: https://news.ycombinator.com/item?id=36144241 https://news.ycombinator.com/item?id=36144241
- thelittleone 3y ago
  Since the government in many ways views citizens as threats to their conntrol / power, the last thing they want is that population armed with a powerful AI.
  - thisiswater 3y ago
    I agree - LLMs are a massive threat to the current state of government power. "<LLM>, analyze the contents of this 500 page bill. Who stands to gain from this bill, and what outcomes is it likely to have for the general public? Is this bill in line with good faith evidence-based policymaking for the good of the population and the planet? What existing legal mechanisms could be used to fight the special-interest aspects of this bill?"
    - pavlov 3y ago
      Is there really a shortage of this kind of analysis today? Using those legal mechanisms requires money and often public support. The LLM can’t conjure either. The world is already full of smart people and good ideas about policy. The reason they’re not getting implemented probably has little to do with things that AI can solve today. For starters, a lot of voters actively dislike policy suggestions from experts and choose politicians who proudly go against expert opinion. Giving AI tools to the experts won’t change that.
      - deleted 3y ago
lithiumii 3y ago
How about stay anonymous and just violate all the copyright laws? There's already pirate bay, libgen, sci-hub, zlibrary, etc., surely it's possible for there to be an opensource & pirate LLM model.
- CamperBob2 3y ago
  If it were practical to mirror sci-hub and libgen, that would be one thing, but despite a lot of talk online I have yet to see a practical way to put my hands on such a thing.
  - vlabakje90 3y ago
    I'm not sure what you are referring to with 'such a thing', but mirroring libgen and zlib is really not hard. Libgen offers Torrent links as does Anna's Archive. The libgen domains are fragile, but here's a link to the Anna's Archive torrents: https://annas-archive.org/torrents https://annas-archive.org/torrents. They even have a page talking about training LLMs on this data: https://annas-archive.org/llm https://annas-archive.org/llm
    - roel_v 3y ago
      Do any of these methods actually work though? Last time I looked (admittedly, 6 months or so ago), there were 0 seeders on the torrents.
      - CamperBob2 3y ago
        Exactly, it's easy to say "Just torrent it," but that requires a lot of people to stick their necks out, including the user who just wants a copy of the data. We need the ability to circulate HDDs physically in a semi-organized fashion, samizdat-style.
  - input_sh 3y ago
    Mirroring libgen is definitely within reach, it's "just" 50 or so terabytes with torrents freely available for bulk downloading. Realistically only maybe 10% of that is actually useful, but reaching that 10% is gonna be very labour-intensive. You would have to do a lot of cleanup of different formats, duplicate uploads, different editions of the same book, scanned PDFs, and what not, while big players with their own ebook stores (Amazon, Google, Apple, any ebook store) already have all of the proper metadata, a common format to work with, and a lot less duplicates.
debaserab2 3y ago
Wow. Did you figure out what your next steps are? (understood if you don't want to discuss)
- sillysaurusx 3y ago
  I spoke with a lawyer earlier tonight. Basically, there are no next steps for me. The next steps for them are to send targeted discovery requests for whatever it is they think I might have in relation to the case, and possibly ask me to testify in a deposition.
  - deleted 3y ago
    [deleted]
mchiang 3y ago
Take a look at Databricks' getting a hand from internal team to create the dolly 15k dataset. ( https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm https://www.databricks.com/blog/2023/04/12/dolly-first-open-... ) For training AGI (artificial general intelligence) maybe only a select few mega companies with massive datasets will be able to come up with training data. There are so many other use cases that OSS projects can enable otherwise. Individuals or smaller companies have unique data that can be used to augment existing open source models. Many use cases are area specific, and without the need for general intelligence. Palantir just did a talk at the AIPCon ( https://youtu.be/o2b0DwNg6Ko https://youtu.be/o2b0DwNg6Ko ), where they recommended the use of many LLMs, open and closed. ( the example had Llama 2 70B, GPT4, Palm coding, claude, + fine-tuned models) feeding into their synthesizer. While I want open source to win, especially as an open source maintainer on Ollama (if you haven't seen it yet, it's one of the easiest ways to run LLMs locally - https://github.com/jmorganca/ollama https://github.com/jmorganca/ollama ), I think the work so far in this space has been a positive sum one - open or closed.
- ca_tech 3y ago
  People may hate on Palantir but sounds like they get it. I agree that the next step is going to be the interaction of multiple models. We have popular models that have been generalized across domains. The next step is models for specific domains, specific enterprise data sources, individual data sets. Each model has the opportunity to leverage the more generalized model above it. And if done properly, the models will be able to then build back references showing what data validates its output as a source of truth.
anon373839 3y ago
This is just a standard demand for preservation of evidence. They’re letting you know that although you’re not a party to the lawsuit, you may be called to produce information as a witness. They are informing you that certain classes of information are relevant to the litigation, and that you have a legal duty to preserve that information in case it is later subpoenaed. Caveat: I am not your lawyer.
- contrarian1234 3y ago
  > Caveat: I am not your lawyer ??? Why would he suddenly think you are?
  - mixedCase 3y ago
    Because they were giving advice about a legal matter, so by stating what seems obvious to you and me, the GP is covering themselves from "implied" lawyer-client relationships which under some jurisdictions could adjudicate them the responsibilities of a lawyer to a client if a person could prove in court that they reasonably believed they were acting as their lawyer. I don't know if this is actually been a problem ever brought to a judge to be honest, but that's the reason why that disclaimer is made and of course, I am not your lawyer.
    - contrarian1234 3y ago
      I want to find out where lawyers hang out online and if they close ever sentence/post with "I am not your lawyer"
      - cyrillite 3y ago
        Check Legal Advice subreddits, it happens. It may be an abundance of caution most of the time, but it doesn’t hurt.
        ghaff 3y ago
        It’s pretty common in situations where you could be seen as giving legal, accounting, financial, etc. advice especially if you have some certification to that effect to disclaim what you’re saying as actually being advice. Lawyers are especially careful about this outside of a specific client relationship.
caesil 3y ago
Sarah Silverman is almost certainly going to be crushed in court. The lawsuit is totally without merit.
pmarreck 3y ago
> I think we’re witnessing the death of open source AI. Nah, we aren't. You can't make any money off an open-source project via litigation, and torrents exist. This baseless fear-driven nonsense will eventually blow over.
3abiton 3y ago
Books3 and The Pile are still available on the internet, just not "legally".
ie9rifikr 3y ago
Isn't there sort of an implication here that Criminally Open-Source ai will win?
- sillysaurusx 3y ago
  Who will give them the compute? It requires million dollar clusters to make any progress.
  - ie9rifikr 3y ago
    Willingly? I wouldn't know. But then it becomespre a problem of obfuscation doesn't it?
  - lucubratory 3y ago
    There's a huge amount of money in AI, and there is no global organisation tracking GPUs and making sure that "X startup which purchased a bunch of GPUs then went bankrupt and sold them all to an unknown buyer" doesn't happen. Even stopping China from getting GPUs is proving extremely difficult and that's much, much more powerful actors than even the largest copyright cartel; NVIDIA is chafing even against that. GPU restrictions aren't going to work unless they're being enforced by the US military, and even then China is only ~5-10 years behind the US in fab tech and could easily custom build chips for AI to fill the demand, in the internal market if nothing else. Basically, there is a lot of money to be made in AI, and if money is available to fund development it will not be that difficult to turn that money into compute, particularly in a context where the current huge "legitimate" buyers like Microsoft, Amazon, Google are banned from using AI chips for AI.
ghaff 3y ago
The copyright issue is a big one and certainly unsettled but the (very good) IP lawyers I know are pretty much universally of the opinion that training models with copyrighted material probably passes muster for a number of reasons. Of course, as someone else noted, this is probably ultimately going to be in the US a question for Congress and the courts.

ozr 3y ago

Reminds me of when the RIAA and MPAA were able to end piracy. Dark days.

fallat 3y ago
They were? how
- jazzyjackson 3y ago
  Definitely a joke, but also maybe not a joke. Enough pressure was put on pirates through ISPs threatening to drop customers (or was it to sue them?) that streaming services saw greater adoption. People are risk averse when it comes to lawsuits from publishers I suppose. So you can still pirate, but most people don't. Maybe LLMs will be the same, sure you could use an uncensored/unlicensed model that doesn't pay royalties to publishers, but why risk it?
  - brucethemoose2 3y ago
    It's not quite the same... maybe even reversed. Big companies can "secretly" use private datasets with copyrighted or legally questionable data with a wink and a nod, without anyone being able to prove it in court. Open source models generally do not have that option. Maybe there are some exceptions like Meta using some private data in future Llamas, but I question how sustainable those efforts are.
  - csomar 3y ago
    Streaming services got higher adoption because the price/trouble cost was worth it. People saw value in putting $7/month into a service that delivers high quality video. This might not last with the fragmentation of streaming services (I don't think people are ready to pay $10x10 for access to all services).
    - ghaff 3y ago
      Part of the trouble side though is taking the proper precautions to not trigger a strike from your ISP or a letter from RIAA. And a lot of people didn’t/don’t understand what those precautions are. Don’t know the size of the effect but there was probably some.
  - somenameforme 3y ago
    As peer comments mentioned, that's completely missing the order of causation. Steam is a better example, because there was never any weird organization threatening everybody they could send a letter to with lawsuits. Yet it grew exponentially. I think, in general, there are relatively few completely ideological 'pirates.' People will buy things when they can easily afford it, and it's easy to do so. When it's too expensive, not locally available, or when the payment methods involve hassles, they will simply download it instead.
  - 29athrowaway 3y ago
    Most piracy is P2P and requires some "computer literacy" to use. Most streaming, AFAIK, is not P2P. Streaming gives you instant access to the content, and streaming clients are often simpler to use, with features such as recommender systems, etc.

jazzyjackson 3y ago

It will take an act of congress or the supreme court to decide the matter one way or another. But copyright is not universal law. Japan, for one, has thrown the fight - they stand to gain more by harboring new technology than ensuring royalties are paid to publishers of the last century. https://web.archive.org/web/20230607091817/https://technomancers.ai/japan-goes-all-in-copyright-doesnt-apply-to-ai-training/ https://web.archive.org/web/20230607091817/https://technoman... discussion: https://news.ycombinator.com/item?id=36144241 https://news.ycombinator.com/item?id=36144241

thelittleone 3y ago
Since the government in many ways views citizens as threats to their conntrol / power, the last thing they want is that population armed with a powerful AI.
- thisiswater 3y ago
  I agree - LLMs are a massive threat to the current state of government power. "<LLM>, analyze the contents of this 500 page bill. Who stands to gain from this bill, and what outcomes is it likely to have for the general public? Is this bill in line with good faith evidence-based policymaking for the good of the population and the planet? What existing legal mechanisms could be used to fight the special-interest aspects of this bill?"
  - pavlov 3y ago
    Is there really a shortage of this kind of analysis today? Using those legal mechanisms requires money and often public support. The LLM can’t conjure either. The world is already full of smart people and good ideas about policy. The reason they’re not getting implemented probably has little to do with things that AI can solve today. For starters, a lot of voters actively dislike policy suggestions from experts and choose politicians who proudly go against expert opinion. Giving AI tools to the experts won’t change that.
    - deleted 3y ago

lithiumii 3y ago

How about stay anonymous and just violate all the copyright laws? There's already pirate bay, libgen, sci-hub, zlibrary, etc., surely it's possible for there to be an opensource & pirate LLM model.

CamperBob2 3y ago
If it were practical to mirror sci-hub and libgen, that would be one thing, but despite a lot of talk online I have yet to see a practical way to put my hands on such a thing.
- vlabakje90 3y ago
  I'm not sure what you are referring to with 'such a thing', but mirroring libgen and zlib is really not hard. Libgen offers Torrent links as does Anna's Archive. The libgen domains are fragile, but here's a link to the Anna's Archive torrents: https://annas-archive.org/torrents https://annas-archive.org/torrents. They even have a page talking about training LLMs on this data: https://annas-archive.org/llm https://annas-archive.org/llm
  - roel_v 3y ago
    Do any of these methods actually work though? Last time I looked (admittedly, 6 months or so ago), there were 0 seeders on the torrents.
    - CamperBob2 3y ago
      Exactly, it's easy to say "Just torrent it," but that requires a lot of people to stick their necks out, including the user who just wants a copy of the data. We need the ability to circulate HDDs physically in a semi-organized fashion, samizdat-style.
- input_sh 3y ago
  Mirroring libgen is definitely within reach, it's "just" 50 or so terabytes with torrents freely available for bulk downloading. Realistically only maybe 10% of that is actually useful, but reaching that 10% is gonna be very labour-intensive. You would have to do a lot of cleanup of different formats, duplicate uploads, different editions of the same book, scanned PDFs, and what not, while big players with their own ebook stores (Amazon, Google, Apple, any ebook store) already have all of the proper metadata, a common format to work with, and a lot less duplicates.

debaserab2 3y ago

Wow. Did you figure out what your next steps are? (understood if you don't want to discuss)

sillysaurusx 3y ago
I spoke with a lawyer earlier tonight. Basically, there are no next steps for me. The next steps for them are to send targeted discovery requests for whatever it is they think I might have in relation to the case, and possibly ask me to testify in a deposition.
- deleted 3y ago
  [deleted]

mchiang 3y ago

Take a look at Databricks' getting a hand from internal team to create the dolly 15k dataset. ( https://www.databricks.com/blog/2023/04/12/dolly-first-open-commercially-viable-instruction-tuned-llm https://www.databricks.com/blog/2023/04/12/dolly-first-open-... ) For training AGI (artificial general intelligence) maybe only a select few mega companies with massive datasets will be able to come up with training data. There are so many other use cases that OSS projects can enable otherwise. Individuals or smaller companies have unique data that can be used to augment existing open source models. Many use cases are area specific, and without the need for general intelligence. Palantir just did a talk at the AIPCon ( https://youtu.be/o2b0DwNg6Ko https://youtu.be/o2b0DwNg6Ko ), where they recommended the use of many LLMs, open and closed. ( the example had Llama 2 70B, GPT4, Palm coding, claude, + fine-tuned models) feeding into their synthesizer. While I want open source to win, especially as an open source maintainer on Ollama (if you haven't seen it yet, it's one of the easiest ways to run LLMs locally - https://github.com/jmorganca/ollama https://github.com/jmorganca/ollama ), I think the work so far in this space has been a positive sum one - open or closed.

ca_tech 3y ago
People may hate on Palantir but sounds like they get it. I agree that the next step is going to be the interaction of multiple models. We have popular models that have been generalized across domains. The next step is models for specific domains, specific enterprise data sources, individual data sets. Each model has the opportunity to leverage the more generalized model above it. And if done properly, the models will be able to then build back references showing what data validates its output as a source of truth.

anon373839 3y ago

This is just a standard demand for preservation of evidence. They’re letting you know that although you’re not a party to the lawsuit, you may be called to produce information as a witness. They are informing you that certain classes of information are relevant to the litigation, and that you have a legal duty to preserve that information in case it is later subpoenaed. Caveat: I am not your lawyer.

contrarian1234 3y ago
> Caveat: I am not your lawyer ??? Why would he suddenly think you are?
- mixedCase 3y ago
  Because they were giving advice about a legal matter, so by stating what seems obvious to you and me, the GP is covering themselves from "implied" lawyer-client relationships which under some jurisdictions could adjudicate them the responsibilities of a lawyer to a client if a person could prove in court that they reasonably believed they were acting as their lawyer. I don't know if this is actually been a problem ever brought to a judge to be honest, but that's the reason why that disclaimer is made and of course, I am not your lawyer.
  - contrarian1234 3y ago
    I want to find out where lawyers hang out online and if they close ever sentence/post with "I am not your lawyer"
    - cyrillite 3y ago
      Check Legal Advice subreddits, it happens. It may be an abundance of caution most of the time, but it doesn’t hurt.
      - ghaff 3y ago
        It’s pretty common in situations where you could be seen as giving legal, accounting, financial, etc. advice especially if you have some certification to that effect to disclaim what you’re saying as actually being advice. Lawyers are especially careful about this outside of a specific client relationship.

caesil 3y ago

Sarah Silverman is almost certainly going to be crushed in court. The lawsuit is totally without merit.

pmarreck 3y ago

> I think we’re witnessing the death of open source AI. Nah, we aren't. You can't make any money off an open-source project via litigation, and torrents exist. This baseless fear-driven nonsense will eventually blow over.

3abiton 3y ago

Books3 and The Pile are still available on the internet, just not "legally".

ie9rifikr 3y ago

Isn't there sort of an implication here that Criminally Open-Source ai will win?

sillysaurusx 3y ago
Who will give them the compute? It requires million dollar clusters to make any progress.
- ie9rifikr 3y ago
  Willingly? I wouldn't know. But then it becomespre a problem of obfuscation doesn't it?
- lucubratory 3y ago
  There's a huge amount of money in AI, and there is no global organisation tracking GPUs and making sure that "X startup which purchased a bunch of GPUs then went bankrupt and sold them all to an unknown buyer" doesn't happen. Even stopping China from getting GPUs is proving extremely difficult and that's much, much more powerful actors than even the largest copyright cartel; NVIDIA is chafing even against that. GPU restrictions aren't going to work unless they're being enforced by the US military, and even then China is only ~5-10 years behind the US in fab tech and could easily custom build chips for AI to fill the demand, in the internal market if nothing else. Basically, there is a lot of money to be made in AI, and if money is available to fund development it will not be that difficult to turn that money into compute, particularly in a context where the current huge "legitimate" buyers like Microsoft, Amazon, Google are banned from using AI chips for AI.

ghaff 3y ago

The copyright issue is a big one and certainly unsettled but the (very good) IP lawyers I know are pretty much universally of the opinion that training models with copyrighted material probably passes muster for a number of reasons. Of course, as someone else noted, this is probably ultimately going to be in the US a question for Congress and the courts.