3 ms·

The existential dread in the media is plainly apparent nowadays. I'm not saying they don't have a point (the media), or that they are publishing outright lies

The existential dread in the media is plainly apparent nowadays. I'm not saying they don't have a point (the media), or that they are publishing outright lies to debase LLMs. But something in the way the tidal wave of "LLM's are bad" articles are written just has this panicked tone of "we are so fucked" underneath them. The hostility is so plainly apparent. It's just fascinating to see it happen in real time since LLM's came seemingly out of nowhere and progressed so fast.

jaredwiener 2y ago
Because technology aside, its a new group of well funded companies who act as parasites on this already struggling industry. I know HN (and probably SV at large) has an anti-journalist bent, but ultimately by draining the individuals and companies that do the reporting, they wont have new material to scrape in the future. (Disclosure - I'm a former journalist building a way for the news industry to fight back. https://letsgo.forth.news https://letsgo.forth.news)
- mrkramer 2y ago
  News industry is struggling but scraping public web content is legal as far as it is not commercially used....at least that's my layman understanding (but Perplexity is using it commercially). The solution for news industry with regards to LLMs is licensing your news and content to LLMs products because they are going to keep scraping it anyways.
  - jaredwiener 2y ago
    "You might as well make a deal with the people stealing your content because they are going to keep stealing it anyway" is not a great option. Part of licensing is the implicit understanding that if the parties do not come to a deal, they cannot use the licensed content. It seems from this article that Perplexity is not operating in good faith, ignoring robots.txt, using unpublished IP addresses, and then lying about the amount of referral traffic generated. Not to mention, whether or not its legal (and im not a lawyer) doesn't seem to be the issue. It's being so shortsighted that you build a business that steals a product from an unwitting third party, and when that third party is drained of resources, where will you pull from?
    - mrkramer 2y ago
      >ignoring robots.txt Robots.txt is not legally binding, it is a good will among web stakeholders to respect it. But if you are repeatedly breaking it, you will have an army of web masters against you. And as far as I remember, big news outlets are already suing LLMs companies or are already licensing their content to them.
      - jaredwiener 2y ago
        Yes to all of your points, but I think you are missing my distinction between "potentially legal" and "not being terrible." And yes, of course they are suing the LLM companies. Because they shouldn't be stealing their stuff, regardless of robots.txt.
- segasaturn 2y ago
  Well, as of the last 10 years the journalism industry has just been a lazy tweet regurgitation factory. The pipeline looks something like Tweets -> Journalists -> LLMs. So maybe it would be best if we cut out the middlemen?
  - jaredwiener 2y ago
    If that were true then why not just index the tweets?
  - claudiulodro 2y ago
    To be blunt, this seems like a lazy regurgitation of a tweet. There are lots of organizations out there doing really good work. Look at a couple of the Pulitzer winners from the last couple years for example: - Mississippi Today spent more than 5 years investigating and reporting on the welfare agency and uncovered huge corruption going all the way to the governor's office: https://mississippitoday.org/2023/05/08/pulitzer-prize-winning-backchannel/ https://mississippitoday.org/2023/05/08/pulitzer-prize-winni... - Lookout Santa Cruz was on the ground reporting during catastrophic flooding this year, including setting up a text message service to get information to people because the internet was unreliable: https://lookout.co/lookout-santa-cruz-wins-pulitzer-prize-for-breaking-news-reporting/ https://lookout.co/lookout-santa-cruz-wins-pulitzer-prize-fo... This is the stuff you lose without journalism.
  - lancesells 2y ago
    LLMs are the middlemen. Search engines are the middlemen. Social networks are the middlemen. Go to the NY Times or LA Times and tell me it's based on Tweets. If you're talking about a place like The Verge which is a tech blog with journalism, they have articles based on tweets but also plenty that are legitimate, researched articles.
    - segasaturn 2y ago
      If that was true then what do journalists have to be worried about? Why would they need to "fight back" against the LLMs as the above poster said - as you know, ChatGPT can't do original research or investigations on its own. The truth is that the bulk of what is called "journalism" today isn't on-the-ground gumshoes reporting, it's rewording tweets/press releases/PR statements and maybe a couple of phone calls mixed in. A good rule of thumb is that anything that can be replaced by a current-gen AI wasn't worth much to begin with.
rsynnott 2y ago
I mean, maybe LLMs are just bad. Like, what possible use is this website? "Google search, only it might unexpectedly make stuff up" is not a thing that anyone sensible actually wants.
rchaud 2y ago
> It's just fascinating to see it happen in real time since LLM's came seemingly out of nowhere and progressed so fast. "Came out of nowhere" seems to ignore the many billions of dollars invested into it, and the millions of person-hours from people in low-income countries used to train it, and the enormous amount of energy required to keep the whole thing running. Let's not pretend that LLM advancement is the work of a few plucky upstarts that have pwned the complacent legacy media. With all that said, I seriously doubt Wired magazine holds much sway in journalistic circles anymore.
lancesells 2y ago
These parasites aren't just feeding off of news but everything published anywhere. I hope journalists continue to call them out, and even though Google is doing the same, or similar, they should mark sites like Perplexity as a link farm or duplicate content. $165M in funding and ignoring robots.txt deserves every amount of scrutiny.

jaredwiener 2y ago

Because technology aside, its a new group of well funded companies who act as parasites on this already struggling industry. I know HN (and probably SV at large) has an anti-journalist bent, but ultimately by draining the individuals and companies that do the reporting, they wont have new material to scrape in the future. (Disclosure - I'm a former journalist building a way for the news industry to fight back. https://letsgo.forth.news https://letsgo.forth.news)

mrkramer 2y ago
News industry is struggling but scraping public web content is legal as far as it is not commercially used....at least that's my layman understanding (but Perplexity is using it commercially). The solution for news industry with regards to LLMs is licensing your news and content to LLMs products because they are going to keep scraping it anyways.
- jaredwiener 2y ago
  "You might as well make a deal with the people stealing your content because they are going to keep stealing it anyway" is not a great option. Part of licensing is the implicit understanding that if the parties do not come to a deal, they cannot use the licensed content. It seems from this article that Perplexity is not operating in good faith, ignoring robots.txt, using unpublished IP addresses, and then lying about the amount of referral traffic generated. Not to mention, whether or not its legal (and im not a lawyer) doesn't seem to be the issue. It's being so shortsighted that you build a business that steals a product from an unwitting third party, and when that third party is drained of resources, where will you pull from?
  - mrkramer 2y ago
    >ignoring robots.txt Robots.txt is not legally binding, it is a good will among web stakeholders to respect it. But if you are repeatedly breaking it, you will have an army of web masters against you. And as far as I remember, big news outlets are already suing LLMs companies or are already licensing their content to them.
    - jaredwiener 2y ago
      Yes to all of your points, but I think you are missing my distinction between "potentially legal" and "not being terrible." And yes, of course they are suing the LLM companies. Because they shouldn't be stealing their stuff, regardless of robots.txt.
segasaturn 2y ago
Well, as of the last 10 years the journalism industry has just been a lazy tweet regurgitation factory. The pipeline looks something like Tweets -> Journalists -> LLMs. So maybe it would be best if we cut out the middlemen?
- jaredwiener 2y ago
  If that were true then why not just index the tweets?
- claudiulodro 2y ago
  To be blunt, this seems like a lazy regurgitation of a tweet. There are lots of organizations out there doing really good work. Look at a couple of the Pulitzer winners from the last couple years for example: - Mississippi Today spent more than 5 years investigating and reporting on the welfare agency and uncovered huge corruption going all the way to the governor's office: https://mississippitoday.org/2023/05/08/pulitzer-prize-winning-backchannel/ https://mississippitoday.org/2023/05/08/pulitzer-prize-winni... - Lookout Santa Cruz was on the ground reporting during catastrophic flooding this year, including setting up a text message service to get information to people because the internet was unreliable: https://lookout.co/lookout-santa-cruz-wins-pulitzer-prize-for-breaking-news-reporting/ https://lookout.co/lookout-santa-cruz-wins-pulitzer-prize-fo... This is the stuff you lose without journalism.
- lancesells 2y ago
  LLMs are the middlemen. Search engines are the middlemen. Social networks are the middlemen. Go to the NY Times or LA Times and tell me it's based on Tweets. If you're talking about a place like The Verge which is a tech blog with journalism, they have articles based on tweets but also plenty that are legitimate, researched articles.
  - segasaturn 2y ago
    If that was true then what do journalists have to be worried about? Why would they need to "fight back" against the LLMs as the above poster said - as you know, ChatGPT can't do original research or investigations on its own. The truth is that the bulk of what is called "journalism" today isn't on-the-ground gumshoes reporting, it's rewording tweets/press releases/PR statements and maybe a couple of phone calls mixed in. A good rule of thumb is that anything that can be replaced by a current-gen AI wasn't worth much to begin with.

rsynnott 2y ago

I mean, maybe LLMs are just bad. Like, what possible use is this website? "Google search, only it might unexpectedly make stuff up" is not a thing that anyone sensible actually wants.

rchaud 2y ago

> It's just fascinating to see it happen in real time since LLM's came seemingly out of nowhere and progressed so fast. "Came out of nowhere" seems to ignore the many billions of dollars invested into it, and the millions of person-hours from people in low-income countries used to train it, and the enormous amount of energy required to keep the whole thing running. Let's not pretend that LLM advancement is the work of a few plucky upstarts that have pwned the complacent legacy media. With all that said, I seriously doubt Wired magazine holds much sway in journalistic circles anymore.

lancesells 2y ago

These parasites aren't just feeding off of news but everything published anywhere. I hope journalists continue to call them out, and even though Google is doing the same, or similar, they should mark sites like Perplexity as a link farm or duplicate content. $165M in funding and ignoring robots.txt deserves every amount of scrutiny.