4 ms·

How a user views a page isn't the same as a startup scraping the internet wholesale for financial gain.

by cdme 2y ago

ulrikrasmussen 2y ago
But it's not scraping, it's retrieving the page on request from the user.
- cdme 2y ago
  With no benefit provided to the creator — they're not directing users out, they're pulling data in.
  - deleted 2y ago
    [deleted]
  - threecheese 2y ago
    They are directing users __in__ in some cases though, no? I’m a perplexity user, and their summaries are often way off which drives me to the references (attribution). The ratio of fetches to clickthroughs is what’s important now though; this new model (which we’ve not negotiated or really asked for) is driving that upward from 1, and not only are you paying more as a provider but your consumer is paying more ($ to perplexity and/or via ad backend) and you aren’t seeing any of it. And you pay those extra costs to indirectly finance the competitor who put you in this situation, who intends to drive that ratio as high as it can in order to get more money from more of your customers tomorrow. Yay.
- JumpCrisscross 2y ago
  > it's not scraping, it's retrieving the page on request from the user Search engines already tried it. It’s not retrieving on request because the user didn’t request the page, they requested a bot find specific content on any page.
  - alexey-salmin 2y ago
    But it's not what happened here. It WAS retrieving on request. > I went into Perplexity and asked "What's on this page rknight.me/PerplexityBot?". Immediately I could see the log and just like Lewis, the user agent didn't include their custom user agent
    - JumpCrisscross 2y ago
      That was to test the user-agent hiding. The broader problem—Perplexity laundering attribution—is where the scraping vs retrieval question comes into play.
      - Dylan16807 2y ago
        Well the example in the post doesn't show any laundering. Do you have an example of it? Unless you mean the entire concept of training launders attribution, but that's basically unrelated to this post and the complaints inside it.
    - threecheese 2y ago
      In this case you are 100% correct, but I think it’s reasonable to assume that the “read me this web page” use case constitutes a small minority of perplexity’s fetches. I find it useful because of the attribution - more so its references - which I almost always navigate to because its summaries are frequently crap.
  - SamBam 2y ago
    This is why this conversation is making me insane. How are people saying straight-faced that the user is requesting a specific page? They aren't, they're doing a search of the web. That's not at all the same as a browser visiting a page.
    - lolinder 2y ago
      Because that's literally what the author does in TFA and then complains about when Perplexity complies. > What is this post about https://rknight.me/blog/blocking-bots-with-nginx/ https://rknight.me/blog/blocking-bots-with-nginx/
gruez 2y ago
That's not a relevant factor in most legal regimes. At best it's a moral argument.

How a user views a page isn't the same as a startup scraping the internet wholesale for financial gain.

by cdme 2y ago

How a user views a page isn't the same as a startup scraping the internet wholesale for financial gain.

ulrikrasmussen 2y ago

But it's not scraping, it's retrieving the page on request from the user.

cdme 2y ago
With no benefit provided to the creator — they're not directing users out, they're pulling data in.
- deleted 2y ago
  [deleted]
- threecheese 2y ago
  They are directing users __in__ in some cases though, no? I’m a perplexity user, and their summaries are often way off which drives me to the references (attribution). The ratio of fetches to clickthroughs is what’s important now though; this new model (which we’ve not negotiated or really asked for) is driving that upward from 1, and not only are you paying more as a provider but your consumer is paying more ($ to perplexity and/or via ad backend) and you aren’t seeing any of it. And you pay those extra costs to indirectly finance the competitor who put you in this situation, who intends to drive that ratio as high as it can in order to get more money from more of your customers tomorrow. Yay.
JumpCrisscross 2y ago
> it's not scraping, it's retrieving the page on request from the user Search engines already tried it. It’s not retrieving on request because the user didn’t request the page, they requested a bot find specific content on any page.
- alexey-salmin 2y ago
  But it's not what happened here. It WAS retrieving on request. > I went into Perplexity and asked "What's on this page rknight.me/PerplexityBot?". Immediately I could see the log and just like Lewis, the user agent didn't include their custom user agent
  - JumpCrisscross 2y ago
    That was to test the user-agent hiding. The broader problem—Perplexity laundering attribution—is where the scraping vs retrieval question comes into play.
    - Dylan16807 2y ago
      Well the example in the post doesn't show any laundering. Do you have an example of it? Unless you mean the entire concept of training launders attribution, but that's basically unrelated to this post and the complaints inside it.
  - threecheese 2y ago
    In this case you are 100% correct, but I think it’s reasonable to assume that the “read me this web page” use case constitutes a small minority of perplexity’s fetches. I find it useful because of the attribution - more so its references - which I almost always navigate to because its summaries are frequently crap.
- SamBam 2y ago
  This is why this conversation is making me insane. How are people saying straight-faced that the user is requesting a specific page? They aren't, they're doing a search of the web. That's not at all the same as a browser visiting a page.
  - lolinder 2y ago
    Because that's literally what the author does in TFA and then complains about when Perplexity complies. > What is this post about https://rknight.me/blog/blocking-bots-with-nginx/ https://rknight.me/blog/blocking-bots-with-nginx/

gruez 2y ago

That's not a relevant factor in most legal regimes. At best it's a moral argument.