4 ms·

I agree with that, but I also think that they should at least identify themselves instead of using a generic user agent.

by KomoD 2y ago

BriggyDwiggs42 2y ago
I’d rather share less information than more to any site I visit. Why does a user want to share that info?
- KomoD 2y ago
  What, users won't share anything? I said I wanted Perplexity to identify themselves in the user agent instead of using the generic "Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.3" they're using right now for the "non-scraper bot". How does that impact users at all?
  - TeMPOraL 2y ago
    I don't, because if it will, then someone like the author of the article will do the obnoxious thing and ban it. We've been there before, 30 years ago. That's why all browsers' user agent strings start with "Mozilla".
    - sensanaty 2y ago
      Why is the author here obnoxious, and not Perplexity? I don't want these scumbag AI companies making money off me, end of story.
      - TeMPOraL 2y ago
        The "scumbag AI company" in question is making money by offering me a way to access information while skipping any and all attention economy bullshit you may have on your site, on top of being just plain more convenient. Note that the author is confusing crawling (which is done with documented User Agent and presumably obeys robots.txt) with browsing (which is done by working as one-off user agent for the user). As for why this behavior is obnoxious, I refer you to 30 years worth of arguing on this, as it's been discussed ever since User-Agent header was first added, and then used by someone to discriminate visitors based on their browsers.
        halostatue 2y ago
        If you want summaries from my website, go to my website. I want a way to deny any licence to any third-party user agent that will apply machine learning on my content, whether you initiated the request or not. LLMs — and more importantly the companies that train and operate them — should not be trusted at all, especially for so-called "summarization": https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actually-does-nothing-of-the-kind/ https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actu... While Perplexity may be operating against a particular URL based on a direct request from you, they are acting improperly when they "summarize" a website as they have an implicit (and sometimes explicit if there's a paywall) licence to read and render the content as provided, but not to process and redistribute such content. There needs to be something stronger than robots.txt, where I can specify the uses permitted by indirect user access (in my case, search indexing would be the only permitted use case; no LLM training, no LLM summarization, no proxying, no "sanitization" by parental proxies, etc.).
        TeMPOraL 2y ago
        > If you want summaries from my website, go to my website. I will. Through Perplexity. My lifespan is limited, and I have better ways to spend it than digging out information while you make a buck from making me miserable (otherwise there isn't much reason to complain, other than some anti-AI ideology stance). > I want a way to deny any licence to any third-party user agent that will apply machine learning on my content, whether you initiated the request or not. That's not how the Internet works. Allowing for that would mean killing user-generated content sites, optimizing proxies, corporate proxies, online viewers and editors, caches, possibly desktop software too. Also, my browser probably already does some ML on the side anyway. You'd catch a lot of regular browsing this way. Ultimately, the rules of the road are what they always have been: whatever your publicly accessible web server spouts out on a request is fair game for the requester to consume however they like, in part or entirely. If you want to limit access for particular tools or people, put up a goddamn paywall. All the noise about scrapping and stuff is attention economy players trying to have their cake and eat it too. As the user in - i.e. the victim of - attention economy, I don't feel much sympathy for that plight. Also: > LLMs — and more importantly the companies that train and operate them — should not be trusted at all, especially for so-called "summarization" That's not your problem. That's my problem. If I use a shitty tool from questionable vendor to parse your content, that's on me. You should not care. In fact, being too interested in what I use for my Internet consumption can be seen as surveillance, which is not nice.
beefnugs 2y ago
I want all people in the world with a dirty arse to change their user agent so i can not serve my website to dirty arses.

I agree with that, but I also think that they should at least identify themselves instead of using a generic user agent.

by KomoD 2y ago

I agree with that, but I also think that they should at least identify themselves instead of using a generic user agent.

BriggyDwiggs42 2y ago

I’d rather share less information than more to any site I visit. Why does a user want to share that info?

KomoD 2y ago
What, users won't share anything? I said I wanted Perplexity to identify themselves in the user agent instead of using the generic "Mozilla/5.0 (Windows NT 10.0) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/111.0.0.0 Safari/537.3" they're using right now for the "non-scraper bot". How does that impact users at all?
- TeMPOraL 2y ago
  I don't, because if it will, then someone like the author of the article will do the obnoxious thing and ban it. We've been there before, 30 years ago. That's why all browsers' user agent strings start with "Mozilla".
  - sensanaty 2y ago
    Why is the author here obnoxious, and not Perplexity? I don't want these scumbag AI companies making money off me, end of story.
    - TeMPOraL 2y ago
      The "scumbag AI company" in question is making money by offering me a way to access information while skipping any and all attention economy bullshit you may have on your site, on top of being just plain more convenient. Note that the author is confusing crawling (which is done with documented User Agent and presumably obeys robots.txt) with browsing (which is done by working as one-off user agent for the user). As for why this behavior is obnoxious, I refer you to 30 years worth of arguing on this, as it's been discussed ever since User-Agent header was first added, and then used by someone to discriminate visitors based on their browsers.
      - halostatue 2y ago
        If you want summaries from my website, go to my website. I want a way to deny any licence to any third-party user agent that will apply machine learning on my content, whether you initiated the request or not. LLMs — and more importantly the companies that train and operate them — should not be trusted at all, especially for so-called "summarization": https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actually-does-nothing-of-the-kind/ https://ea.rna.nl/2024/05/27/when-chatgpt-summarises-it-actu... While Perplexity may be operating against a particular URL based on a direct request from you, they are acting improperly when they "summarize" a website as they have an implicit (and sometimes explicit if there's a paywall) licence to read and render the content as provided, but not to process and redistribute such content. There needs to be something stronger than robots.txt, where I can specify the uses permitted by indirect user access (in my case, search indexing would be the only permitted use case; no LLM training, no LLM summarization, no proxying, no "sanitization" by parental proxies, etc.).
        TeMPOraL 2y ago
        > If you want summaries from my website, go to my website. I will. Through Perplexity. My lifespan is limited, and I have better ways to spend it than digging out information while you make a buck from making me miserable (otherwise there isn't much reason to complain, other than some anti-AI ideology stance). > I want a way to deny any licence to any third-party user agent that will apply machine learning on my content, whether you initiated the request or not. That's not how the Internet works. Allowing for that would mean killing user-generated content sites, optimizing proxies, corporate proxies, online viewers and editors, caches, possibly desktop software too. Also, my browser probably already does some ML on the side anyway. You'd catch a lot of regular browsing this way. Ultimately, the rules of the road are what they always have been: whatever your publicly accessible web server spouts out on a request is fair game for the requester to consume however they like, in part or entirely. If you want to limit access for particular tools or people, put up a goddamn paywall. All the noise about scrapping and stuff is attention economy players trying to have their cake and eat it too. As the user in - i.e. the victim of - attention economy, I don't feel much sympathy for that plight. Also: > LLMs — and more importantly the companies that train and operate them — should not be trusted at all, especially for so-called "summarization" That's not your problem. That's my problem. If I use a shitty tool from questionable vendor to parse your content, that's on me. You should not care. In fact, being too interested in what I use for my Internet consumption can be seen as surveillance, which is not nice.

beefnugs 2y ago

I want all people in the world with a dirty arse to change their user agent so i can not serve my website to dirty arses.