5 ms·
The headline seems pretty aspirational. The licensing standard they're talking about will achieve nothing. Anti-bot companies selling scraping protections wil
by jsnell 1y ago
The headline seems pretty aspirational.
The licensing standard they're talking about will achieve nothing.
Anti-bot companies selling scraping protections will run out of runway: there's a limited set of signals, and none of them are robust. As the signals get used, they're also getting burned. And it's politically impossible to expand the web platform to have robust counter-abuse capabilities.
Putting the content behind a login wall can work for large sites, but not small ones.
The free-for-all will not end until adversarial scraping becomes illegal.
- carlosjobim 1y ago> Putting the content behind a login wall can work for large sites, but not small ones. Syndication is the answer. Small artists are on Spotify, small video makers are on YouTube.
- salawat 1y agoYes. Conglomeration and centralization. More, more, more! See the problem?
- carlosjobim 1y agoYou don't have to syndicate a million small creators to have a product worthwhile for consumers, it could be a thousand, a hundred, ten thousand creators in a syndicate. You can have a huge number of syndicates, which benefits creators and consumers.
- orbisvicis 1y agoBut in such an environment syndicates will have an incentive to centralize.
- carlosjobim 1y agoI don't see why. In general, there are competing syndicates and businesses of every size in most sectors of the economy.
- em-bee 1y agowhich sectors would that be? not the tech sector, not the oil sector, not the car sector. i see companies buying up properties in real state, i hear about companies buying up retirement homes (or some other kind of care facilities). retail? online retail? fast food? processed food, everywhere i see massive dominating brands. music labels? movies are consolidating in major studios. although they recently got some new players with netfix, apple and amazon. but those are still dominating companies.
- carlosjobim 1y agoIt is clear that no matter which examples I would give you, you would not acknowledge that there are any sectors with competition. Anybody can look at any sector of their interest and see that there is competition, and it is trivially easy to do so. Including in the sectors you gave as examples. If you don't believe there is competition within fast food, then please list all the fast food companies in your country below.
- em-bee 1y agoi am not denying that there is competition. the problem is that you reject that there is an incentive to centralize. if that was true, then none of the consolidations we have seen would have happened.
- carlosjobim 1y agoYes, there are incentives to centralize. But since customers are such an incredibly diverse group, it will be very difficult to make any huge centralization unless one company delivers an incredibly good product for a very good price, which also satisfies creators. And if that happens, then great.
- deleted 1y ago[deleted]
- atm3ga 1y agoAs AI companies like Perplexity introduce AI enabled browsers like Comet, they will scrape web sites through the interaction of end-users with whatever site they are using. Therefore, indeed anti-bot companies are absolutely running out of runway.
- thelittleone 1y agoWow hadn't even considered this... so say I have a members only section of my site where I share high value content, one of the members browses using Comet, and that scrapes the private content and sends to perplexity?
- lupire 1y agoThis also happens with covert botnets running secretly on user machines.
- datadrivenangel 1y agoAny user could manually download your data anyways. Access is access.
- tempodox 1y agoAnd a browser can do it automated and behind user’s back.
- kanemcgrath 1y agoNot sure if its still an issue, but companies were buying popular web extensions, then auto updating malware/spyware into them. I haven't heard much about this in a while, but I think chrome still forces auto updates for extensions, so I would expect this to be the biggest vector for scraping walled data now.
- ec109685 1y agoThe way comet browses the web is weird enough that it’s easily detectable.
- gdulli 1y agoDid you stop getting non-compliant spam when that became illegal?
- Gigachad 1y agoIt’s pretty easy. Most sites will get locked behind accounts, likely with phone number verification. Then they will be able to easily spot automated scraping.
- chrsw 1y agoEvery time I think the web is finally dead, it somehow gets deader.
- observationist 1y agoCopyright law perpetuating the institutions that are no longer providing value to the commons means copyright law has completely and utterly failed. We don't need these institutions. We don't need these publishing platforms. It's ok for them to die. They no longer provide value. Adversarial scraping is not a thing, and it can't hurt you. Fair use, however, is a thing, and what we need to be doing is totally overhauling copyright law such that it maximizes protections for individual creative types, and does away with the exploitable corporatized loopholes and bureaucracy. 99% of all sales for nearly all copyrighted products are done within the first 4 years of a work hitting the market. Give ironclad copyright to the creator for 5 years. The creator can assign their rights, explicitly, in writing, to a third party, for any particular work, or any particular fraction of their work, but each and every assignment of rights has to be explicitly documented and notarized. No more DMCA automated bullshit. The creator can submit a copyright claim. They need to provide evidence. If the evidence of wrongdoing is false, they should be fined. If a third party files a claim, they should be fined, zero exceptions, even if they have assigned rights. Artists and creators and writers should get the recognition - if someone creates a thing, they attach a name to it, and they can lease rights to corporations or the like. After 5 years, extend fair use to something liberal and generous, requiring both acknowledgments of source works and royalties, no more than 15%, paid to the creator/s. If multiple post-5 year "fair use" creators are involved, the 15% is split between them. From 5-15 years, you have to give credit and pay a fair use royalty. If you're a trillion dollar company, you're shelling out a lot of royalties. If you're an artist reusing other art, or writing fanfic for profit, or whatever, you're buying other artists a coffee in tribute. After 15 years, it becomes public domain. Anything older than 5 years becomes fair game for training AI or otherwise using in software. You set aside 15% for distribution and reimbursement once a year, and notify any creator of your use of their material. We need something sane, that scales, that doesn't hand power to corrupt cadres of lawyers and middle men who do nothing but leach from creatives and ruin innocent people's lives. AI is here to stay. Let's set up a system in which they contribute back to the commons in a significant way, that doesn't favor byzantine licensing and gatekeeping schemes designed to keep lawyers fat and happy off the efforts of people actually contributing to the common good. Let's allow the corporate media platforms and publishing outfits to die off. We have much better ways of doing things and better ways of rewarding people for their work. We don't need lawyers sucking up 80% of the profits for "facilitating deals" or whatever it is they tell themselves to sleep at night. Raze the old system and salt the ground. Simplify everything for the practical and creative people to maximize on the value all around, get people the credit and profit they deserve, and foster a vibrant public good. It doesn't need to be thousands of pages of technicalities and byzantine law and legal tradecraft. That game was built for the lawyers, and we should stop playing it.