4 ms·
If search engines hadn't started scraping the web back in the 90s, imagine the outcry if you tried to start a search engine today. You would instantly be sued i
by modeless 7y ago
If search engines hadn't started scraping the web back in the 90s, imagine the outcry if you tried to start a search engine today. You would instantly be sued into oblivion. Copyright, trademark, TOS violations, etc. Nothing remotely similar to a modern search engine would ever be allowed to start if it wasn't grandfathered in by the time people started getting serious about law enforcement on the internet.
The law is not as clear cut as people imagine it to be. In many cases the text of the law is less important than precedent and established practice.
- deleted 7y ago[deleted]
- beamatronic 7y agoIf you tried it today, you would need a paid subscription to get access to a search engine
- quickthrower2 7y agoThere were paid search engines in the 90's. I don't recall the names. Maybe the fact I don't remember the name says it all!
- ReinholdNiebuhr 7y agoI'd be okay with a paid subscription service if I got privacy and some other things as well. There's room for both free and pay. Duckduckgo is ok, but not google. Would be nice to have google, without the bias and without the privacy issues.
- treerock 7y agoExcept perhaps Google is actually better at search because it knows so much about you. In that case a Google without the privacy issues is basically the same as duckduckgo.
- mosselman 7y agoI always hear people complain about DuckDuckGo and it gives me the impression that they haven't actually used it for a while. I only use DuckDuckGo and I find it to be a great search engine.
- fauigerzigerk 7y ago>I'd be okay with a paid subscription service if I got privacy and some other things as well Payment and privacy are a very problematic combination, especially where end-to-end encryption is essentially impossible.
- benoliver999 7y agoYeah "I'd be OK with a paid subscription" means "I want my searches tied to one account". You pretty much get to choose one or the other.
- 0xcde4c3db 7y agoAlso: how many executives, seeing that corpus of data on people who are demonstrably willing to spend money, would pass up the opportunity to extract more money by finding a way to sell it to marketers?
- 481092 7y agoYou'll never have a search engine without bias. A search engine's job is just that, to bias various data, not soak up any and all data on the internet. That's why humans or anyone or anything in this universe is biased, because we can only soak up so much info from so many proverbial directories in a lifetime.
- nine_k 7y agoThe web may be biased. But a search engine that reflects that bias is just doing its job correctly. One inevitable and useful bias a search engine has is the bias against spam. It's aligned with the interests of the user. There can be (and are) biases that are against the users' interests but are aligned with operator's: promotion of some content, censorship of other content. This bias I would rather see gone.
- kamfc 7y agoFor a second, thought you were describing Netflix's algorithm.
- cortesoft 7y agoThe original search engines (like Yahoo) requires users to submit their websites. It worked well enough.
- mgamache 7y agoreally? IMHO Yahoo was great only because we did't have pagerank(TM). It lead to purgatory, but produced superior results.
- jasode 7y ago>Yahoo) requires users to submit their websites. It worked well enough. It didn't work well enough because back in 2000, Yahoo was one of the first companies to license Google's search engine. (The previous year, AOL licensed Google's search engine.) Yahoo incorrectly thought that their "portal" and user-submitted "directories" were more important to web surfers than search algorithms like Google's Pagerank. They eventually realized their misjudgment and then tried to build a Google clone through acquisitions like Inktomi and Overture: https://www.cnet.com/g00/news/yahoo-to-acquire-inktomi/ https://www.cnet.com/g00/news/yahoo-to-acquire-inktomi/ https://www.cbsnews.com/news/yahoo-buys-search-engine/ https://www.cbsnews.com/news/yahoo-buys-search-engine/ https://www.cnet.com/g00/news/yahoo-dumps-google-search-technology/ https://www.cnet.com/g00/news/yahoo-dumps-google-search-tech...
- 3xblah 7y agoWho imagines the law to be clear cut? There is a lack of precedent and therefore ambuiguity because the cases were never strong enough. They still are not strong enough. The ambiguity favors the incumbents, who have enough cash to finance any litigation they so choose. They do not want the law on scraping clarified through legal precedent lest they end up the losers. The price of losing a weak case against a new search engine would be worse for an existing one than avoiding litigation and instead relying on other methods of blocking competition. The OP who admits he is not a lawyer yet proceeds to advise others. Why? He cannot distinguish the difference between scraping and publishing scraped content or between something being "illegal", i.e. a criminal offense, versus something legal that someone perceives as violating their "rights" and prompts them to file a civil suit. As is typical in the US, such suits may ultimately be meritless and purely used to intimidate. There are law firms that make money from convincing people to file civil suits against potential competitors. They might list a number of potential causes of action against someone who visits your website. You might be able to sue your users. It does not mean you will win. In losing, you may create legal precedent that in effect enables more of them to violate your "rights". LinkedIn is taking a calculated risk. If they continue to sue web users who use automation (as does LinkedIn), not every defendant is going be intimidated. Some will defend themselves. I think one of them is going to win. It is one thing to purport to limit how someone can use some data that you provide to them. That is not "scraping". That is something else. It is another thing to purport to limit how someone can choose to use their own computer, i.e., interactively versus non-interactively (automation).
- tyingq 7y agoTicket Master might be the exception...they were totally owned. https://www.vice.com/en_us/article/mgxqb8/the-man-who-broke-ticketmaster https://www.vice.com/en_us/article/mgxqb8/the-man-who-broke-...
- harryh 7y agoThere is no reason at all to believe this. Search engines provide a huge benefit to sites which is why they let search crawlers access them. Even today anyone who wants to opt out of being in the Google (or MSFT, or whoever) search engine can trivially do so by updating their robots.txt. Very very few sites choose to do so.
- pflenker 7y agoYou are definitely right - Google (News) provides a huge benefit to news publishers, it is responsible for most of their traffic, and still German Lawmakers tried making a law against that so that on top of all what Google does for the publishers, they should also pay for that privilege - the infamous Leistungsschutzrecht[1]. [1]https://en.wikipedia.org/wiki/Ancillary_copyright_for_press_publishers https://en.wikipedia.org/wiki/Ancillary_copyright_for_press_...