4 ms·

Interesting. What do you mean? I know they compete in AI research, but has (no longer quite so) OpenAI shown any sign of competing for Google's cash cows?

amelius 4y ago
They made great progress in NLP, which is a big part of what Google Search is about. Also, they have the compute infrastructure for it.
- cmrdporcupine 4y ago
  The problem with taking on Google Search is that of taking on the document corpus they build and the spidering / crawling work they do. Google and Bing are permitted by hosts to crawl, but new entrants effectively barred by most site's robots.txt. The barrier to entry is massive on that front alone. We really need a web infrastructure where the index (or the indexer) is somehow public and neutral, not owned by any particular corporate entity. This should have been built into WWW architecture from day one. But legislation could attempt to make it happen now. Companies should be free to innovate on top of it, but we're effectively in a state now where monopoly power has been granted to the people who got there first (well, close to it, anyways). EDIT: to be clearer, imagine a crawling infrastructure run by industry consortium. You pay a membership fee and usage / traffic fees and you get to upload a WebAssembly program that is called with the contents of pages every time the crawler hits them. What you do with them from that point on is up to you, but would presumably be something like building your own corpus for a search engine, etc. Moderation and regulatory oversight on the usages of this data would obviously have to be put in place. But to me it seems preferable to the status quo now.
  - amelius 4y ago
    How does DDG do it?
    - dekhn 4y ago
      they use bing. See: https://en.wikipedia.org/wiki/DuckDuckGo https://en.wikipedia.org/wiki/DuckDuckGo and https://help.duckduckgo.com/duckduckgo-help-pages/results/sources/ https://help.duckduckgo.com/duckduckgo-help-pages/results/so...
      - cmrdporcupine 4y ago
        Why is this comment downvoted? Factual.

amelius 4y ago

They made great progress in NLP, which is a big part of what Google Search is about. Also, they have the compute infrastructure for it.

cmrdporcupine 4y ago
The problem with taking on Google Search is that of taking on the document corpus they build and the spidering / crawling work they do. Google and Bing are permitted by hosts to crawl, but new entrants effectively barred by most site's robots.txt. The barrier to entry is massive on that front alone. We really need a web infrastructure where the index (or the indexer) is somehow public and neutral, not owned by any particular corporate entity. This should have been built into WWW architecture from day one. But legislation could attempt to make it happen now. Companies should be free to innovate on top of it, but we're effectively in a state now where monopoly power has been granted to the people who got there first (well, close to it, anyways). EDIT: to be clearer, imagine a crawling infrastructure run by industry consortium. You pay a membership fee and usage / traffic fees and you get to upload a WebAssembly program that is called with the contents of pages every time the crawler hits them. What you do with them from that point on is up to you, but would presumably be something like building your own corpus for a search engine, etc. Moderation and regulatory oversight on the usages of this data would obviously have to be put in place. But to me it seems preferable to the status quo now.
- amelius 4y ago
  How does DDG do it?
  - dekhn 4y ago
    they use bing. See: https://en.wikipedia.org/wiki/DuckDuckGo https://en.wikipedia.org/wiki/DuckDuckGo and https://help.duckduckgo.com/duckduckgo-help-pages/results/sources/ https://help.duckduckgo.com/duckduckgo-help-pages/results/so...
    - cmrdporcupine 4y ago
      Why is this comment downvoted? Factual.