7 ms·

Seems like a private cache of the web would solve the problem? Why does it need to be public?

by sesuximo 6y ago

lisper 6y ago
Seriously? Google is a private cache of the web. That is the problem.
- sesuximo 6y ago
  Google doesn’t give anyone access to said cache. I mean one crawler with a shared api among competitors. So exactly the same as the public cache, but run my a private company and accessed for a small fee
  - ajcp 6y ago
    > run [by] a private company and accessed for a small fee That is exactly the opposite of a public cache.
    - sesuximo 6y ago
      Not really. It serves the same function. Either you pay this hypothetical company or ??? pays to keep up the public one.
      - ajcp 6y ago
        Just because it serves the same function does not mean the implementation is the same. Private military contractors and a US infantry squad serve the same function, but the implementation completely changes their context. That being said what I think you're arguing for would be the implementation of a public utility or private-public business. If that's the case then yes, what you're saying is correct.
  - lisper 6y ago
    I don't think you're quite clear on what the words "public" and "private" mean. "Public" is not a synonym for "run by the government" and "private" is not a synonym for "closed to everyone but the owner". Restaurants, for example, are generally open to the public, but they are not public. A restaurant owner is, with a few exceptions, free to refuse service to anyone at any time. If it's "exactly the same as a public cache" then it's public, even if it is managed by a private company. The difference is not in who has access, the difference is in who decides who has access.
    - sesuximo 6y ago
      Ok I am not clear then, but I’m less clear after your comment! In a public cache, who would you want to decide who has access? Is simply saying “anyone who pays has access” enough to qualify as public? if so, then I agree and this was my (possibly poorly phrased) intention in the original comment. But imo the restaurant model is also fine; in most cases people have access and it works.
      - lisper 6y ago
        > Is simply saying “anyone who pays has access” enough to qualify as public? No because someone has to set the price, which is just an indirect method of controlling who has access. > the restaurant model is also fine It works for restaurants because there is competition. The whole point here is that web crawling/caching is a natural monopoly. A better analogy here would be Apple's app store, or copyrighted standards with the force of law [1]. These nominally follow the "anyone who pays has access" model but they are not public, and the result is the same set of problems. [1] https://www.thebrandprotectionblog.com/public-laws-private-standards-copyright-fair-use/ https://www.thebrandprotectionblog.com/public-laws-private-s...
  - sct202 6y ago
    You can API google search results to make a meta-search engine if you want to but it's like $5 / 1k requests.
    - twiddlebits 6y ago
      Google's TOS prevents blending (alterations, etc.) though.
  - deleted 6y ago
    [deleted]
  - visarga 6y ago
    > Google doesn’t give anyone access to said cache. It would also be useful for deep searches, exceeding the 1000 result limit, empowering all sorts of NLP applications.

Seems like a private cache of the web would solve the problem? Why does it need to be public?

lisper 6y ago

Seriously? Google is a private cache of the web. That is the problem.

sesuximo 6y ago
Google doesn’t give anyone access to said cache. I mean one crawler with a shared api among competitors. So exactly the same as the public cache, but run my a private company and accessed for a small fee
- ajcp 6y ago
  > run [by] a private company and accessed for a small fee That is exactly the opposite of a public cache.
  - sesuximo 6y ago
    Not really. It serves the same function. Either you pay this hypothetical company or ??? pays to keep up the public one.
    - ajcp 6y ago
      Just because it serves the same function does not mean the implementation is the same. Private military contractors and a US infantry squad serve the same function, but the implementation completely changes their context. That being said what I think you're arguing for would be the implementation of a public utility or private-public business. If that's the case then yes, what you're saying is correct.
- lisper 6y ago
  I don't think you're quite clear on what the words "public" and "private" mean. "Public" is not a synonym for "run by the government" and "private" is not a synonym for "closed to everyone but the owner". Restaurants, for example, are generally open to the public, but they are not public. A restaurant owner is, with a few exceptions, free to refuse service to anyone at any time. If it's "exactly the same as a public cache" then it's public, even if it is managed by a private company. The difference is not in who has access, the difference is in who decides who has access.
  - sesuximo 6y ago
    Ok I am not clear then, but I’m less clear after your comment! In a public cache, who would you want to decide who has access? Is simply saying “anyone who pays has access” enough to qualify as public? if so, then I agree and this was my (possibly poorly phrased) intention in the original comment. But imo the restaurant model is also fine; in most cases people have access and it works.
    - lisper 6y ago
      > Is simply saying “anyone who pays has access” enough to qualify as public? No because someone has to set the price, which is just an indirect method of controlling who has access. > the restaurant model is also fine It works for restaurants because there is competition. The whole point here is that web crawling/caching is a natural monopoly. A better analogy here would be Apple's app store, or copyrighted standards with the force of law [1]. These nominally follow the "anyone who pays has access" model but they are not public, and the result is the same set of problems. [1] https://www.thebrandprotectionblog.com/public-laws-private-standards-copyright-fair-use/ https://www.thebrandprotectionblog.com/public-laws-private-s...
- sct202 6y ago
  You can API google search results to make a meta-search engine if you want to but it's like $5 / 1k requests.
  - twiddlebits 6y ago
    Google's TOS prevents blending (alterations, etc.) though.
- deleted 6y ago
  [deleted]
- visarga 6y ago
  > Google doesn’t give anyone access to said cache. It would also be useful for deep searches, exceeding the 1000 result limit, empowering all sorts of NLP applications.