3 ms·

Turning the Web into a giant database will crush any attempt to segregate the “best” information into a separate database so that it can be processed and search

Turning the Web into a giant database will crush any attempt to segregate the “best” information into a separate database so that it can be processed and searched more deeply. This is pure speculation, and I personally doubt its true. First, much of the content of on the web amounts to echoing, filtering, and distorting high-quality source data. If you can tap that source data, are the gains you realize from what's been derived from it worth the extra cost of going out and doing the work to acquire it and then separate out the noise? Second, there is a lot of real or perceived value in knowing where your data is coming from. Tons of companies pay tons of money to get data and statistics from sources that they can trust. Until Google's willing to vouch for these results beyond "we tried", the bespoke curatorial approach is going to keep capturing these dollars.

robryan 17y ago
I think it can rival a structured dataset if it is done correctly. The big advantage that google has, and I know there was a paper they put out about it a couple of months ago, is that they have access to such a large source of data that they can use a great deal of sources to weed out the noise by only using the stuff that appears over many sources. Then you could also assign sources a reliability rating based on how acurite the information they provide is compared with other reliable sources. Kind of like a new pagerank but for data integrety.
- calambrac 17y ago
  Don't get me wrong, I think it's very cool that they're doing this kind of work, and I hope it takes off. What I was really responding to was the idea that this approach would "crush" a more traditionally curated database. At least in the short term, I don't see that happening, because the number of really high-quality data sources is still pretty tractable, because a lot of what's on the web is just going to be echoing/distorting those primary sources anyways, and because trust is such a huge part of that market.
  - ErrantX 17y ago
    The issue is though that Wolfram is not going to provide the mudane sort of data that people want in the instant way Google are going to claim to do it. Search camera on Wolfram and I expect you get lost of data on the history of camera's and other such stuff. Google seem to be offering to provide a list of camera models with some pertinent data for each... I suspect the latter will be more "useful" (especially as Wikipedia would probably be fairly reliable for the camera background...) I think your point is valid: a curated database will be the #1 source for info on camera history and facts. Unfortunately that means Wolfram is competing with Wikipedia not Google... and that is probably even worse for them :(
    - calambrac 17y ago
      You make a really good point. For more consumer-oriented data, a more casual level of accuracy is probably acceptable, at least acceptable enough to hang ads off of. I had my head stuck in the world of data that people are actually willing to pay for directly.

robryan 17y ago

I think it can rival a structured dataset if it is done correctly. The big advantage that google has, and I know there was a paper they put out about it a couple of months ago, is that they have access to such a large source of data that they can use a great deal of sources to weed out the noise by only using the stuff that appears over many sources. Then you could also assign sources a reliability rating based on how acurite the information they provide is compared with other reliable sources. Kind of like a new pagerank but for data integrety.

calambrac 17y ago
Don't get me wrong, I think it's very cool that they're doing this kind of work, and I hope it takes off. What I was really responding to was the idea that this approach would "crush" a more traditionally curated database. At least in the short term, I don't see that happening, because the number of really high-quality data sources is still pretty tractable, because a lot of what's on the web is just going to be echoing/distorting those primary sources anyways, and because trust is such a huge part of that market.
- ErrantX 17y ago
  The issue is though that Wolfram is not going to provide the mudane sort of data that people want in the instant way Google are going to claim to do it. Search camera on Wolfram and I expect you get lost of data on the history of camera's and other such stuff. Google seem to be offering to provide a list of camera models with some pertinent data for each... I suspect the latter will be more "useful" (especially as Wikipedia would probably be fairly reliable for the camera background...) I think your point is valid: a curated database will be the #1 source for info on camera history and facts. Unfortunately that means Wolfram is competing with Wikipedia not Google... and that is probably even worse for them :(
  - calambrac 17y ago
    You make a really good point. For more consumer-oriented data, a more casual level of accuracy is probably acceptable, at least acceptable enough to hang ads off of. I had my head stuck in the world of data that people are actually willing to pay for directly.