4 ms·

I have been wanting this for years... If you look at the original Yahoo Page when Yahoo first started out it attempted to solve this problem. I believe this i

I have been wanting this for years... If you look at the original Yahoo Page when Yahoo first started out it attempted to solve this problem. I believe this index could be regionally or language based... In the United States one could use Dewey Decimal https://en.wikipedia.org/wiki/Dewey_Decimal_Classification https://en.wikipedia.org/wiki/Dewey_Decimal_Classification Library of Congress https://en.wikipedia.org/wiki/Library_of_Congress_Classification https://en.wikipedia.org/wiki/Library_of_Congress_Classifica...

tokyodude 7y ago
It won't work without a central authority. See Soundcloud as an example. People tag their music with whatever they think will get them traffic. So, in order to do this you'll need a mass of volunteers which will lead to politics "XYZ should be classified as G! No it should be F!", "classifying ABC as DEF is racists/sexist/..." and other arguments. You'll also get people lobbying to have things removed (right to be forgotten, pornography, drug ad, prostitution ads, disparaging the government -China, -Thailand, etc..., etc... I'm not saying it shouldn't be done but I think it will be way more work than expected and there will be all kinds of issues.
- zdragnar 7y ago
  If anything, that sounds like a solid argument to decentralize it. I don't want China's government, white supremacists, churches, soccer moms, Jihadis or grievance-of-the-month activists controlling how information is indexed; I would rather use multiple indexes that balance out controlling interests and biases.
  - rabidrat 7y ago
    Unfortunately if it's decentralized, then it becomes controlled by spamlords, SEO artists, advertisers, and anyone else who stands to gain from manipulating the index to their advantage. At least if it's centralized, the fights are out in the open and have a chance of converging on something reasonable (like e.g. wikipedia).
    - snaky 7y ago
      Decentralized doesn't mean flat. You can trust to some actors only (and to some they trust to).
- mathnmusic 7y ago
  I believe the problems are far smaller when building web directories in narrower contexts such as people using the web to learn new topics / acquire new skills. News, politics is where most of controversies lie, compared to, say algorithms or abstract algebra. Our project LearnAwesome[1] currently relies on volunteers to curate topics, but classfication / ontology engineering is in fact seems to be a hard problem. [1]: https://github.com/learn-awesome/learn-awesome https://github.com/learn-awesome/learn-awesome
- dredmorbius 7y ago
  No, you wouldn't need a central authority, though various subject indices and/or search interfaces (divorced from the crawl/index) would supply rank and/or reputation scores.
peterburkimsher 7y ago
I agree that regions & languages are one way to classify data, but there are other more meaningful sub-culture categories. I tried to simplify all data into ~30 categories. My own interests fit into 16, so I drew a visual representation of them. https://github.com/peterburk/sortlikes https://github.com/peterburk/sortlikes Next, I need to figure out the sub-categories. Genres for music, countries for travel, etc. What interests me most is the cross-cultural connections. For example, Taiwanese punk rock (Fire Ex), or Mongolian folk metal (Nine Treasures, Hanggai). I like that music because it's the same sub-category I'm interested in (Music/Rock). It's also possible to model the flow of finance around the world through this categorisation. Some of the categories are innately human and don't seem to exist in animals (music, cooking). Email me if you'd like to chat more about how to categorise culture - I think it's important and I've got lots of ideas about it, but I haven't yet met any other people with this same passion.

tokyodude 7y ago

It won't work without a central authority. See Soundcloud as an example. People tag their music with whatever they think will get them traffic. So, in order to do this you'll need a mass of volunteers which will lead to politics "XYZ should be classified as G! No it should be F!", "classifying ABC as DEF is racists/sexist/..." and other arguments. You'll also get people lobbying to have things removed (right to be forgotten, pornography, drug ad, prostitution ads, disparaging the government -China, -Thailand, etc..., etc... I'm not saying it shouldn't be done but I think it will be way more work than expected and there will be all kinds of issues.

zdragnar 7y ago
If anything, that sounds like a solid argument to decentralize it. I don't want China's government, white supremacists, churches, soccer moms, Jihadis or grievance-of-the-month activists controlling how information is indexed; I would rather use multiple indexes that balance out controlling interests and biases.
- rabidrat 7y ago
  Unfortunately if it's decentralized, then it becomes controlled by spamlords, SEO artists, advertisers, and anyone else who stands to gain from manipulating the index to their advantage. At least if it's centralized, the fights are out in the open and have a chance of converging on something reasonable (like e.g. wikipedia).
  - snaky 7y ago
    Decentralized doesn't mean flat. You can trust to some actors only (and to some they trust to).
mathnmusic 7y ago
I believe the problems are far smaller when building web directories in narrower contexts such as people using the web to learn new topics / acquire new skills. News, politics is where most of controversies lie, compared to, say algorithms or abstract algebra. Our project LearnAwesome[1] currently relies on volunteers to curate topics, but classfication / ontology engineering is in fact seems to be a hard problem. [1]: https://github.com/learn-awesome/learn-awesome https://github.com/learn-awesome/learn-awesome
dredmorbius 7y ago
No, you wouldn't need a central authority, though various subject indices and/or search interfaces (divorced from the crawl/index) would supply rank and/or reputation scores.

peterburkimsher 7y ago

I agree that regions & languages are one way to classify data, but there are other more meaningful sub-culture categories. I tried to simplify all data into ~30 categories. My own interests fit into 16, so I drew a visual representation of them. https://github.com/peterburk/sortlikes https://github.com/peterburk/sortlikes Next, I need to figure out the sub-categories. Genres for music, countries for travel, etc. What interests me most is the cross-cultural connections. For example, Taiwanese punk rock (Fire Ex), or Mongolian folk metal (Nine Treasures, Hanggai). I like that music because it's the same sub-category I'm interested in (Music/Rock). It's also possible to model the flow of finance around the world through this categorisation. Some of the categories are innately human and don't seem to exist in animals (music, cooking). Email me if you'd like to chat more about how to categorise culture - I think it's important and I've got lots of ideas about it, but I haven't yet met any other people with this same passion.