4 ms·

Out of curiosity I have no idea how to approach this. Are there indexes of websites available to the public that you are suggesting that can be searched via gre

Out of curiosity I have no idea how to approach this. Are there indexes of websites available to the public that you are suggesting that can be searched via grep or some simple scripting? Or are you suggesting writing a web crawler to build our own index? Or are you suggesting finding curated links to sites?

takeda 7y ago
I miss the time when Internet was so small that a curated list was possible (Yahoo was priding itself to be biggest curated list[1]) [1] http://web.archive.org/web/19990208021747/http://yahoo.com/ http://web.archive.org/web/19990208021747/http://yahoo.com/
- ntsplnkv2 7y ago
  There was a post a month or two ago that talked about the design of the Yahoo homepage back then, and looking at the link again, it's striking how much more pleasurable an experience it is.
- perl4ever 7y ago
  I still have a paperback book that claimed to be a comprehensive, not curated list of internet sites. It didn't even come with electronic media as became customary with computer books later on.
bovermyer 7y ago
There are multiple ways that I know of to approach this problem, but part of the experiment is to explore for yourself.
101404 7y ago
There was the Open Directory Project. Then AOL bought it and killed it. This is the archived remains: https://www.dmoz-odp.org/ https://www.dmoz-odp.org/
jaredsohn 7y ago
Just open up the Internet Yellow Pages book. :) https://www.amazon.com/Internet-Yellow-Pages-6th-ed/dp/1562057847 https://www.amazon.com/Internet-Yellow-Pages-6th-ed/dp/15620... My family actually had a book like and I learned about a bunch of the net culture sites through it.
blondin 7y ago
i might be wrong but i didn't think OP was suggesting to re-index the web. that's a totally different beast from finding an answer to a question/problem or learning a new topic. depending on what i am looking for there are a few avenues i would explore if google wasn't available. github, gitlab, & stackoverflow/stackexchange for code related stuff. wikipedia for general topics. youtube for how-tos. twitter for the news. newsletters and podcasts for links to new articles. &c.
techslave 7y ago
yup. use gopher instead
perl4ever 7y ago
I've idly wondered what would happen if you made a search engine based on a custom crawler that only logged sites without any references to Google.

takeda 7y ago

I miss the time when Internet was so small that a curated list was possible (Yahoo was priding itself to be biggest curated list[1]) [1] http://web.archive.org/web/19990208021747/http://yahoo.com/ http://web.archive.org/web/19990208021747/http://yahoo.com/

ntsplnkv2 7y ago
There was a post a month or two ago that talked about the design of the Yahoo homepage back then, and looking at the link again, it's striking how much more pleasurable an experience it is.
perl4ever 7y ago
I still have a paperback book that claimed to be a comprehensive, not curated list of internet sites. It didn't even come with electronic media as became customary with computer books later on.

bovermyer 7y ago

There are multiple ways that I know of to approach this problem, but part of the experiment is to explore for yourself.

101404 7y ago

There was the Open Directory Project. Then AOL bought it and killed it. This is the archived remains: https://www.dmoz-odp.org/ https://www.dmoz-odp.org/

jaredsohn 7y ago

Just open up the Internet Yellow Pages book. :) https://www.amazon.com/Internet-Yellow-Pages-6th-ed/dp/1562057847 https://www.amazon.com/Internet-Yellow-Pages-6th-ed/dp/15620... My family actually had a book like and I learned about a bunch of the net culture sites through it.

blondin 7y ago

i might be wrong but i didn't think OP was suggesting to re-index the web. that's a totally different beast from finding an answer to a question/problem or learning a new topic. depending on what i am looking for there are a few avenues i would explore if google wasn't available. github, gitlab, & stackoverflow/stackexchange for code related stuff. wikipedia for general topics. youtube for how-tos. twitter for the news. newsletters and podcasts for links to new articles. &c.

techslave 7y ago

yup. use gopher instead

perl4ever 7y ago

I've idly wondered what would happen if you made a search engine based on a custom crawler that only logged sites without any references to Google.