4 ms·

Ask HN: Ways to discover stuff online without a search engine

What are some tools or methods (whether real, theoretical, academic, or otherwise) for discovering things on the Internet, as an alternative to using a search engine. I'm not focusing on practicality here, just looking some interesting and uncommon methods for getting novel results. I suspect there's a few specific to certain types of media. The methods I'm aware of are: - Recommendation engines (e.g. Youtube, most social media feeds) - Organic / constructed sequences of data (e.g. a playlist, or sorted lists using some measurement like price) - Non-text search (e.g. search via image, sound, or geolocation) - Stochastic (e.g. guessing a domain name, or entering some random text into a search engine and skipping 100 first suggestions) - Automatic link traversal (e.g. a web crawler algorithm) - Private sharing (e.g. sending a link in an email) - Public sharing (e.g. web portal, or blogroll) - Hybrid (e.g. algorithmic search engine results, news aggregators) But what am I missing?

gostsamo 5y ago
At least two of your suggestions include a search engine. What I can add are: word of mouth (things shared by people), news feed (someone shared something on FB/TW), ads (random ad on a random website), agrregators (specialized website for curating information on a certain topic), forum (get an answer from a random person in an internet form like HN or SO).
- sargstuff 5y ago
  check out the bibliography section / resources used for a project/paper/work (on-line & offline) univeristies publically publish thesis papers / technical reports / project documentation. A bibliography is a required feature of published stuff! also magazines / trade organizations (ieee, acm, etc)
  - lasagna_coder 5y ago
    Ah yeah that's a good one too. Funny, one could say that this is a good way of finding interesting information, while also finding methods of finding interesting information.
    - sargstuff 5y ago
      neat when find a reference to things related/outside of project scope. more detailed, but following the bibliography chain also can give the equivalent of a change log history. with the idea of, "hey, noone took this fork in the road because <insert issue(s) at time of publication>. currently though, the <issues aka missing/way to complex at the time/to costly/more supporting research needed> are reasonable/not unsurmountable." lot of thesis papers / projects are "proof of concept"
      - lasagna_coder 5y ago
        Changelog is a good analogy. It would be nice if crawling, building a graph, and extracting some information was more straight forward and didn't involve so many paid platforms, PDFs, varying reference standards, with also a fairly high dependency on a search engine to find referenced papers. It's still not quite so easy to just "click through" and "Ctrl + F".
        sargstuff 5y ago
        search engines are about as easy as it gets, at least as far as searching for available things. searching on library of congress id for published works about the only thing worse than a google site dump. certainly a step up from library index cards, microfiche, and/or writing a snail mail letter to a university department for list of deparment thesis papers. library congress on-line stuff : https://eresources.loc.gov https://eresources.loc.gov https://eresources.loc.gov/search~S9/m?SEARCH=Dissertation https://eresources.loc.gov/search~S9/m?SEARCH=Dissertation "normalize" electronic data : https://sindice.com https://sindice.com
- lasagna_coder 5y ago
  It's interesting how many discovery practices rely on asking another person. That has got me thinking about the process of inquiry / learning. Another facet I guess is using some combination of tools, but developing a "method", similar to an experiment, where I have a goal/hypothesis/measurement and test some input against its output, see how close it brought me to the goal, assign some kind of value, then iterate until I've developed a kind of manual/automatic algorithm that can be applied. I also think the idea of 2D and 3D virtual spatial positioning of links/abstracts is an interesting idea, but I can't recall anywhere I've found that done in a way I was impressed by.
  - sargstuff 5y ago
    historically, things were by word of mouth. internet & ease of communication have spead up the 'word of mouth'
    - lasagna_coder 5y ago
      True, I feel like automated systems also seem to still be pretty far behind in terms of providing useful results for certain forms of discovery, but is still pretty easy for people. Take this question and most of Ask HN, there's some pretty interesting/useful results within a few minutes. But providing a question like mine to google and most results will yield pages of "alternative search engines to google" websites, which is no where near correct, but does "sound" correct if you didn't understand the context of my question, such as, what the Internet is and what a search engine is (and isn't). But then, I feel like it doesn't really scale super well, and doesn't always handle vagueness. For example, even of HN I doubt there will be very many quality/useful answers, simply because there's a certain kind of popularity required to get that here. Looking at something like Quora, there's too much room for non-answers and what amounts to spam. Then, there's the difficulty of being specific enough that people can answer but also do so in a timely manner (the speed of a search engine to provide simply a "close enough" result is part of its utility). Like the bibliography suggestion, although I said practicality doesn't matter, its an example of how interesting stuff can be discovered (without asking anyone) but requires a huge amount of time (after the initial "look up") and cognitive effort (but that's a kind of curve that tapers off once one is familiar with a particular area of research).
- sargstuff 5y ago
  somewhat topic/project specific: makezine / instructables personal web page(s) of project/thesis author(s) on-line courses typically list resources (used, additional, related) looking at project documentation for things on a site such as thingiverse / instructables of references to other things aka what did to be able to enable project to be done. hopefully not a wrote own os.
  - lasagna_coder 5y ago
    I feel like this begins to blur the lines between discovery tool and resource to be discovered. The idea of instructable /project blog is interesting as a discovery tool though. It makes me wonder if the act of creating one is also a discovery tool, similar to a lecture pad or research notebook. It's kind of like, active vs passive discovery, kind of like I wrote about in the other comment, where one might have to develop a certain "method" using various tools to create a reliable + repeatable process. Once I start searching for an answer or move towards a goal, by keeping a log of the steps, the "discovery" of something "novel" might be just where the knowledge gaps are - knowing what I don't know. But like asking questions of other people, this isn't automated, and requires skill/practice/time.
    - sargstuff 5y ago
      even a simplified version of snarfing bibliograph data for setting up web "automata" munging with available tools, such as https://ahrefs.com/blog/link-building-tools/ https://ahrefs.com/blog/link-building-tools/ , requires some skill/practice/time before "return key" worthy.
      - lasagna_coder 5y ago
        Ah I need to learn more about this.
manx 5y ago
Web directories come to mind. You didn't need to know what to look for, instead you were able to browse a huge tree. Are there any modern web directories out there?
- lasagna_coder 5y ago
  I think mostly managed by institutions. Don't know of any surrounding niches, or managed by independent people / organisations. But also this is kind of similar to a more structured web portal.
freediver 5y ago
TinyGem is built exactly for this purpose. It automates the process of discovery based on previous interest. https://tinygem.org https://tinygem.org
- sargstuff 5y ago
  bit more practical than https://www.semanticscholar.org/ https://www.semanticscholar.org/ & beach boys "AI good citations" review. https://www.semanticscholar.org/paper/CiteSeer%3A-an-automatic-citation-indexing-system-Giles-Bollacker/592462425a4d23547dd0f3c9318350e5dcceb1a6 https://www.semanticscholar.org/paper/CiteSeer%3A-an-automat... https://www.semanticscholar.org/paper/Emerging-Challenges-for-Digital-Resources-S.-Vijayarani/438d66d481f4aa04ade14524edcb6d8731a1b1ee https://www.semanticscholar.org/paper/Emerging-Challenges-fo... https://www.semanticscholar.org/paper/Deep-learning-in-citation-recommendation-models-Ali-Kefalas/0e812307073c0e6c3a344a0ea4cf69541608a9ab https://www.semanticscholar.org/paper/Deep-learning-in-citat... https://www.semanticscholar.org/paper/Towards-reproducibility-in-recommender-systems-Beel-Breitinger/d31eea303654928916ba39036e33ecfb9a1fe979 https://www.semanticscholar.org/paper/Towards-reproducibilit... https://www.semanticscholar.org/paper/Scienstein-%3A-A-Research-Paper-Recommender-System-Gipp-Beel/deab9886bd1f39bea5b85fa76ca8f705fec9a85c https://www.semanticscholar.org/paper/Scienstein-%3A-A-Resea... https://www.semanticscholar.org/paper/Similarity-measures-for-document-mapping%3A-A-study-Sternitzke-Bergmann/fa9c0c2e1e744c569aff3cebcd8393ddb4f5aa03 https://www.semanticscholar.org/paper/Similarity-measures-fo... https://www.semanticscholar.org/paper/Semantic-audio-content-based-music-recommendation-Bogdanov-Haro/9987d626a2c2e784f0f64bb3b66e85c1b0d79f95 https://www.semanticscholar.org/paper/Semantic-audio-content...
sargstuff 5y ago
docear : https://docear.org/ https://docear.org/ (comparison page with similar software ) ** recommender system conferences: recsys -> https://dl.acm.org/doi/proceedings/10.1145/3240323 https://dl.acm.org/doi/proceedings/10.1145/3240323 umap -> https://dl.acm.org/doi/proceedings/10.1145/3450613 https://dl.acm.org/doi/proceedings/10.1145/3450613 sigweb -> https://www.sigweb.org/conferences/acm-sigweb-conferences https://www.sigweb.org/conferences/acm-sigweb-conferences ** "Recommender Systems Handbook" with source code : https://link.springer.com/book/10.1007/978-0-387-85820-3 https://link.springer.com/book/10.1007/978-0-387-85820-3

gostsamo 5y ago

At least two of your suggestions include a search engine. What I can add are: word of mouth (things shared by people), news feed (someone shared something on FB/TW), ads (random ad on a random website), agrregators (specialized website for curating information on a certain topic), forum (get an answer from a random person in an internet form like HN or SO).

sargstuff 5y ago
check out the bibliography section / resources used for a project/paper/work (on-line & offline) univeristies publically publish thesis papers / technical reports / project documentation. A bibliography is a required feature of published stuff! also magazines / trade organizations (ieee, acm, etc)
- lasagna_coder 5y ago
  Ah yeah that's a good one too. Funny, one could say that this is a good way of finding interesting information, while also finding methods of finding interesting information.
  - sargstuff 5y ago
    neat when find a reference to things related/outside of project scope. more detailed, but following the bibliography chain also can give the equivalent of a change log history. with the idea of, "hey, noone took this fork in the road because <insert issue(s) at time of publication>. currently though, the <issues aka missing/way to complex at the time/to costly/more supporting research needed> are reasonable/not unsurmountable." lot of thesis papers / projects are "proof of concept"
    - lasagna_coder 5y ago
      Changelog is a good analogy. It would be nice if crawling, building a graph, and extracting some information was more straight forward and didn't involve so many paid platforms, PDFs, varying reference standards, with also a fairly high dependency on a search engine to find referenced papers. It's still not quite so easy to just "click through" and "Ctrl + F".
      - sargstuff 5y ago
        search engines are about as easy as it gets, at least as far as searching for available things. searching on library of congress id for published works about the only thing worse than a google site dump. certainly a step up from library index cards, microfiche, and/or writing a snail mail letter to a university department for list of deparment thesis papers. library congress on-line stuff : https://eresources.loc.gov https://eresources.loc.gov https://eresources.loc.gov/search~S9/m?SEARCH=Dissertation https://eresources.loc.gov/search~S9/m?SEARCH=Dissertation "normalize" electronic data : https://sindice.com https://sindice.com
lasagna_coder 5y ago
It's interesting how many discovery practices rely on asking another person. That has got me thinking about the process of inquiry / learning. Another facet I guess is using some combination of tools, but developing a "method", similar to an experiment, where I have a goal/hypothesis/measurement and test some input against its output, see how close it brought me to the goal, assign some kind of value, then iterate until I've developed a kind of manual/automatic algorithm that can be applied. I also think the idea of 2D and 3D virtual spatial positioning of links/abstracts is an interesting idea, but I can't recall anywhere I've found that done in a way I was impressed by.
- sargstuff 5y ago
  historically, things were by word of mouth. internet & ease of communication have spead up the 'word of mouth'
  - lasagna_coder 5y ago
    True, I feel like automated systems also seem to still be pretty far behind in terms of providing useful results for certain forms of discovery, but is still pretty easy for people. Take this question and most of Ask HN, there's some pretty interesting/useful results within a few minutes. But providing a question like mine to google and most results will yield pages of "alternative search engines to google" websites, which is no where near correct, but does "sound" correct if you didn't understand the context of my question, such as, what the Internet is and what a search engine is (and isn't). But then, I feel like it doesn't really scale super well, and doesn't always handle vagueness. For example, even of HN I doubt there will be very many quality/useful answers, simply because there's a certain kind of popularity required to get that here. Looking at something like Quora, there's too much room for non-answers and what amounts to spam. Then, there's the difficulty of being specific enough that people can answer but also do so in a timely manner (the speed of a search engine to provide simply a "close enough" result is part of its utility). Like the bibliography suggestion, although I said practicality doesn't matter, its an example of how interesting stuff can be discovered (without asking anyone) but requires a huge amount of time (after the initial "look up") and cognitive effort (but that's a kind of curve that tapers off once one is familiar with a particular area of research).
sargstuff 5y ago
somewhat topic/project specific: makezine / instructables personal web page(s) of project/thesis author(s) on-line courses typically list resources (used, additional, related) looking at project documentation for things on a site such as thingiverse / instructables of references to other things aka what did to be able to enable project to be done. hopefully not a wrote own os.
- lasagna_coder 5y ago
  I feel like this begins to blur the lines between discovery tool and resource to be discovered. The idea of instructable /project blog is interesting as a discovery tool though. It makes me wonder if the act of creating one is also a discovery tool, similar to a lecture pad or research notebook. It's kind of like, active vs passive discovery, kind of like I wrote about in the other comment, where one might have to develop a certain "method" using various tools to create a reliable + repeatable process. Once I start searching for an answer or move towards a goal, by keeping a log of the steps, the "discovery" of something "novel" might be just where the knowledge gaps are - knowing what I don't know. But like asking questions of other people, this isn't automated, and requires skill/practice/time.
  - sargstuff 5y ago
    even a simplified version of snarfing bibliograph data for setting up web "automata" munging with available tools, such as https://ahrefs.com/blog/link-building-tools/ https://ahrefs.com/blog/link-building-tools/ , requires some skill/practice/time before "return key" worthy.
    - lasagna_coder 5y ago
      Ah I need to learn more about this.

manx 5y ago

Web directories come to mind. You didn't need to know what to look for, instead you were able to browse a huge tree. Are there any modern web directories out there?

lasagna_coder 5y ago
I think mostly managed by institutions. Don't know of any surrounding niches, or managed by independent people / organisations. But also this is kind of similar to a more structured web portal.

freediver 5y ago

TinyGem is built exactly for this purpose. It automates the process of discovery based on previous interest. https://tinygem.org https://tinygem.org

sargstuff 5y ago
bit more practical than https://www.semanticscholar.org/ https://www.semanticscholar.org/ & beach boys "AI good citations" review. https://www.semanticscholar.org/paper/CiteSeer%3A-an-automatic-citation-indexing-system-Giles-Bollacker/592462425a4d23547dd0f3c9318350e5dcceb1a6 https://www.semanticscholar.org/paper/CiteSeer%3A-an-automat... https://www.semanticscholar.org/paper/Emerging-Challenges-for-Digital-Resources-S.-Vijayarani/438d66d481f4aa04ade14524edcb6d8731a1b1ee https://www.semanticscholar.org/paper/Emerging-Challenges-fo... https://www.semanticscholar.org/paper/Deep-learning-in-citation-recommendation-models-Ali-Kefalas/0e812307073c0e6c3a344a0ea4cf69541608a9ab https://www.semanticscholar.org/paper/Deep-learning-in-citat... https://www.semanticscholar.org/paper/Towards-reproducibility-in-recommender-systems-Beel-Breitinger/d31eea303654928916ba39036e33ecfb9a1fe979 https://www.semanticscholar.org/paper/Towards-reproducibilit... https://www.semanticscholar.org/paper/Scienstein-%3A-A-Research-Paper-Recommender-System-Gipp-Beel/deab9886bd1f39bea5b85fa76ca8f705fec9a85c https://www.semanticscholar.org/paper/Scienstein-%3A-A-Resea... https://www.semanticscholar.org/paper/Similarity-measures-for-document-mapping%3A-A-study-Sternitzke-Bergmann/fa9c0c2e1e744c569aff3cebcd8393ddb4f5aa03 https://www.semanticscholar.org/paper/Similarity-measures-fo... https://www.semanticscholar.org/paper/Semantic-audio-content-based-music-recommendation-Bogdanov-Haro/9987d626a2c2e784f0f64bb3b66e85c1b0d79f95 https://www.semanticscholar.org/paper/Semantic-audio-content...

sargstuff 5y ago

docear : https://docear.org/ https://docear.org/ (comparison page with similar software ) ** recommender system conferences: recsys -> https://dl.acm.org/doi/proceedings/10.1145/3240323 https://dl.acm.org/doi/proceedings/10.1145/3240323 umap -> https://dl.acm.org/doi/proceedings/10.1145/3450613 https://dl.acm.org/doi/proceedings/10.1145/3450613 sigweb -> https://www.sigweb.org/conferences/acm-sigweb-conferences https://www.sigweb.org/conferences/acm-sigweb-conferences ** "Recommender Systems Handbook" with source code : https://link.springer.com/book/10.1007/978-0-387-85820-3 https://link.springer.com/book/10.1007/978-0-387-85820-3