6 ms·

Out of curiosity, how viable is a search engine for hackers (in terms of a business model)? What are some "weaknesses" of Google that this search engine might t

by jmstfv 9y ago

Out of curiosity, how viable is a search engine for hackers (in terms of a business model)? What are some "weaknesses" of Google that this search engine might take advantage of? One thing I noticed is the amount of "low-quality" pages that appear on the page#1 for popular keywords.

wayn3 9y ago
google provides qualitative answers. what would be really neat was a search engine that provides quantitative answers or raw data. i'd like to be able to search google for "all the pages that contain a particular javascript snippet", for example. obviously, google has that data. obviously, they dont want to make it available. there are services that offer such things but they suck because their datasets are way too small. on google specifically: the results pages dont matter anymore. nobody clicks anything but the "#1 results" if they click even that. for questions, google very often responds with something like a quora question or some widget of its own, or something else that is really useful - and if its not useful, you try another query instead of clicking through the results pages. results pages are mostly kept for nostalgia it seems.
- ljw1001 9y ago
  This sounds a lot like Wolfram Alpha, although google does now provide some quantitative results. It works as a calculator and as a units translator for example. that's a good point about only the first few results mattering.
  - wayn3 9y ago
    this does not sound like wolframalpha at all. wolframalpha is computer algebra software turned into a web service which allows for searching some papers, pre-written articles and a bunch of interesting, curated numbers. wolframalpha provides qualitative answers to "questions about math". answering the physics problem "whats the speed of light" with "er its fast" is qualitative. saying its 2.99+/- epsilon 10^6 km/s is a quantitative result based on data gathered from some experiment. answering the search engine problem "whats the speed of light" with "2.99+/- epsilon 10^6 km/s" is not a quantitative result. in a search engine contexts, thats just the most likely correct result . qualitative. just a guess. some computer looked at a bunch of data and decided that one of them was probably right based on some popularity heuristic. its like asking an expert who makes an educated guess and claims its right because seniority. a quantitative search engine would be capable of answering questions like "how many sites do X". wolframalpha doesnt do that. its just maple in a browser.
  - bloaf 9y ago
    I like WA, but most of the quantitative info is what I'd call "highschool stuff." It is improving all the time, and some of it's math capabilities can reach into advanced undergraduate territory, though. https://www.wolframalpha.com/input/?i=IR+spectrum+of+toluene https://www.wolframalpha.com/input/?i=IR+spectrum+of+toluene http://webbook.nist.gov/cgi/cbook.cgi?ID=C108883&Units=SI&Type=IR-SPEC&Index=19#IR-SPEC http://webbook.nist.gov/cgi/cbook.cgi?ID=C108883&Units=SI&Ty...
- futhey 9y ago
  Not intended to be a satisfying answer to your question, but publicwww can do that specific task for you, if that is a problem you need solved for yourself.
sparkzilla 9y ago
Google's search results pages try to do all things for everyone: give some links, some news, some data (knowledge graph) and answer some questions. There is room for new entrants to improve on each of these other areas and make that the dominant search method for that channel. For example, making voice answers for Alexa. You may be interested in my essay about Google's search result problems: http://newslines.org/blog/googles-black-hole/ http://newslines.org/blog/googles-black-hole/
FLUX-YOU 9y ago
https://www.shodan.io/ https://www.shodan.io/ is still around
timr 9y ago
My last startup started this way. My advice: don't do it. It's a bad market. First, there are competitors in the space. Some have done an objectively nice job, but they're not blowing up. Second, you don't realize it, but you're not just competing with Google (the search engine) -- you're competing with every product in the Google ecosystem. If you want people to use your new niche search engine, you have to find a value proposition so compelling that people will explicitly choose to use your search, even though every other tool they use is tied back to Google somehow. That's insanely hard. Finally, but most importantly: programmers are a tiny market, they're terribly picky, conservative, and they don't like to pay for things. If you're going to take on the challenge of beating Google in a vertical, at least make it one that's huge or has huge value. Focusing on programmers means that you'll spend a ton of time trying to optimize for a highly technical, low-value use case that doesn't generalize well.
- manquer 9y ago
  from a pure business model sense it is true programmer focused search is not good space to be in .. Solving problems you face daily and are tired of has always been significant driver of innovation. This is an important and satisfying problem to be working on, not all startups have to become the next google or the next unicorn.
  - timr 9y ago
    "not all startups have to become the next google or the next unicorn" No, but you do have to survive. That's hard enough for any company, but it's much harder when you pick a small target market that can't be trivially replicated in other verticals. So sure, pick a problem that you know well, but be realistic about the achievable market size, and don't start a business that requires huge scale to work (i.e. a search engine), when your target market is tiny. (Also, as in all things, beware survivorship bias: just because counterexamples to what I'm saying exist, does not mean that the likelihood of creating one is high. Nor does it mean that the counterexamples are doing as well as you think they are.)
deleted 9y ago
[deleted]
tyingq 9y ago
> What are some "weaknesses" of Google A search engine that also has a near monopoly in browsers and phones is a pretty big moat. Since the bulk of browsing is moving to mobile, it's almost like you have to start with an Android killer and worry about the search engine as a second step.
nathan_f77 9y ago
DuckDuckGo [1] is exactly this - A search engine for hackers (and people who value privacy.) I'm surprised that no-one has mentioned them yet. They're doing fine, but not great, and it doesn't look like they're going to "take over the world". [1] https://duckduckgo.com/ https://duckduckgo.com/
- kukabynd 9y ago
  Who doesn’t know about DuckDuckGo though? This is HackerNews after all.

by jmstfv 9y ago

wayn3 9y ago

google provides qualitative answers. what would be really neat was a search engine that provides quantitative answers or raw data. i'd like to be able to search google for "all the pages that contain a particular javascript snippet", for example. obviously, google has that data. obviously, they dont want to make it available. there are services that offer such things but they suck because their datasets are way too small. on google specifically: the results pages dont matter anymore. nobody clicks anything but the "#1 results" if they click even that. for questions, google very often responds with something like a quora question or some widget of its own, or something else that is really useful - and if its not useful, you try another query instead of clicking through the results pages. results pages are mostly kept for nostalgia it seems.

ljw1001 9y ago
This sounds a lot like Wolfram Alpha, although google does now provide some quantitative results. It works as a calculator and as a units translator for example. that's a good point about only the first few results mattering.
- wayn3 9y ago
  this does not sound like wolframalpha at all. wolframalpha is computer algebra software turned into a web service which allows for searching some papers, pre-written articles and a bunch of interesting, curated numbers. wolframalpha provides qualitative answers to "questions about math". answering the physics problem "whats the speed of light" with "er its fast" is qualitative. saying its 2.99+/- epsilon 10^6 km/s is a quantitative result based on data gathered from some experiment. answering the search engine problem "whats the speed of light" with "2.99+/- epsilon 10^6 km/s" is not a quantitative result. in a search engine contexts, thats just the most likely correct result . qualitative. just a guess. some computer looked at a bunch of data and decided that one of them was probably right based on some popularity heuristic. its like asking an expert who makes an educated guess and claims its right because seniority. a quantitative search engine would be capable of answering questions like "how many sites do X". wolframalpha doesnt do that. its just maple in a browser.
- bloaf 9y ago
  I like WA, but most of the quantitative info is what I'd call "highschool stuff." It is improving all the time, and some of it's math capabilities can reach into advanced undergraduate territory, though. https://www.wolframalpha.com/input/?i=IR+spectrum+of+toluene https://www.wolframalpha.com/input/?i=IR+spectrum+of+toluene http://webbook.nist.gov/cgi/cbook.cgi?ID=C108883&Units=SI&Type=IR-SPEC&Index=19#IR-SPEC http://webbook.nist.gov/cgi/cbook.cgi?ID=C108883&Units=SI&Ty...
futhey 9y ago
Not intended to be a satisfying answer to your question, but publicwww can do that specific task for you, if that is a problem you need solved for yourself.

sparkzilla 9y ago

Google's search results pages try to do all things for everyone: give some links, some news, some data (knowledge graph) and answer some questions. There is room for new entrants to improve on each of these other areas and make that the dominant search method for that channel. For example, making voice answers for Alexa. You may be interested in my essay about Google's search result problems: http://newslines.org/blog/googles-black-hole/ http://newslines.org/blog/googles-black-hole/

FLUX-YOU 9y ago

https://www.shodan.io/ https://www.shodan.io/ is still around

timr 9y ago

My last startup started this way. My advice: don't do it. It's a bad market. First, there are competitors in the space. Some have done an objectively nice job, but they're not blowing up. Second, you don't realize it, but you're not just competing with Google (the search engine) -- you're competing with every product in the Google ecosystem. If you want people to use your new niche search engine, you have to find a value proposition so compelling that people will explicitly choose to use your search, even though every other tool they use is tied back to Google somehow. That's insanely hard. Finally, but most importantly: programmers are a tiny market, they're terribly picky, conservative, and they don't like to pay for things. If you're going to take on the challenge of beating Google in a vertical, at least make it one that's huge or has huge value. Focusing on programmers means that you'll spend a ton of time trying to optimize for a highly technical, low-value use case that doesn't generalize well.

manquer 9y ago
from a pure business model sense it is true programmer focused search is not good space to be in .. Solving problems you face daily and are tired of has always been significant driver of innovation. This is an important and satisfying problem to be working on, not all startups have to become the next google or the next unicorn.
- timr 9y ago
  "not all startups have to become the next google or the next unicorn" No, but you do have to survive. That's hard enough for any company, but it's much harder when you pick a small target market that can't be trivially replicated in other verticals. So sure, pick a problem that you know well, but be realistic about the achievable market size, and don't start a business that requires huge scale to work (i.e. a search engine), when your target market is tiny. (Also, as in all things, beware survivorship bias: just because counterexamples to what I'm saying exist, does not mean that the likelihood of creating one is high. Nor does it mean that the counterexamples are doing as well as you think they are.)

deleted 9y ago

[deleted]

tyingq 9y ago

> What are some "weaknesses" of Google A search engine that also has a near monopoly in browsers and phones is a pretty big moat. Since the bulk of browsing is moving to mobile, it's almost like you have to start with an Android killer and worry about the search engine as a second step.

nathan_f77 9y ago

DuckDuckGo [1] is exactly this - A search engine for hackers (and people who value privacy.) I'm surprised that no-one has mentioned them yet. They're doing fine, but not great, and it doesn't look like they're going to "take over the world". [1] https://duckduckgo.com/ https://duckduckgo.com/

kukabynd 9y ago
Who doesn’t know about DuckDuckGo though? This is HackerNews after all.