4 ms·

SearchYC Performance Enhancements

54 points by chengmi 18y ago

fallentimes 18y ago
I've found SearchYC to be very helpful. I use it quite a bit to look up old information I only vaguely remember or to find people. Glad to see it's only going to get better - congrats Mike and Jerry.
deleted 18y ago
[deleted]
e1ven 18y ago
I always love hearing about behind the scenes updates, thanks for posting. As a more dev-heavy optimization, it might be possible to return cached results (user-search:PG), and then trigger a backend server to refresh the cache, possibly even using ajax to inject the new results. Thanks for the site, it's my top choice for searching HN. As an aside, I'd love to see a standard for integrating site-specific search into a larger search engine. Imagine if you could go to Google, and type in "Site:News.ycombinator.com user:e1ven" and it would send those as post-requests to the search engine designated by robots.txt on the host? That would make search dramatically "deeper", by letting google act as a front-end to each site's results. -CPD
- sam_in_nyc 18y ago
  There's the "OpenSearch" protocol, which just tells your browser "to search this site go to [someurl]?search={SearchTerms}" SearchYC could certainly add this, though I think it may be limited to 1 search only, and not something like: "I have two types of searches, one for everything, and and another for users." But beyond that, OpenSearch it's pretty limited. I expect this area of semi-semantic searching to explode, which is why I'm investing my own time into it. So, hang tight for a couple of weeks, then the problem will be solved :)
vaksel 18y ago
How come pg links to webmynd and not searchyc? Searchyc is a lot more useful, and you don't need to install anything to use it.
- unalone 18y ago
  Isn't webmynd YC? I dunno why, but the guy seems to have a bias towards YCombinator-funded companies.
  - vaksel 18y ago
    I think so, although nowhere on their website do they say this
    - unalone 18y ago
      http://ycombinator.com/faq.html http://ycombinator.com/faq.html It's in the cloud of names.
      - vaksel 18y ago
        I meant on webmynd's website
        unalone 18y ago
        Is there a reason for them to display their funders?
        ryanwaggoner 18y ago
        Credibility.
skmurphy 18y ago
Thanks for developing and maintaining this, I use it several times a week to get context on news.yc posts and comments.
jacquesm 18y ago
searchyc seems to be down for me, is this related to the improvements or is it just that I can't reach it and everybody else can ? edit. thanks to the people that tested it, ok so that seems to be my problem then, here is a traceroute of the last steps before it stops responding: 11 * pos-1-4-0-0-cr01.mclean.va.ibone.comcast.net (68.86.86.29) 251.887 ms 252.502 ms 12 pos-1-11-0-0-cr01.atlanta.ga.ibone.comcast.net (68.86.85.241) 267.698 ms 268.398 ms 234.520 ms 13 pos-1-13-0-0-cr01.dallas.tx.ibone.comcast.net (68.86.85.253) 227.691 ms * 153.851 ms 14 pos-0-13-0-0-cr01.losangeles.ca.ibone.comcast.net (68.86.85.145) 202.909 ms 204.094 ms 204.818 ms 15 * pos-0-14-0-0-cr01.sacramento.ca.ibone.comcast.net (68.86.85.189) 215.876 ms 216.499 ms 16 pos-0-15-0-0-ar01.oakland.ca.sfba.comcast.net (68.86.90.138) 219.741 ms 220.384 ms * 17 po-30-ur02.santaclara.ca.sfba.comcast.net (68.87.226.222) 222.161 ms 223.248 ms 222.742 ms 18 te-8-4-ur04.santaclara.ca.sfba.comcast.net (68.87.226.85) 224.225 ms 224.801 ms 225.416 ms
- Shamiq 18y ago
  I'm able to get in. And it's not taking a minute to load the front page... Good progress, guys!
- weaksauce 18y ago
  http://www.downforeveryoneorjustme.com/searchyc.com http://www.downforeveryoneorjustme.com/searchyc.com up for me.
  - jacquesm 18y ago
    nice link by the way, thanks !
- chengmi 18y ago
  That's strange. We made changes to DNS a few days ago. I suppose it's possible that if you're on the very outskirts of the internet, that it hasn't completely propagated yet.
  - jacquesm 18y ago
    does this help: jam@jam:~$ nslookup searchyc.com Server: 192.168.1.254 Address: 192.168.1.254#53 Non-authoritative answer: Name: searchyc.com Address: 24.7.119.0 isn't a .0 a network address ?
sam_in_nyc 18y ago
I like SearchYC a lot, but when it's slow I tend to use Google site search: http://www.google.com/search?q=site:news.ycombinator.com+search+term http://www.google.com/search?q=site:news.ycombinator.com+sea...
- tokenadult 18y ago
  Google gives more comprehensive results too. But there is a time lag with either. I was mining my bookmarks tonight to find links worth posting, and searched to see if one had been posted before. Search YC indicated it had not been, so I posted it, only to discover to my chagrin that someone else had just submitted it just before I did. (My submission is now deleted, to avoid duplication.) I try to search before I submit, so accurate and timely search is helpful.
  - sam_in_nyc 18y ago
    I haven't done a lot of submitting myself.. I usually just do "Ask HN" type stuff. Hacker News doesn't automatically tell you that the URL you're submitting has already been submitted? Or was the URL slightly different?
    - tokenadult 18y ago
      HN automatically upvotes the previous post if you submit an EXACT duplicate URL. But some webpages live under more than one URL, and not everyone submits the canonical URL.
johnrob 18y ago
Here is a feature request that is relevant only because you are already archiving hackernews: Send a notification to my email __________ whenever: (checkbox) Someone replies to a comment from user _________ (checkbox) User _________ makes a submission/comment (submit button)
- chengmi 18y ago
  Notifications was one of the original motivations for creating SearchYC, but there are a couple of fundamental flaws: 1) How would we know if you've already read a comment/submission? It would get spammy really quick unless we did some kind of periodic digest. We decided that creating RSS feeds for search terms (e.g. username) is a happy medium for this, but we're open to suggestions. 2) New items can take quite a bit of time to get indexed (usually 0-2 hours depending on the number of new items and position in the queue).
- sam_in_nyc 18y ago
  That feature should be part of any internet discussion software, but 99% of the time, it isn't. We're still in the stonge-age of discussion.

54 points by chengmi 18y ago

I've found SearchYC to be very helpful. I use it quite a bit to look up old information I only vaguely remember or to find people. Glad to see it's only going to get better - congrats Mike and Jerry.

deleted 18y ago

[deleted]

e1ven 18y ago

I always love hearing about behind the scenes updates, thanks for posting. As a more dev-heavy optimization, it might be possible to return cached results (user-search:PG), and then trigger a backend server to refresh the cache, possibly even using ajax to inject the new results. Thanks for the site, it's my top choice for searching HN. As an aside, I'd love to see a standard for integrating site-specific search into a larger search engine. Imagine if you could go to Google, and type in "Site:News.ycombinator.com user:e1ven" and it would send those as post-requests to the search engine designated by robots.txt on the host? That would make search dramatically "deeper", by letting google act as a front-end to each site's results. -CPD

sam_in_nyc 18y ago
There's the "OpenSearch" protocol, which just tells your browser "to search this site go to [someurl]?search={SearchTerms}" SearchYC could certainly add this, though I think it may be limited to 1 search only, and not something like: "I have two types of searches, one for everything, and and another for users." But beyond that, OpenSearch it's pretty limited. I expect this area of semi-semantic searching to explode, which is why I'm investing my own time into it. So, hang tight for a couple of weeks, then the problem will be solved :)

vaksel 18y ago

How come pg links to webmynd and not searchyc? Searchyc is a lot more useful, and you don't need to install anything to use it.

unalone 18y ago
Isn't webmynd YC? I dunno why, but the guy seems to have a bias towards YCombinator-funded companies.
- vaksel 18y ago
  I think so, although nowhere on their website do they say this
  - unalone 18y ago
    http://ycombinator.com/faq.html http://ycombinator.com/faq.html It's in the cloud of names.
    - vaksel 18y ago
      I meant on webmynd's website
      - unalone 18y ago
        Is there a reason for them to display their funders?
        ryanwaggoner 18y ago
        Credibility.

skmurphy 18y ago

Thanks for developing and maintaining this, I use it several times a week to get context on news.yc posts and comments.

jacquesm 18y ago

searchyc seems to be down for me, is this related to the improvements or is it just that I can't reach it and everybody else can ? edit. thanks to the people that tested it, ok so that seems to be my problem then, here is a traceroute of the last steps before it stops responding: 11 * pos-1-4-0-0-cr01.mclean.va.ibone.comcast.net (68.86.86.29) 251.887 ms 252.502 ms 12 pos-1-11-0-0-cr01.atlanta.ga.ibone.comcast.net (68.86.85.241) 267.698 ms 268.398 ms 234.520 ms 13 pos-1-13-0-0-cr01.dallas.tx.ibone.comcast.net (68.86.85.253) 227.691 ms * 153.851 ms 14 pos-0-13-0-0-cr01.losangeles.ca.ibone.comcast.net (68.86.85.145) 202.909 ms 204.094 ms 204.818 ms 15 * pos-0-14-0-0-cr01.sacramento.ca.ibone.comcast.net (68.86.85.189) 215.876 ms 216.499 ms 16 pos-0-15-0-0-ar01.oakland.ca.sfba.comcast.net (68.86.90.138) 219.741 ms 220.384 ms * 17 po-30-ur02.santaclara.ca.sfba.comcast.net (68.87.226.222) 222.161 ms 223.248 ms 222.742 ms 18 te-8-4-ur04.santaclara.ca.sfba.comcast.net (68.87.226.85) 224.225 ms 224.801 ms 225.416 ms

Shamiq 18y ago
I'm able to get in. And it's not taking a minute to load the front page... Good progress, guys!
weaksauce 18y ago
http://www.downforeveryoneorjustme.com/searchyc.com http://www.downforeveryoneorjustme.com/searchyc.com up for me.
- jacquesm 18y ago
  nice link by the way, thanks !
chengmi 18y ago
That's strange. We made changes to DNS a few days ago. I suppose it's possible that if you're on the very outskirts of the internet, that it hasn't completely propagated yet.
- jacquesm 18y ago
  does this help: jam@jam:~$ nslookup searchyc.com Server: 192.168.1.254 Address: 192.168.1.254#53 Non-authoritative answer: Name: searchyc.com Address: 24.7.119.0 isn't a .0 a network address ?

sam_in_nyc 18y ago

I like SearchYC a lot, but when it's slow I tend to use Google site search: http://www.google.com/search?q=site:news.ycombinator.com+search+term http://www.google.com/search?q=site:news.ycombinator.com+sea...

tokenadult 18y ago
Google gives more comprehensive results too. But there is a time lag with either. I was mining my bookmarks tonight to find links worth posting, and searched to see if one had been posted before. Search YC indicated it had not been, so I posted it, only to discover to my chagrin that someone else had just submitted it just before I did. (My submission is now deleted, to avoid duplication.) I try to search before I submit, so accurate and timely search is helpful.
- sam_in_nyc 18y ago
  I haven't done a lot of submitting myself.. I usually just do "Ask HN" type stuff. Hacker News doesn't automatically tell you that the URL you're submitting has already been submitted? Or was the URL slightly different?
  - tokenadult 18y ago
    HN automatically upvotes the previous post if you submit an EXACT duplicate URL. But some webpages live under more than one URL, and not everyone submits the canonical URL.

johnrob 18y ago

Here is a feature request that is relevant only because you are already archiving hackernews: Send a notification to my email __________ whenever: (checkbox) Someone replies to a comment from user _________ (checkbox) User _________ makes a submission/comment (submit button)

chengmi 18y ago
Notifications was one of the original motivations for creating SearchYC, but there are a couple of fundamental flaws: 1) How would we know if you've already read a comment/submission? It would get spammy really quick unless we did some kind of periodic digest. We decided that creating RSS feeds for search terms (e.g. username) is a happy medium for this, but we're open to suggestions. 2) New items can take quite a bit of time to get indexed (usually 0-2 hours depending on the number of new items and position in the queue).
sam_in_nyc 18y ago
That feature should be part of any internet discussion software, but 99% of the time, it isn't. We're still in the stonge-age of discussion.