13 ms·
Hacker News Search
- krzysz00 15y agoThis should cut down on duplicate submissions. That is, IF people remember to use it.
- 6ren 15y agosubmission could automatically give search results on the title and/or url before confirming (like SO).
- petercooper 15y agoMetaFilter has been doing this for years too and it works ridiculously well. You have to see a list of potential matches before you can commit your post. So +1 to that.
- ColinWright 15y agoPeople won't use it. Decent search has been available via SearchYC for years, and no one seemed to use it to prevent duplicate and/or similar submissions. Further, if you submit via the bookmarklet there is a huge incentive not to search.
- jaxonrice 15y agoThis was by far the single feature that I wanted most for HN. Thank you
- tokenadult 15y agoThe feature that jumps out at me is that I can see comment karma scores for all the comments that I search up with a keyword search. Evidently the upvote/downvote buttons I see next to the comments in the search results don't change the comment karma scores.
- woodrow 15y agoI noticed the same thing, especially for current discussions. But then it changed. It looks like there's now a 5-day threshold for karma point exposure.
- revorad 15y agoHow is ThriftDB helping HN search? It'd be interesting to hear the details.
- andres 15y agoHNSearch consists of a simple javascript webapp hosted at hnsearch.com which sends ajax requests directly to ThriftDB. ThriftDB returns items in the search response which makes it ideal for this type of architecture. For a more detailed explanation checkout the API docs: http://www.hnsearch.com/api http://www.hnsearch.com/api And here's a sample search response from ThriftDB: http://api.thriftdb.com/api.hnsearch.com/items/_search?q=facebook&pretty_print=true http://api.thriftdb.com/api.hnsearch.com/items/_search?q=fac...
- revorad 15y agoSo is HN data now being stored in ThriftDB? Or is it being replicated on your server? If so, what's the lag?
- andres 15y agoWe wrote a crawler to download data from HN, parse it into JSON, and upload it to ThriftDB. The webapp at hnsearch.com sends requests directly to ThriftDB. There's a ~15min lag between HN and HNSearch.
- evangineer 15y agoShould be a good test of the ThriftDB technology. Too bad the HN discussion about ThriftDB didn't come up in my test search, found it via Google instead. http://news.ycombinator.com/item?id=2581652 http://news.ycombinator.com/item?id=2581652
- bnewbold 15y agoReally? That exact story is result #4 searching for "thriftdb" (http://www.hnsearch.com/search#request/all&q=thriftdb http://www.hnsearch.com/search#request/all&q=thriftdb) and the only result when searching for stories (http://www.hnsearch.com/search#request/submissions&q=thriftdb&start=0 http://www.hnsearch.com/search#request/submissions&q=thr...) These results are as of 1pm EST on Saturday, as this very comment thread gets indexed the ordering may change...
- evangineer 15y agoAh, I had JavaScript disabled. Once enabled, I got good search results!
- evanrmurphy 15y agoDo you typically browse with JavaScript disabled? I would think that these days it would seriously change the browsing experience and many sites would be unusable. Just curious. :)
- evangineer 15y agoI have JavaScript disabled by default, and selectively enable it for the sites where I feel it is worthwhile. I'm a longtime NoScript user, and these days I use NotScripts with Chrome.
- Wilya 15y agoThe app flow is a bit unnatural. You get to it directly from HN, but the only back link to the main page is hidden in the footer. That's still a great thing to have. Simple and efficient.
- andres 15y agoGood point. The UI is designed for someone coming directly to hnsearch.com. I'll see if I can tweak it for people coming from HN.
- akikuchi 15y agoI search HN a lot, so am excited by this native implementation. I would be curious to know more about how the "relevance" sorting algorithm works though. When I did a test search for "domain registrar," for example, the top result was a comment with a score of -4. It seems like there are many ways to implement that feature, so would would be quite interested to hear more if the creators were able to share some thoughts on the general "relevance" problem.
- andres 15y agoWe're using the HN hotness algorithm with some points boosts to surface older items. For a full explanation of the ranking algorithm check out the API docs: http://www.hnsearch.com/api http://www.hnsearch.com/api
- tptacek 15y agoI like hnsearch a lot, but I'd like us to take a second to thank whoever was running SearchYC, which for the past couple years has been practically indispensable in keeping up with this community.
- kylec 15y agoSearchYC was also more fully featured, with lists of top users based on points per submission, points per comment, etc. - at least until pg removed comment points from the site.
- andres 15y agoThat could be a good app for the HNSearch API contest: http://www.hnsearch.com/contest http://www.hnsearch.com/contest
- andres 15y agoDefinitely! I'm actually trying to get in touch with the SearchYC folks. Recently they put up a link to HNSearch which was very nice of them.
- Sukotto 15y agoI miss searchyc and feel sad that they went compleatly dark with no real explanation.
- 15y ago
- mrvc 15y agoHow about opening up an API and letting the hackers here have a bash at it? It would be a fun challenge to see who can come up with the best and most useful solution :)
- deleted 15y ago[deleted]
- bnewbold 15y agoThere is an API, linked from the bottom of every page: http://www.hnsearch.com/api http://www.hnsearch.com/api For the next two weeks there's even an API contest going on! http://www.hnsearch.com/contest http://www.hnsearch.com/contest
- daniel-cussen 15y agotypo: "versionso" should be "version_so"
- dnlk 15y agoany chance of integrating the search box into the top bar? i use some auto pagerize plugins that sort of get in the way. i know i could just turn it off, but that would brake its purpose. also, i suppose this would help with general visibility of the search box, as most sites have theirs on top of their pages! but it's definitely great to see search build into the site! will make finding stuff on hn a lot more convenient.
- peng 15y agoDo you use Chrome? You can list this search engine as a custom search engine in Preferences > Basics > Manage Search Engines... Add a new search engine -> Hacker News Keyword -> hn Url with %s in place of query -> http://www.hnsearch.com/search#request/all&q=%s http://www.hnsearch.com/search#request/all&q=%s You can do the same thing with a keyword bookmark in Firefox. I've been searching HN all this time with a custom Google search including "site:news.ycombinator.com". I'll have to see if this is better. Google offers results filtered by date, which is really nice, since technology moves so incredibly fast.
- bzupnick 15y agoyeh but google wouldnt bring the higher ranked comments or stories to the top, which i think is a great feature
- andres 15y agoYou can add HNSearch as a custom search engine in Firefox as well. Just visit hnsearch.com, click on the favicon in the browser's search box and select "Add HNSearch". For the Chrom custom search engine it would be better to use the non-javascript url: http://www.hnsearch.com/search?q= http://www.hnsearch.com/search?q= That will get redirected automatically and won't change with the webapp implementation. You can filter by date/points with HNSearch.
- tokenadult 15y agoI can't reproduce the Firefox steps you mention. I click on the favicon in the browser's search box and I don't get a chance to select "Add HNSearch," but rather a message saying "This website does not supply identity information."
- amichail 15y agoThere should be a prominent link back to news.combinator.com from HNSearch pages.
- pg 15y agoI should add that HNSearch has an API http://www.hnsearch.com/api http://www.hnsearch.com/api which should be a lot faster than crawling HN itself to get the same data.
- k7d 15y agoThis is cool, but the API looks a bit cryptic. Is there a way to simply lookup an item by URL? Edit: Nevermind, figured it out: http://api.thriftdb.com/api.hnsearch.com/items/_search?pretty_print=true&filter[fields][url][]=http%3A%2F%2Fycombinator.com%2Fnewsnews.html%234jun11 http://api.thriftdb.com/api.hnsearch.com/items/_search?prett...
- news-yc 15y agoOr, by "id"? I looked, but I wasn't able to figure out, for example, how I'd get pg's top-level comment here from the id "2619846". I was able to find it with this search, but it requires a second-part to the "_id" that I don't know where to find: http://api.thriftdb.com/api.hnsearch.com/items/_search?q=&filter[fields][_id]=2619846-a30f9&pretty_print=true http://api.thriftdb.com/api.hnsearch.com/items/_search?q=... (Trying to filter for "id" instead of "_id" throws an error, for some reason.)
- andres 15y agoThe `_id` attribute for each item is its submission id plus a signature. ThriftDB references objects by their unique `_id` attributes. The `id` attribute on the other hand is an item's submission id. It's not indexed though so you can't do lookup by id.
- milkshakes 15y agocould you please index the submission ids? i can think of a few reasons this would be useful
- andres 15y ago
- stevenj 15y agoWill you move the search box to the top right hand corner? Perhaps just below my username?
- injekt 15y ago+1 for this. I use AutoPatchWork so it's not very accessible (I can imagine this the same for those not using AutoPatchWork too, though)
- arctangent 15y agoPlease do this. Having the search box at the bottom of the page makes no sense to me.
- evanrmurphy 15y agoOne downside of your suggested placement is it would push down page content about 20px. But I agree that the footer is sub-optimal.
- stevenj 15y agoOr actually, maybe if it was just in the top bar -- between "submit" and my username.
- cheez 15y agoIs there any search engine that returns the right results for the term "C++"?
- andres 15y agoIt's on the todo list.
- util 15y agoBoth Google and Bing seem to handle it: http://www.google.com/search?q=site%3Ahttp%3A%2F%2Fnews.ycombinator.com%2F+c%2B%2B http://www.google.com/search?q=site%3Ahttp%3A%2F%2Fnews.ycom... http://www.bing.com/search?q=site%3Ahttp%3A%2F%2Fnews.ycombinator.com%2F+c%2B%2B http://www.bing.com/search?q=site%3Ahttp%3A%2F%2Fnews.ycombi... Are there some particular cases where you see them screwing up?
- cheez 15y agoYeah sorry, I realize Google and Bing have fixed those issues but for the longest time, even Google didn't do it!
- rbreve 15y agoThe search box should be on the top
- evanrmurphy 15y agoI agree, but where would you fit it in? I'm trying to figure that out as well.
- duck 15y agoTo who ever runs it: 'Yeserday' is spelled wrong. :)
- brupm2 15y agoWow, what an achievement.
- vicngtor 15y agoPardon me and I don't want to come off as ignorant, what is the big deal with this? I have been using Google to search hackernews articles by querying `<subject> site:news.ycombinator.com` and it has worked wonders. Since hackernews is all publicly available, Googlebots must have indexed every single page of this site. And we all know how good Google has been with ranking and relevance. Why did HN decide to reinvent the wheel? Why didn't HN use the Google Custom Search plugin? (I am just curious to know and not criticizing.)
- simonw 15y agoThe problem with Google Search for a site like Hacker News is that it doesn't have the same level of understanding of the metadata that makes up the site. A good example is sort-by-date (which Google can approximate based on the date something was first spotted by its crawlers, but it's not nearly as accurate as having access to the "date" field in the underlying data structures) - another is "just search comments by this username".
- andres 15y agoHNSearch is an Octopart/ThriftDB project to test out our search technology and give back to the HN community. Google has access to the link structure of the internet so it's great for macro searches. However, if you want to do a site-specific micro search then it usually helps to have access to the underlying metadata (e.g. points, karma, timestamps).
- ColinWright 15y agoSearchYC allowed results to be sorted by date of submission, allowed searches by username and points, and other aspects that Google simply doesn't understand and can't provide.
- revorad 15y agoGoogle is great for simple searches, but fails utterly for any site-specific filtering of results.
- deleted 15y ago[deleted]
- deleted 15y ago[deleted]
- deleted 15y ago[deleted]
- jasonshen 15y agoI'm really excited to start using this! Also - "built outside of Octopartitself" needs a space.
- ZackOfAllTrades 15y agoPG: Need to change the link in the bottom right of the page that says search to hnsearch instead of google with site:...
- Typhon 15y agoFinally. Could we get one on the arc forum as well ?
- evanrmurphy 15y agoThank you! I was wondering exactly the same thing: http://news.ycombinator.com/item?id=2620297 http://news.ycombinator.com/item?id=2620297
- sigil 15y agoThriftDB is the software they had to write to do Octopart's electronic parts searches; no existing software was up to the task. Can the Octopart guys comment on why no existing software was up to the task? Also, I'm curious if there's something that makes ThriftDB a particularly good choice for HN search as well.
- rdoherty 15y agoI agree, I'm not sure what's so different about HN's information where something like Lucene, Sphinx or any other full text search tool couldn't handle.
- andres 15y agoThriftDB uses Solr internally so I don't want to give the impression that we've built new full-text indexing technology. Our goal with ThriftDB was to solve the search infrastructure problem. We wanted to give developers access to fast, scalable, on-demand, cloud-based search. Here's a link to the ThriftDB docs in case you'd like to learn more: http://www.thriftdb.com/documentation http://www.thriftdb.com/documentation
- andres 15y agoA search engine has many moving parts and we found that the existing technology worked but with great pain. Lucene/Solr is great for search but it's useless without a fast datastore. Additionally, you can't expose it to the open internet so you need to build a REST API wrapper around the DB/index. One particularly acute problem we had at Octopart was search across a frequently changing schema. We have 15M parts in our database so it was a real pain to change the schema. ThriftDB uses the Thrift serialization protocol internally to maintain a flexible schema so you can change the schema independently of the underlying data. The current implementation of ThriftDB takes advantage of solr in smart ways to simplify app development dramatically. At Octopart we found that we were spending a lot of time building custom search solutions. Now we're using the same backend technology to search for electronic parts on Octopart and comments on Hacker News. Using ThriftDB for search is like going from a compiled language to a scripting language. It takes care of a lot of issues so you can focus on app development. Hope that explanation helps! Please let me know if you have more questions.
- mrjbq7 15y agoIt's a shame that, rather than embracing SearchYC which has existed and filled a needed feature for several years, you chose to use this as a plug for one of your companies...
- chimingin 15y agoo the lulz- pg today: "Sorry this took so long. For a while we hoped to implement our own search, but it became apparent that would take a lot of work to do right" pg 392 days ago: "The explanation is a lot less involved than your question: what makes users happy is not features, but the quality of the submissions and comments. So I focus on the latter instead of the former. The result is a spartan site with good content. There's a YC funded startup that may solve the problem. If they do I'll use them. But frankly the issue is not at the top of my list. This is a classic example of how one should give users what they want, not what they say they want. Lots of people say they want search, but I would be suprised if there was a single user who'd left HN because it lacked search. Whereas if I let the frontpage get filled up with crap, or the comment threads filled up with mean or stupid comments, people would start leaving pretty quickly. So almost all the time I spend thinking about HN is spent thinking about how to avoid that." -source: http://news.ycombinator.com/item?id=1329334 http://news.ycombinator.com/item?id=1329334 To your credit you did come back with the YC investment product, but please don't disguise the fact that you didn't think search was necessary in the first place. Seems pumping up your investment was the impetuous that changed your mind. Nothing wrong with that at all, just let's keep the story factual. This is a classic example of how one should give users what increases value of a portfolio sister company and dressing it up as something else, not what they say they want.
- lt 15y agoComment scores are back! (in a convoluted way, through the API). I can see that at this time that the top comment of this page currently (tptacek's) has 58 points: http://api.thriftdb.com/api.hnsearch.com/items/_search?filter[fields][username][]=tptacek&sortby=create_ts%20desc&pretty_print=true http://api.thriftdb.com/api.hnsearch.com/items/_search?filte...
- lt 15y agoLooks like it's been changed. Points seems to be always null through the API now. edit: just removed from the search. still here: http://api.thriftdb.com/api.hnsearch.com/items/2619811-02e34?pretty_print=true http://api.thriftdb.com/api.hnsearch.com/items/2619811-02e34... expect to be gone soon.
- brndnhy 15y agoHow many more posts do we need expressing our displeasure that searchyc has gone AWOL? Wake me up when there's an official status update.
- fara 15y agoWhy at the bottom? I'm sure it's supposed to be more important than the footer. You could use SmashingMagazine resources when it comes to UX and design. It looks like you made almost all the frequent mistakes: Placing the search box at the bottom of the page, or hiding it in the navigation menu. Making the input field too short; users are forced to use short, imprecise queries, because longer queries would be hard and inconvenient to read. Making the submit button too small, so that users have to point the mouse very precisely. Making the search box hard to find. Source: http://www.smashingmagazine.com/2008/12/04/designing-the-holy-search-box-examples-and-best-practices/ http://www.smashingmagazine.com/2008/12/04/designing-the-hol...
- evanrmurphy 15y agoThis is a wonderful gain in functionality. Any chance pg and Octopart can implement search for pg's other forum? (Relevant HN thread at http://news.ycombinator.com/item?id=2620297 http://news.ycombinator.com/item?id=2620297 .)
- dawie 15y agoAny reason for the search to be at the bottom? PG: Can we please have it at the top, after submit?
- gnosis 15y agoInterface improvement suggestions: When the search results page comes up, the search box is active. This makes any keystrokes I type appear in the search box instead of being passed on to my browser to do things like scroll down the page of results. This is pretty annoying, and makes me have to click outside the search box before I can use the keyboard to scroll down the page, etc. So please make it so that the search box does not have focus when the search results are returned. The other suggestion I have is to allow people to use their keyboards to go to the next page of search results. In Opera, I do this simply by hitting the space bar once I get to the bottom of the page. This is how Google's interface works. In general, I think you can't go wrong by studying the design of their search interface carefully and copying it.
- chimingin 15y agowhy was my comment killed?? o the lulz- pea gee today: "Sorry this took so long. For a while we hoped to implement our own search, but it became apparent that would take a lot of work to do right" pea gee 392 days ago: "The explanation is a lot less involved than your question: what makes users happy is not features, but the quality of the submissions and comments. So I focus on the latter instead of the former. The result is a spartan site with good content. There's a YC funded startup that may solve the problem. If they do I'll use them. But frankly the issue is not at the top of my list. This is a classic example of how one should give users what they want, not what they say they want. Lots of people say they want search, but I would be suprised if there was a single user who'd left HN because it lacked search. Whereas if I let the frontpage get filled up with crap, or the comment threads filled up with mean or stupid comments, people would start leaving pretty quickly. So almost all the time I spend thinking about HN is spent thinking about how to avoid that." -source: http://news.ycombinator.com/item?id=1329334 http://news.ycombinator.com/item?id=1329334 To your credit you did come back with the YC investment product, but please don't disguise the fact that you didn't think search was necessary in the first place. Seems pumping up your investment was the impetuous that changed your mind. Nothing wrong with that at all, just let's keep the story factual. This is a classic example of how one should give users what increases value of a portfolio sister company and dressing it up as something else, not what they say they want.
- chimingin 15y agowhy am I being censored about pointing to this?: http://news.ycombinator.com/item?id=1329334 http://news.ycombinator.com/item?id=1329334
- alanh 15y agoWould love to see the search box placed at the top-right of the site. Kind of a universal standard these days (and would mean no scrolling required to search). The HTML5 `type=search` attribute would be cool as well :) Edit: Additionally, use of `placeholder=Search` and the ditching of the label shouldn’t be terribly controversial here. Given the audience, I have to imagine most users will see the placeholder text, no polyfill required.
- nabaraj 15y agoHow about adding the search box in the top?
- thomasswift 15y agoI put together a simple safari extension that puts the bar after the submit link in the header. (not sure on the amount of safari users, but maybe a chrome version?) http://1821design.com/HNTopBarSearchField/ http://1821design.com/HNTopBarSearchField/
- nhebb 15y agoI don't know if it was left intentionally, but the search link in the footer menu bar still goes to google instead of hnsearch.
- chimingin 15y agowhy do my comments to a past discussion on search get auto killed? using the new search box searching for the title, it still comes up, but when I make a critical point and link to it my post goes straight to dead?? try searching for : What's the deal with search on HN seems there was a very different attitude towards it then...
- seanp2k 15y agoFirst thought: Seriously, you guys couldn't just make SQL or something lucene-based work for this (like elasticsearch) ? Second thought: Hmm, ooh well, I just use google with site: news.ycombinator.com EDIT: It annoys me when people hate on SQL and act like it's OMGSOSLOW. I would agree that fulltext search BY ITSELF isn't amazing, but you can use something like Sphinx to wrap it and support awesome things like Soundex / Metaphone fuzzy matching. It's also stupid easy to set up. The NoSQL movement is, IMO, largely misguided. If you need to shard data and scale ridiculously, something like Lucene can probably do it. If you need bigger than that, use memcached /redis and shard and map/reduce queries....that makes sense, but NoSQL doesn't seem like a great idea for PRIMARY data storage. It seems like you better have amazing backups :)
- blackstag 15y agoI noticed a good percentage of the spam I received originated from SearchYC links (via links placed on HN under a different account name). I would be interested to know if HN also receives less spam with the removal of SearchYC. Does HN track stats on this?
- dang 2y ago(Original URL was http://ycombinator.com/newsnews.html#4jun11 http://ycombinator.com/newsnews.html#4jun11.)