6 ms·
PourOver: A library for simple, fast filtering and sorting in the browser
- anarchy8 12y agoAnyone have a demo?
- dmix 12y agoThis page needs a giant "demo" button near the top. The examples are all code.
- paulcnichols 12y agoReminds me of crossfilter (http://square.github.io/crossfilter/ http://square.github.io/crossfilter/) by square. It has a killer demo, however.
- esmooov 12y agoCrossfilter was absolutely the inspiration. I just wanted to make a Crossfilter that worked with the arbitrary boolean composition, could be dynamically added to and updated, and supported some of the less numerical patterns I was encountering.
- victorhooi 12y agoCoolies =). Would you ever add some of the Crossfilter functionality, to make it easy for people to plug in their own data to PourOver?
- esmooov 12y agoI would absolutely like to add a Crossfilter Filter type. I'd even accept a pull request ... ;)
- barkingcat 12y agoMore details at http://open.blogs.nytimes.com/2014/04/16/introducing-pourover-and-tamper http://open.blogs.nytimes.com/2014/04/16/introducing-pourove... There are a few links to projects at the nyt that has used these two libraries.
- danso 12y agoSo I visited one of the PourOver examples, this Academy Awards fashion feature published earlier this year: http://www.nytimes.com/interactive/2014/02/02/fashion/red-carpet-project.html http://www.nytimes.com/interactive/2014/02/02/fashion/red-ca... I opened the dev tools to inspect the traffic and code, and this pops up in the console: 0000000 000 0000000 111111111 11111111100 000 111111111 00000 111111111111111111 00000 000000 000 1111111111111111111111111100000 000 000 1111 1111111111111111100 000 000 11 0 1111111100 000 000 1 00 1 000 000 00 00 1 000 000 000 00000 1 000 00000 0000 00000000 1 00000 11111 000 00 000000 000 11111 00000 0000 000000 00000 00000 000 10000 000000 000 0000 000 00000 000000 1 000 000 000000 10000 1 0 000 000 1000000 00 1 00 000 000 1111111 1 0000 000 000 1111111100 000000 000 0000 111111111111111110000000 0000 111111111 111111111111100000 111111111 0000000 00000000 0000000 NYTimes.com: All the code that's fit to printf() We're hiring: http://nytimes.com/careers ....You sneaky audience-targeting bastards
- chimeracoder 12y ago> NYTimes.com: All the code that's fit to printf() This gave me a chuckle. I was at a hackathon that the NYTimes sponsored a few years ago, and my group used almost this exact joke for our project, an ncurses reader for browsing the New York Times in the terminal[0]. I don't know who thought to put this in the source code, but I'd find it very amusing if they got the inspiration from hidden slogan from a hackathon project. Kind of like the old competitions in the 50s/60s to create company jingles. [0] https://github.com/ChimeraCoder/ncnyt https://github.com/ChimeraCoder/ncnyt
- JangoSteve 12y agoIt seems similar to our Dynatable plugin [1], which is basically the functionality of this plugin with some additional table-read/write functions included. The main difference being that this library depends on underscore, while Dynatable depends on jQuery (which is mainly used for its browser compatibility functions). Given both library's emphasis on speed, it looks like I have something to benchmark against! [1] http://www.dynatable.com http://www.dynatable.com
- esmooov 12y agoThis looks really cool. I wouldn't say PourOver is focuses on speed as much as focused on being fast enough to get 60fps for 100k items. Cheers!
- bestest 12y agoBenchmarks comparing PourOver to Backbone would be nice. Anyone?
- esmooov 12y agoThere are some benchmarks against the base underscore filter methods in the tests. These are what Backbone uses by default. However, the real utility of PourOver is less in its raw speed -- I'm sure that it could be improved by smarter folks than I -- but in the patterns it abstracts. I hope you find it useful.
- nathanhammond 12y agoI much prefer a basic construct that makes it simple to do filtering and sorting that can be simply extended to any complexity. Sorting and filtering are really nothing more than set manipulation (which they state themselves) so with simple data binding this becomes a trivial exercise to build an impressive client-side search. In Ember that might look like this: Ember.ArrayController.extend({ filterA: Ember.computed.filter('fieldName1', function comparator() {}), filterB: Ember.computed.filter('fieldName2', function comparator() {}), joined: Ember.computed.union('filterA', 'filterB'), filtered: Ember.computed.uniq('joined'), sorted: Ember.computed.sort(function comparator() {}) });
- esmooov 12y agoThis is exactly what PourOver offers. You can chain filter results to any boolean complexity, as well extend the default filter types to optimize indexing and caching. Indeed, PourOver was trivial to make, an outgrowth of the very pattern you define above. PourOver is just an attempt to scrap that boilerplate, allow for the queries to be indexed, combined with sorts, and automatically rebuild when the collection changes.
- nathanhammond 12y agoThinking about it more, I believe that my visceral reaction was to the imperative nature of the library. I was imagining trying to code something that generated a filter (e.g. PourOver.makeExactFilter("mythology", ["greek","norse"]);) and it seems like it would be unnecessarily complex without data-binding propagation/invalidation. If I wanted to add "roman" mythology to that filter I would imagine something like: var mythologies = ["greek", "norse"]; // ... create a collection with filters mythologies.push("roman"); PourOver.makeExactFilter("mythology", mythologies); But you don't get that last statement for free, nor the one where you join it into a collection, nor do I see a way to replace the previous mythologies filter. Any time you need to reprocess a filter you've got to run it through a series of imperative actions triggered from one of the events, and possibly throw away the collection and regenerate it (if you can't remove disjoint filters). I would be pushing for a Object.observe/dirty checking/get-set version of this to take PourOver to the next level of utility. (?/Angular/Ember)
- simonsarris 12y agoHalf the comments so far are looking for a non-code demo, and I imagine they mean something visual, so here's a bare-bones visualized version of the Basic PourOver sample code: http://gojs.net/temp/pourover.html http://gojs.net/temp/pourover.html It just takes the queries and uses the resulting data to make some nodes in GoJS (Disclaimer: a Diagramming library I develop. Not free, but easy to set up with this kind of data and see stuff fast). If I had more time I'd make it prettier. The results are just visual representations of the data results getting filtered. Its very easy to take PourOver's example collections and data-bind some stuff to them (color of the nodes is data-bound to monster gender, etc). I'm sure it can't be hard to do the same in other data-bound visualization libraries. This is very cool. I'll try to make a much prettier example tonight.
- esmooov 12y agoOh this is great! I really should port something like this to the docs.
- Neff 12y agoAnd here I thought I would be reading about the App.net PourOver posting service[0], which the NY Times Opinion account uses for posting[1] [0]: https://directory.app.net/app/255/pourover/ https://directory.app.net/app/255/pourover/ [1]: https://alpha.app.net/nytopinion https://alpha.app.net/nytopinion
- esmooov 12y agoHi, all. A lot of folks are asking for a demo and, you're right. I should have included one. My apologies. I'll get to working on one as soon as I can. In the meantime, I encourage readers to check out the source for http://www.nytimes.com/interactive/2014/02/02/fashion/red-carpet-project.html http://www.nytimes.com/interactive/2014/02/02/fashion/red-ca... That's probably the clearest "demo" of PourOver at the moment. More to come!
- j_s 12y agoThanks for doing your 'open source gardening'¹ so exuberantly! I didn't see any contact info on your profile to let you know that the proportion of folks here using Ghostery² or Disconnect³ is probably higher than normal. Installing one will reveal how things break when JavaScript libraries [usually orthogonal to page functionality] are blocked. -- ¹https://news.ycombinator.com/item?id=7593242 https://news.ycombinator.com/item?id=7593242 ²https://www.ghostery.com/ https://www.ghostery.com/ ³https://disconnect.me/ https://disconnect.me/
- b0z0 12y agoI love how New York Times has the coolest devs. Anyone here who works there? What's it like?
- jashkenas 12y agoI do. It's pretty fun. This might be a bit embarrassing ... but a couple of years ago, Dan Sinker came by and asked a few of the developers in the newsroom to talk about exactly that question. Here's me: https://www.youtube.com/watch?v=k3HanxEFFlk https://www.youtube.com/watch?v=k3HanxEFFlk Here's David Nolen (although he won't be around for much longer ;) https://www.youtube.com/watch?v=qOdoCqoqXlU https://www.youtube.com/watch?v=qOdoCqoqXlU Here's Tiff Fehr: https://www.youtube.com/watch?v=8GOWLNWifxc https://www.youtube.com/watch?v=8GOWLNWifxc
- akilism 12y agonice...but yikes that seems like an intimidating place to apply for a job...you guys are all heavy hitters...
- esmooov 12y agoI totally wouldn't be intimidated. We all just love what we do. The best way to get a job is often to get to know the folks on the team, hack with them on projects. I'd really hate to work at a place that I felt was exclusive.
- akilism 12y agoOh I didn't mean to imply it was exclusive, just that I'd feel like I was out of my league. Like I've built a few things with D3 but I still find myself getting confused sometimes, and I've gone through some of David Nolen's clojurescript posts / tutorials and I'm still not 100% clear on the advantages of core.async...but then again I don't use them all that often either.
- esmooov 12y ago
- deleted 12y ago[deleted]
- rpedela 12y agoOff-topic, but related. Does anyone know of any JS libraries that allow for Excel-like interaction? However I am looking for something that just implements the presentation layer. The problem I am running into is that the libraries that do exist conflate the presentation and data layers and try to do everything for you. In other words, they make the assumption that all data will be downloaded to the client and then manipulated on the client. These libraries do not fit my use case.
- cfeduke 12y agoI am actually in the middle of an implementation of a grid view myself; I'm giving Handsontable[1] a whirl and going to look at using pourover for filtering my dataset. Definitely meets your Excel-like interaction though not certain that it doesn't try to do everything for you since I haven't made it that far yet. (Looks promising at least.) 1. http://handsontable.com http://handsontable.com
- kylebrown 12y agoNoice! Btw, the docs page is acting funky on an iPad (iOS 7.1). The layout seems to be alternating between mobile and desktop with each pan/scroll event. [ps: only happens in landscape orientation. portrait is unaffected]
- nashequilibrium 12y agoIf you going to handle that many obejcts in the browser wouldn't you use something like pouchdb? http://pouchdb.com/ http://pouchdb.com/ Also "Pagination strategies with PouchDB" http://pouchdb.com/2014/04/14/pagination-strategies-with-pouchdb.html http://pouchdb.com/2014/04/14/pagination-strategies-with-pou...
- earless1 12y agoI've used the https://mixitup.kunkalabs.com/ https://mixitup.kunkalabs.com/ library for similar functionality in the past. I am not sure how well it works with hundreds of thousands of items, but it does work well for the 600ish items that I am using it for.
- daemonk 12y agoSimilar to crossfilter?
- drv 12y agoI wonder if the misspelling is intentional. (The idiom is "pore over".)
- dandelany 12y agoI'm sure it is - "pour over" is a way of making coffee, and its sister project is called "Tamper" which is also coffee related. The "pore over" double meaning is clever though.
- drv 12y agoIndeed, looking at the logo again, it appears to be a pour-over brewer - clever. (I am a coffee neophyte and wasn't aware of this brewing method.)
- reshambabble 12y agoThis is awesome, and exactly the type of thing I've been looking for. Thanks so much for sharing! Can't wait to try this out.
- mbesto 12y agoIs this basically what Google Refine[0] does with their facet filters? (but obviously open source and allowing anyone to build their own platform) [0] - http://openrefine.org/ http://openrefine.org/
- grumblestumble 12y agoIt would be a great move to isolate the Views/UI part of this out from the rest of the code base, which would make it more usable inside other MV* frameworks like Angular or Ember.
- alixaxel 12y agoHaving been recently exposed to the benefits (specially performance-wise) of lodash vs underscore, I wonder why it depends on the later.
- jdd 12y agoFrom a glance of the source it doesn't look like PourOver is doing any heavy lifting with Underscore. PourOver works lodash.underscore build of Lo-Dash without issue though because it's not doing any heavy lifting you probably won't see a perf difference.