3 ms·
As other comments here point out, there are significant privacy issues with sending every visited URL to a "trusted" server to check for comments. There are al
by nkurz 9y ago
As other comments here point out, there are significant privacy issues with sending every visited URL to a "trusted" server to check for comments. There are also load issues for the server that is getting all the unwanted requests. So if doing this, you'd probably want at least the initial lookup to be local.
So, can this be done with an initial download of some small number of megabytes, then incremental updates of a few kilobytes as often as desired? I think so.
Guessing at numbers (see other comments here), there are probably fewer than 1,000,000 URL's that have comments associated with them on HN. For each, you store hash of the canonicalized URL and a comment count. Collisions aren't deadly, so you could probably get down to 8B per hash (7B for hash, 1B for count). Updates can be a list of all new and revised URL-hash-counts since a given date.
Lookup the hash of the URL of each visited page with binary search on the 7B prefix in the ~8MB ordered list of data. If found, report the number of comments in something clickable that loads a sidebar. The only data that leaves the machine is based on the active click to load the comments. Maybe store a "false positive list" so that the rare collisions are only visible to the user once. Maybe use a bit for "visited" so you can distinguish pages with new comments?
The numbers seem surprisingly manageable.
- jdormit 9y agoExcellent idea, and a nice summary of the discussion of this from further down the thread. I've opened an issue to implement this here: https://github.com/jdormit/looped-in/issues/4 https://github.com/jdormit/looped-in/issues/4 As time allows, I will implement this later this week.