5 ms·
Slaves of the feed - This is not the realtime we've been looking for
- tdoggette 17y agoHe correctly identifies a problem, but no amount of interdevice communication is going to tell me which 5 of boingboing's posts are most likely to be of personal interest. Here's what'd help me with the bottleneck: a unified inbox with a wicked smart relevance algorithm. VR? Probably not.
- JulianMorrison 17y agoWe've had plain old naive-Bayesian filtering for ages - I'm surprised more feed-readers don't use it.
- ThomPete 17y agoThe idea is that interdevice communication creates the knowledge of context by logging everything you do but don't have the ability to to do. I know it needs more thinking but there is something there imho
- fragmede 17y agoSo how do can we solve this problem of drinking from a firehose? I think a system like Google Reader system might have some traction here, wherein friends can recommend items to each other. With enough friends, you could make a metafeed out of those items. Make it a new service; you get paid for drinking from the firehose, and you pay to get human-filtered feeds. Throw some 'liked/hated this firehose drinker' and a pile of people at it, and you get interest spaces.
- metajack 17y agoThere are decades of research in information retrieval that can easily be applied to this problem. Before Google showed up with PageRank, many algorithms were based on properties of a given document and the historical properties of the set (example, TF-IDF). You don't need to drink from a firehose. The firehose will be throttled and filtered to given you the useful information that you want. This is where the future of real-time search lies. Disclosure: I'm CTO of Collecta, one of these real-time search companies.
- evlapix 17y agoSearch clearly isn't the answer. Can you create a search term that encompasses all of your interests? I hope not. Surely filtering the "fire hose" with the query "programming news" will not yield satisfactory results.
- olefoo 17y agoThat's pretty much what we have now, except we haven't really made it so that the monkeys picking out the good information are being rewarded proportional to the value they create. Right now, it's the monkey who can put the most ads in front of the most monkeys who gets the banana, not the monkeys who find the most worthwhile information from the most obscure sources.
- NathanKP 17y agoI don't think computers can help anyone find truly interesting things. An algorithm would have to be really complicated to discover the fact that just because so many people are interested in New Moon, or Tiger Woods, that doesn't mean I am. When "relevancy" algorithms are attempted they don't do nearly as good a job at serving niche groups as "community" moderated algorithms such as HN. HN is an algorithm, in which each person's brain is an equation that estimates the "worth" of piece of information. As a result the final product is an extremely complicated result of "equations", as it were, that are good at finding relevant information. I don't expect computers to be good at doing that for another ten years or so.
- joeyo 17y agoI don't think computers can help anyone find truly interesting things. An algorithm would have to be really complicated to discover the fact that just because so many people are interested in New Moon, or Tiger Woods, that doesn't mean I am. I don't see why, in principle, the full Bayesian inference couldn't be done. In other words, automatically compute how interesting the item is based on how the masses liked it in aggregate, how individuals with similar tastes to you liked it, how similar the item is to other items that you have liked, etc.
- loganfrederick 17y agoThe problem with this approach is feasibility. Some companies like Netflix and Amazon have done some complex recommendation engines, and I still don't think they're processing as much data as an all-encompassing social network data aggregating recommendation engine would. I believe Reddit was at one point going to try and create a news recommendation engine but found it impractical as the number of users grew. The number of permutations for what people in just one user's network like and dislike is staggering.
- joeyo 17y agoGiven that it's presently possible to offload data processing to a number of cloud-based computing services, I'm not sure that we should be afraid of the computation cost. If you are willing to make some assumptions about independence (for example) you can do a lot of the computations in parallel. A more subtle issue would be how much the use of social net data would gain Amazon or Netflix over their current recommendation engines. I suspect it would be quite a lot, but that's just a guess.
- sunkencity 17y agoThe feed items that truly contain new information is not one trains for traditionally with bayes or so, because it would imply something that has not been seen before, rather than just being similar to something already seen. Searching for meaningfully different stuff rather than similar stuff is a whole different problem. I'd like to have a treemap view of google reader rather than a list, and where I could visually, in real time filter the feeds by adding negative and positive keywords/tags, and click around in various levels of a treemap. Click on "technology" => make a new treemap of significant tags in that area, click on the next tag to dive into that space etc, and eventually see all the feed items somehow. Perhaps present them in all levels by some sidebar or hoverthing. Color code items for popularity / activity / freshness. I'm thinking something like dabble.db but with a treemap interface and cappuchino js interface.
- tdoggette 17y agoHere's another way to approach the user's end of it: have them log in to their Google, Facebook, Twitter, et al, and shove all of it into a single stream. Let them filter by source, topic, relevance, time, and anything else you can implement. Record their preferences by watching what items they click through or spend time on or interact with, and your relevance sort gets better. Add other users' data, and you have a fantastic recommendation engine without ever making anyone rate anything. Another problem to solve with current news reading is duplication. Even if you do want to read about Tiger Woods, there's a lot of duplication in the echo chamber of the internet.
- sunkencity 17y agoYep, that's the position google has to analyze data, it's incredibly powerful to have almost the whole activitystream on the internet. An interesting sidenote, apparently google could activate face recognition for google goggles but have chosen not to do that at this moment for privacy issues.
- evlapix 17y agoAs much as I hate to say it, It seems like this is something that could be easily accomplished by piggybacking off of Twitter. Just drop the link, some tags, and misc attributes along with a short description.. Is it much more complicated than that?
- ThomPete 17y agoLots of great comments, thanks so much.