3 ms·
Am curious, any ensemble methods involved? Or was it just a bunch of heuristics mixed in with learning algorithms? I think Metamarkets, like Palantir and other
by izendejas 16y ago
Am curious, any ensemble methods involved? Or was it just a bunch of heuristics mixed in with learning algorithms?
I think Metamarkets, like Palantir and others, understand this very well and thus, are focused on building interactive tools to help humans (particularly domain experts) process and visualize large amounts of data to find interesting patterns more efficiently, rather than trying to automate everything.
- bane 16y agoThe problem with visualization as the goal is that eventually the visual models become so complex that finding patterns, even by a human, become virtually impossible. http://sixdegrees.hu/last.fm/images/lastfm_800_graph_white.png http://sixdegrees.hu/last.fm/images/lastfm_800_graph_white.p... You might be able to "see" some large scale constructs, but finding patterns in the details is what this is all about and humans simply can't do that with graphs like the above...and it's amazing how quickly you can arrive at hairballs just like this... even specialized tools for visualizing large graphs don't help much. http://www.caida.org/tools/visualization/walrus/ http://www.caida.org/tools/visualization/walrus/
- oakenshield 16y agoYes, I worked on quite a few boosting / bagging / random forests classifiers. The key thing I learned, though, was that it wasn't possible to take the _entire_ dataset for some problem, throw even an ensemble method at it, and expect high detection and low FP rates. As you noted, one has to filter and prune using domain knowledge or hand-verified heuristics before you'd see decent performance.