This is mildly related, but it's an interesting data point. I was at a bar tonight and I chatted with a guy who works for a major cable provider. Apparently th

This is mildly related, but it's an interesting data point. I was at a bar tonight and I chatted with a guy who works for a major cable provider. Apparently they generate 10 petabytes of data an hour on what you watch, when you watch it etc. That's a lot when you take into account that its all text columns in a MSQL database somewhere. I think they have enough room to have it stashed in S3 for a week before it goes into cold storage because otherwise it's just too bloody expensive. What are they storing you ask? There's a whole tree on what you're demographics are, a tree on what your device is, a tree on what you are watching etc. And they're all cross referenced with what ad you watch, for how long, when in the show etc. All of this whizbang tech to find out how to make sure you spend that extra dollar on whatever crap they want you to. I'm sure that the coming automation is pretty horrifying, but what we have now is pretty terrible as well.