3 ms·

The overall question is, is that statement by Facebook misleading? In my opinion, yes. They are tracking individual behavior, not producing overall website anal

The overall question is, is that statement by Facebook misleading? In my opinion, yes. They are tracking individual behavior, not producing overall website analytics. There is no question that they can and do have that data in some way associated with a specific individual, not concentrating on time of access as if that’s the important part of the data. Are you saying you believe that Facebook just leave that data in difficult to access data-stores and doesn’t use it for anything? Considering it’s practically the core of their business? The implementation details are fairly unimportant. It’s what they do with the data, and the satement that it is “stored by” time of access is intended to be misleading to non-experts, in my opinion. Apparently they are even managing to distract experts and technical people.

chillacy 8y ago
I’m not sure they are “tracking individual user behavior”, what do you mean by that? Usually when you train a model you extract features for each event and then you apply your model at prediction time. There is no step in which you have the user’s entire history set aside in an easy to query fashion. Instead the model learns aggregate user behavior. Out of curiosity do you work with analytics scale data? Nothing in their statements seemed odd to me, it would easily take a few months at my current company for those reasons. For analogy it would be like if fb said “oh we support IE 7 to 11 but it’s taking us longer to support IE6”, and people said “it’s just one lower than 7 how hard can it be? Facebook is clearly hiding something, I write for IE7 all the time”. And I’m not quite sure how to convey the fact that IE6 is a different beast altogether.
- code_duck 8y ago
  Okay, so your position is that Facebook is unable to query for a list of activity by user, and does not have it divided by user in any other way? That seems nonsensical to me, but if you say so.
  - chillacy 8y ago
    Let's take Facebook out of the picture, it's too hot a topic. Let's talk about the datawarehouse of some other big tech company like amazon or adcrunch. The DW can be queried by user but each query might take 10 mins to evaluate, so it can't be used for realtime UIs. Instead you would have to denormalize the data to display to a user. That's a project, it might take a few months.
- code_duck 8y ago
  To add to this, your questions are bizarre. What do I mean by ‘Facebook tracking individual user behavior’? Um... I don’t know what to say to that. What a strange question. It’s pretty clear what is meant by that. It’s the primary focus of their business. Next, “usually when you train a model you extract features for each event and then you apply your model at prediction time”. That’s nice, but when did we start discussing ‘training models’? Facebook has a list of internet activity. It’s stored by user. They’re in the business of selling personal data. Nothing you said has anything to do with that. I have no idea what we are even discussing. Whether someone can make an ad hoc a query by user is completely not the point and has nothing to do with anything. I would assume that they have extracted the information into other reports or data stores, obviously, not process trillions of lines of entries whenever they want to access it by individual.
  - chillacy 8y ago
    > I have no idea what we are even discussing. Yea I figured. I'm not talking about Facebook at this point, this is a general engineering comment. My point with the modeling comment is that the might not access it by individual, because models can be trained on statistical averages and time-ordered instead of by individual. Also the models are trained offline (meaning not in real-time). If you have an analytics table sitting in HDFS ordered by timestamp and you want to produce a realtime UI for users, that'll take some engineering work, right?

chillacy 8y ago

I’m not sure they are “tracking individual user behavior”, what do you mean by that? Usually when you train a model you extract features for each event and then you apply your model at prediction time. There is no step in which you have the user’s entire history set aside in an easy to query fashion. Instead the model learns aggregate user behavior. Out of curiosity do you work with analytics scale data? Nothing in their statements seemed odd to me, it would easily take a few months at my current company for those reasons. For analogy it would be like if fb said “oh we support IE 7 to 11 but it’s taking us longer to support IE6”, and people said “it’s just one lower than 7 how hard can it be? Facebook is clearly hiding something, I write for IE7 all the time”. And I’m not quite sure how to convey the fact that IE6 is a different beast altogether.

code_duck 8y ago
Okay, so your position is that Facebook is unable to query for a list of activity by user, and does not have it divided by user in any other way? That seems nonsensical to me, but if you say so.
- chillacy 8y ago
  Let's take Facebook out of the picture, it's too hot a topic. Let's talk about the datawarehouse of some other big tech company like amazon or adcrunch. The DW can be queried by user but each query might take 10 mins to evaluate, so it can't be used for realtime UIs. Instead you would have to denormalize the data to display to a user. That's a project, it might take a few months.
code_duck 8y ago
To add to this, your questions are bizarre. What do I mean by ‘Facebook tracking individual user behavior’? Um... I don’t know what to say to that. What a strange question. It’s pretty clear what is meant by that. It’s the primary focus of their business. Next, “usually when you train a model you extract features for each event and then you apply your model at prediction time”. That’s nice, but when did we start discussing ‘training models’? Facebook has a list of internet activity. It’s stored by user. They’re in the business of selling personal data. Nothing you said has anything to do with that. I have no idea what we are even discussing. Whether someone can make an ad hoc a query by user is completely not the point and has nothing to do with anything. I would assume that they have extracted the information into other reports or data stores, obviously, not process trillions of lines of entries whenever they want to access it by individual.
- chillacy 8y ago
  > I have no idea what we are even discussing. Yea I figured. I'm not talking about Facebook at this point, this is a general engineering comment. My point with the modeling comment is that the might not access it by individual, because models can be trained on statistical averages and time-ordered instead of by individual. Also the models are trained offline (meaning not in real-time). If you have an analytics table sitting in HDFS ordered by timestamp and you want to produce a realtime UI for users, that'll take some engineering work, right?