12 ms·

Debunking Princeton

812 points by friggeri 13y ago

tokenadult 13y ago
A suitably joking response for a "study" that didn't need to be taken too seriously (and wasn't taken very seriously by most media outlets, including the one that led a story with it). As we discussed yesterday here on Hacker News[1], the study methodology was not sufficiently validated to convince most people that Facebook will massively lose users, even if all the data were correct. More likely, the study's model was just flat wrong. [1] https://news.ycombinator.com/item?id=7104904 https://news.ycombinator.com/item?id=7104904
- hatred 13y ago
  Some people on the Wall Street did take it up a bit seriously perhaps going by the price movement today.
  - poopsintub 13y ago
    My investment account seems to disagree too. The stock was down more than $1 before the market even opened based on that sham study by Princeton.
    - jakewalker 13y ago
      Why are you so sure that was the cause of the price decline?
      - poopsintub 13y ago
        The other big news was that Facebook was testing ads outside of their own platform. The possibility of taking on Google's Adwords and adding more than a few billion dollars in revenue should have increased the price?
        patmcc 13y ago
        You're layering a lot of your believe about Facebook onto that assumption - maybe people saw that news and concluded Facebook would fail in external ads and lose a bunch of money, which decreased the stock price. It's hardly unambiguously good news (like record profits or something).
    - Sssnake 13y ago
      Irony overload. What was wrong with that paper again? Correlation != causation?
    - jaynos 13y ago
      Pretty much every stock was down yesterday.
  - nostromo 13y ago
    Wait... didn't we just learn that correlation != causation? :)
    - yid 13y ago
      Apparently not all of us did :)
    - frostmatthew 13y ago
      http://xkcd.com/552/ http://xkcd.com/552/
  - gfodor 13y ago
    S&P was down a lot too. Just one of an infinite number of reasons to not assume price action on a company can be traced back to a given news story.
    - chronic25 13y ago
      No, you're wrong. The article definitely caused price movements.
  - richardwhiuk 13y ago
    In the short term, stocks on Wall Street are essentially a random walk.
DaniFong 13y ago
Big data: 0. Smart analysis: 1. Snark: 100. :-)
augustocallejas 13y ago
One of the top comments on that page is that they loved the tone of article. Personally I hate sarcastic responses (no matter how correct you are), which sound very unprofessional. I hope to see less of these types of responses in the future.
- deleted 13y ago
  [deleted]
- heyheyhey 13y ago
  Why do you expect them to be professional? Obviously, the response was in jest and wasn't some formal response by FB's PR group. This was just done by a couple FB workers who laughed at Princeton's methodology. To expect a certain level of professionalism for someone's FB post seems a bit absurd and pedantic.
- soup10 13y ago
  I also hate sarcastic responses, america, and cake.
- elwell 13y ago
  /s
- seanhandley 13y ago
  You hate sarcasm? That's terrible!
curiousAl 13y ago
Google Trends Extrapolation: The Nostradamus of our times.
- stretchydeath 13y ago
  http://www.google.com/trends/explore#q=google%20trends http://www.google.com/trends/explore#q=google%20trends
  - tethis 13y ago
    Google Trends predicts own demise?
smoyer 13y ago
Hmmmm ... I think I believe both articles! Higher education and social media are both in decline. Who needs critical thinking and the ability to gossip 24/7? I'm going to go rewatch Idiocracy (http://www.imdb.com/title/tt0387808/ http://www.imdb.com/title/tt0387808/).
- johnjlocke 13y ago
  Finally...a rational statement about what is really going on. I do think Facebook will survive, as infrastructure for the internet, but the history of all social networks says they have a time limit before they are replaced by something cooler. Right now, there is nothing better, so no one is jumping ship en masse.
- ballard 13y ago
  For anyone that hasn't seen it, Idiocracy goes after the present by exaggerating the future. It's a sad but true trend that people are too busy playing World of Warcraft to understand much less resist what is going on around them: automating of warfare[0], climate change[1], police militarization[2], sources of terrorism (including JSOC ("American Taliban") killing American citizens without a trial in countries without a declared war)[3] and peak oil [4]. That's a lot of heavy shit that that isn't being addressed, which defers consequences to be much more painful sooner or later. With commodities, often the consequences are spread out gradually (epic rise in cost of gasoline) whereas others are not so gradual. That seems far more important than whether some pre-press study is important or not. References: [0] http://www.ted.com/talks/daniel_suarez_the_kill_decision_shouldn_t_belong_to_a_robot.html http://www.ted.com/talks/daniel_suarez_the_kill_decision_sho... [1] California is having the worst drought in five decades. North Texas is having the worst drought in over a century. http://droughtmonitor.unl.edu/ http://droughtmonitor.unl.edu/ [2] https://www.aclu.org/militarization https://www.aclu.org/militarization [3] http://dirtywars.org/ http://dirtywars.org/ [4] http://www.theguardian.com/environment/peak-oil http://www.theguardian.com/environment/peak-oil
joelgrus 13y ago
This is pretty funny, although they ruined it with their "HEY GUYS THIS WAS A JOKE" at the end.
- ultimatedelman 13y ago
  agreed, but they have to put that in there to cover their asses in case some moron doesn't get it and gets offended.
  - elwell 13y ago
    agreed, but they wouldn't have done that when they were still a startup.
- GBond 13y ago
  Judging from some of the comments here, that line at the end was apparently necessary.
  - afhsfsfdsss88 13y ago
    FATAL: sense_of_humor.dll not found.
debacle 13y ago
> not all research is created equal You mean "not all linkbait is created equal?"
hooande 13y ago
I hope this becomes a bigger story than the original. I can't tell you how many times someone has thrown a junk science article in my face, thinking that the issue in question was now settled. A survey with a small sample size, outsized extrapolations and numbers that don't match the accompanying conjecture. There should be consequences for the people who publish these things. People have a tendency to believe anything that someone in a lab coat says, especially if it supports their point of view or anecdotal experience. In many cases the people who do the research present it with few qualifications while not standing behind assumed implications. If someone publishes sensational and link baity findings they should say, unequivocally, "I'm willing to stake my reputation on the idea this trend is real and will continue" or "These are just data and I'm not willing to say that they have any bearing on reality". Facebook may not have been right to dignify the initial post with a response, but I hope it works for the best. They say that some attention is better than no attention at all. It's important that this applies to self promotion and persona creation and not science. If somebody has something crazy to say, they should start a personal blog. Those who want to intentionally attract media attention should present themselves as such, instead of pretending to be doing any kind of meaningful experiment and hypothesis testing.
- argumentum 13y ago
  > There should be consequences for the people who publish these things. The question you pose here isn't whether or not something is "good science", it's about who gets to decide what constitutes "good science" and "bad science". Science has been able to handle itself quite well over the centuries since it has come into existence. Publishing something, in your name, as these authors have done, is an implicit declaration of what you phrased as "I'm willing to stake my reputation on the idea this trend is real and will continue". The spirit of free inquiry, on which science is based, demands that we hold nothing sacred and immune from questions. Good science, almost by definition, requires "bad science". Be careful what you wish for, because you just might get it.
- Fomite 13y ago
  Technically, this isn't even published, it's a preprint service, and it's hardly the worst thing I've ever seen come out of arXiv. If you want to blame someone, blame an uncritical media.
trollingineer 13y ago
Couldn't take this seriously given the writer is just mad he is a Data Scientist at Facebook and not doing research at an academic powerhouse.
btown 13y ago
What drove Cannarella and Spechler, two mechanical engineering Ph.D. students, to prerelease a paper about network theory and epidemiology, without coauthoring/consulting with epidemiologists or people experienced in viral communications theories, is beyond me. But the title "Debunking Princeton" seems to suggest this should be considered representative of all of the quality of research Princeton outputs, which is certainly not the case. Many posts have been made questioning the strength of the paper in question, and the fact that one with such a link-bait title is rising on HN is unfortunate in my view. Once again, a reminder to everyone that there are no peer review requirements for papers posted to arXiv. There is no evidence that the original paper was ever accepted by any journal or conference, and not surprisingly given the speculative nature of the study, the advisor of the two Ph.D.-candidate coauthors declined to place his name on the paper. So as an institution, Princeton is no more responsible for this paper than Obama is responsible for that drink machine being broken down the hallway (thanks Obama), even though it happens to be a drink machine affiliated with his country. In that regard, with all due respect towards Mr. Develin, I'm going to have to "debunk" "Debunking Princeton." [Full disclosure: I am a Princeton alumnus.]
- yetanotherphd 13y ago
  I don't think that an expert is viral communications theory would have done any better. Almost everything involving networks is a fad. Networks are an interesting mathematical structure, but on their own, they rarely enable one to build a compelling statistical model. E.g. people join Facebook for very complex social reasons, that go beyond local interactions between individuals. The image of Facebook has changed significantly since it started, and new products have entered the market, like snapchat and kik.
- lbrandy 13y ago
  Is this a clever meta-troll? Or are you actually taking the title seriously? It should be obvious that the title itself is part of the self-parody. The whole thing is a giant exercise in reductio ad absurdum. In fact, the 'joke' only works based on Princeton's reputation as an excellent university, if that makes you feel any better. [Full disclosure: I am a facebook employee]
dmazin 13y ago
That was... unprofessional.
yetanotherphd 13y ago
Very nice. I think the bigger issue is that there is a huge amount of academic research in which a fancy model (usually involving the latest academic fads) is considered a substitute for serious statistics. I think Facebook's models with linear time trends are actually much more believable than the original paper.
raverbashing 13y ago
Oh and wasn't the Facebook research based on Google trends? Meaning: people that can't type "facebook.com" on the address bar and type it into google then click? So... based on that, we can assume people are getting more familiar with the internet maybe?
- Sssnake 13y ago
  The original article isn't very long. If you want to discuss it, you could, I dunno, try reading it? Yes, it is from google trends. Just like the data on yahoo and google itself that they compared it to. So no, there is no rational basis to assume that the data indicates people are getting more familiar with the internet.
jjcm 13y ago
Sassy, but I feel like Princeton was deserving of a response like this. It was a pretty poor study that had a lot of lazy data and speculation behind it.
S4M 13y ago
Call yourself "Data Scientist" and publish graphics without units. Congrats!
- owenversteeg 13y ago
  It's clearly a satirical post. I'm sure that if he was publishing a nonsatirical article he would also not have implied that correlation is causation.
- minimaxir 13y ago
  The exact values for the axes are proprietary to Facebook. (in fact, in ggplot2 which he used for the first 2 graphs, you explicitly have to tell it to not show the units on the axes, and it's very unintuitive)
md224 13y ago
So, if this all snark, does that mean there's no "strong correlation between the undergraduate enrollment of an institution and its Google Trends index"? Because that would be a pretty interesting correlation. I'm also curious why Princeton's search volume seems to have declined. Obviously it doesn't mean Princeton is going to disappear, but what does it mean? Could be statistically insignificant, perhaps. EDIT: Here are some comparisons: http://www.google.com/trends/explore#q=%2Fm%2F05zl0%2C%20%2Fm%2F08815%2C%20%2Fm%2F01bm_%2C%20%2Fm%2F01w5m%2C%20%2Fm%2F01w3v&cmpt=q http://www.google.com/trends/explore#q=%2Fm%2F05zl0%2C%20%2F... Interesting how the initial spread seems to narrow.
- hammock 13y ago
  >I'm also curious why Princeton's search volume seems to have declined. Obviously it doesn't mean Princeton is going to disappear, but what does it mean? http://www.google.com/trends/explore#q=harvard%2C%20princeton%2C%20yale%2C%20uva http://www.google.com/trends/explore#q=harvard%2C%20princeto... Google trends represents share of search volume, not absolute volume [1]. As the higher education bubble continues to grow, ivy league schools have not grown as fast as other schools, and there are new schools being created as well. All of this activity contributes to Princeton's (for example) share of search decrease within their sector, and overall. [1]en.wikipedia.org/wiki/Google_Trends
- jules 13y ago
  In other news: http://glengilchrist.co.uk/wp-content/uploads/2012/06/pirateChart1.png http://glengilchrist.co.uk/wp-content/uploads/2012/06/pirate... It is really easy to find something that correlates with your data, when your data follows a simple trend. Now if princeton enrollment was going up and down a lot, and the google trends followed the same shape, that would be something.
- bhouston 13y ago
  md224 wrote: "I'm also curious why Princeton's search volume seems to have declined. Obviously it doesn't mean Princeton is going to disappear, but what does it mean? Could be statistically insignificant, perhaps." I have seen this in most nerdy topics associated with early internet adoption in Western English speaking countries -- or at least that is my theory. Basically as other countries come online and late adopters come online, they do not search for the same types of topics earlier adopters looked for. This trend for academic or technical topics decreasing as a percentage of Google trends is universal, see here: http://www.google.com/trends/explore?hl=en-US&q=Science,+/m/05qfh&cmpt=q&content=1 http://www.google.com/trends/explore?hl=en-US&q=Science,+/m/...
snowwrestler 13y ago
The article jokes about "the scientific principle 'correlation equals causation,'" but I feel like I see people make this mistake all the time, particularly with statistical concepts like expected value, and epidemiological risk factors. Statistics is descriptive--it's not predictive. It tells you about the data you have. It doesn't tell you why the systems produced that data, and whether they might produce very different data under different conditions.
dynamic99 13y ago
This is just awesome
yuvadam 13y ago
Hey, Facebook, wanna prove Princeton wrong? How about publishing actual metrics proving that user engagement is at an all-time high? (Oh, you don't have metrics to prove that? Is that because users are jumping ship?)
- ajju 13y ago
  You mean the actual metrics linked to in the 2nd sentence of the post? https://www.facebook.com/photo.php?fbid=10101235005815241&set=a.766892728661.2325161.4599&type=1&theater https://www.facebook.com/photo.php?fbid=10101235005815241&se... (Oh, you missed that? Is that because you didn't actually read the post?)
  - RBerenguel 13y ago
    Personally I didn't even check what the FB link was since it was clearly a link to a Facebook album. For anyone interested in data munching, a graphic is worthless, you need the numbers. And more data, always
  - yuvadam 13y ago
    Those look like numbers from Facebook's SEC filings, which says all there is to say about which metrics they're choosing to publish. (Read the post, didn't miss it.)
    - ajju 13y ago
      Not sure what you expect them to publish beyond MAUs and DAUs by region. Those are pretty standard ways to measure engagement and if I were Facebook I'd be loathe to publish more granular data just to "quiet the critics".
      - deleted 13y ago
        [deleted]
  - lotso 13y ago
    I'm not sure DAU is the best stat for looking at engagement.
devindotcom 13y ago
Seems like Facebook is responding to the media interpretation of the Princeton paper rather than the paper itself. I saw no problems with the paper: it applied an epidemiologically-inspired statistical technique to Facebook using a defunct precursor as exemplar. The limitations of this technique are obvious, but it's an interesting idea. People do "research" like this all the time - you throw shit at the wall and see what sticks. Most of the stuff that ends up on the floor never receives any attention at all, so you don't hear about it on first-tier news sites. But when it's about Facebook, it goes viral, and suddenly is the subject of intense scrutiny. They didn't bring this to the UN for a call to action. They didn't start a company around it. They just applied an idea to some data and wrote it up. And now the entire Internet is making fun of some exaggerated version of their idea, summarized by Huffington Post hit-mongers. Personally, I applaud these guys for putting in the work to test out a theory. If it's not correct, it will go in the bin with the other ten million papers with flawed theories, premises, methods, or other aspects that have been published in the last day or two.
- ellyagg 13y ago
  Well, sure. They have a right to explore it, just as Facebook has a right to make fun of it. Maybe it's not the original authors' fault it blew up, but that doesn't mean Facebook has to ignore the consequences.
- rogerbinns 13y ago
  Alex St John has good details on why using existing epidemiologically formulas don't work: http://www.alexstjohn.com/WP/2014/01/23/forecasting-demise-facebook/ http://www.alexstjohn.com/WP/2014/01/23/forecasting-demise-f... Example: "The amount of time you spend sick with the flu is constant no matter how many of your friends ALSO contract the flu"
  - 1stop 13y ago
    Not if they contract a different flu... then there is a higher chance you'll get that too, and be sick longer.
- chaz 13y ago
elwell 13y ago
I wonder if this article would exist if the first one had been by Harvard instead of Princeton (given that Harvard was the origin of Facebook).
- username223 13y ago
  The apparent author is also a Harvard alum, fwiw.
dapvincent 13y ago
"What's beef?"
lucb1e 13y ago
> every Like for this post counts as a peer review. Start reviewing! Wait did Facebook just start likewhoring themselves too?
nonconfermist 13y ago
Maybe all sites on the Internet are getting less attention as the number of things to do on the Internet increases.
abus 13y ago
I see this as Facebook's willingness to use its unfair influence to discredit anyone who dares challenge it. In other words, "if you have a page here, we won't hesitate to use it against you".
- scholia 13y ago
  Facebook takes a lot of flack: it's today's standard whipping boy. In this case, it did respond to a fundamentally flawed research paper, but I doubt it would have bothered if that paper hadn't been so widely and so shoddily reported. I though it was quite generous of Facebook to resort to gentle mockery rather than shred its numerous stupidities.
nilkn 13y ago
For anyone curious about the author of this post... http://en.wikipedia.org/wiki/Mike_Develin http://en.wikipedia.org/wiki/Mike_Develin
tedsumme 13y ago
Not to support the princeton conclusion but to initiate your debunk by pointing at "like' trends first is silly.
adharmad 13y ago
Nice comeback by facebook. Instead of defending facebook against the claims in the paper (facebook is not dying! - a very hard-to-defend position) they turned it around and used the same techniques used in the paper to show that Princeton is dying. Offense is the best defense!
lafar6502 13y ago
Sciencey stuff didn't work? Time to call lawyers
teaneedz 13y ago
Isn't Facebook a carrier of bugs?
charleswalter 13y ago
The fact that Facebook even responded reeks of fear. If the original article had been completely ridiculous, Facebook could've laughed it off and wouldn't have had to respond at all. It's like if someone tells me I'm fat. Because I'm in good shape, I wouldn't react and just think the person is weird for telling me that. But if I was anywhere close to overweight, you'd see a strong reaction of some kind from me. What the reaction is doesn't matter so much as the fact that there is a reaction.
protez 13y ago
What is causation exactly? There's no way to prove any instance of causations at all, if we delve deep into the problem of causation. Even though all data, or correlations, support F=ma, a single exceptional case can disapprove the "correlation," therefore we cannot assert force causes accelerations, and so on, since no amount of correlations is ever enough to prove causation.
jokoon 13y ago
no, I still hate facebook.
crm416 13y ago
As a Princeton student, it's been pretty frustrating to watch this devolve into (drawing from comments on this page) a "sham study by Princeton", with this response being a criticism of "Princeton's methodology", and that "Princeton was deserving of a response like this". It's been pointed out by a few other HNers, but this type of logic does a massive injustice and disservice to all the institution's undergraduate and graduate students, as well as its professors, who work hard to produce some of the highest-quality research in the world. To say that this is a "Princeton study" is to present this as if it were endorsed or produced by the administration or some department or even a tenured professor. Instead, let's remind ourselves that this was a pre-peer reviewed paper posted on _arXiv_ by two PhD students (who have likely been at the university for a few years, tops). To paint this as the Princeton community getting together as a collective and putting forth their best attempt to "debunk" Facebook is just hilariously unfair. Look, there's a thick anti-higher education slant on HN. People love referencing the higher education bubble and the 'demise' of the current university system or whatnot. But it'd be nice if we could keep things in perspective here and at least do better than the media, who can't wait to pounce on a Princeton vs. Facebook feud.
- aristus 13y ago
  Let's try a thought experiment. What if two Princeton students published an un-peer-reviewed paper on arXiv that said something useful, like revealing the origins of life, or the perfect cookie recipe? Would you argue quite as hard about how unfair it all is to call it a "Princeton study?" Or would you smile and take the reflected credit? Note that all the press now is about how "Facebook" responded to the study. It was actually a single person, Mike, who posted a note to his personal wall with some help from friends. Being a member of an institution means you represent the institution in all your actions. That's what being a member of an institution means.
  - judk 13y ago
    Mike is employed by Facebook as an audience researcher, which makes him more of an official representative of his institution (in regard to the topic at hand) than a mechanical engineering student is of theirs.
logicallee 13y ago
If they weren't being satirical, this response would be a logical error, this one: http://en.wikipedia.org/wiki/Tu_quoque http://en.wikipedia.org/wiki/Tu_quoque (The logical error is that it could very well be true that Princeton is declining, and proving this would not affect the results of the Princeton study that Facebook is declining.) However, the fact that they did so satirically sounds like when a politician says, "that's preposterous" instead of "that's false." In addition, the data they use is pretty weak. The first chart shows that Princeton "crashed" or died between 2010 and 2011, since that's when that graph tanked. Since that data isn't good to make their Tu Quoque argument (that Princeton "will die") they are being satirical and ignore it. The second graph shows that more and more Non-Princeton articles are published. But this is due to more and more non-Princeton publications. Princeton has a fairly static amount of output, as the world's universities started outputing scholarly articles in English, you would expect Princeton's share to drop. More interesting would be if its share of what it is trying to go for, Nobel Prize Laureates, publication in Science and Nature, whatever - were on the decline. This isn't addressed, just a global proportion of all scholarly articles: not Princeton's aim. However, Facebook's goal is to get a majority of daily active users. The second to last graph actually shows a pretty good case that larger institutions (by enrolment) correlate with search relevancy. But that is not a case study of an institution whose enrolment fell, which is what it would take to make a parallel case with the Princeton paper. They would have to pick an institution whose enrollment fell with its relevancy, and then show that Princeton is on the same track. As it stands, it is not "longitudinal" but just a static cross-section of enrollment and mentions. Perhaps enrolment is static at all major institutions, regardless of search relevancy, and their enrollment remains full even if they become irrelevant? This small switcheroo is a major one, and shows why the article has to be satirical. Of course they did respond quickly :) It seems to indicate that Facebook did some research, but then found their results too weak to publish straight.
bhartzer 13y ago
I think there is a big difference here between Princeton, Harvard, and Yale. Harvard and Yale clearly bought "likes" to their pages, while Princeton kept it clean and thus doesn't have any fake followers or fake page "likes".
Ind007 13y ago
Leaving all these articles, facebook has to die - this is my intuition.
xtc 13y ago
Rather than refute the negative prediction for Facebook we're presented with a sarcastic appeal to those who don't think Facebook could ever dissipate. This only speaks more towards an argument that Facebook isn't being serious enough with its own statistics. Of all organizations Facebook should be the first to spot a trend especially with esteemed data scientists like Mr. Develin.
waylandsmithers 13y ago
Yikes. I found this to be shockingly childish and the fact that facebook got this defensive makes me think they believe the article has at least a kernel of truth to it.
xordon 13y ago
There are three kinds of lies: lies, damned lies, and statistics.
joshvm 13y ago
Good fun reading that, but.. an R^2 of 0.54? Not a convincing fit.
lockone 13y ago
My guess is that the real research is the response to the paper and not the conclusion presented in the paper. Kind of a shoot in the air to see which way people run. The information gained by observing Facebook's response could prove useful to other big companies that take a hit in today's media driven society(internet). What works and doesn't work in their PR response. They could also be tracking other data, such as stock prices of other social media companies. It's what makes research so much fun.

812 points by friggeri 13y ago

A suitably joking response for a "study" that didn't need to be taken too seriously (and wasn't taken very seriously by most media outlets, including the one that led a story with it). As we discussed yesterday here on Hacker News[1], the study methodology was not sufficiently validated to convince most people that Facebook will massively lose users, even if all the data were correct. More likely, the study's model was just flat wrong. [1] https://news.ycombinator.com/item?id=7104904 https://news.ycombinator.com/item?id=7104904

hatred 13y ago
Some people on the Wall Street did take it up a bit seriously perhaps going by the price movement today.
- poopsintub 13y ago
  My investment account seems to disagree too. The stock was down more than $1 before the market even opened based on that sham study by Princeton.
  - jakewalker 13y ago
    Why are you so sure that was the cause of the price decline?
    - poopsintub 13y ago
      The other big news was that Facebook was testing ads outside of their own platform. The possibility of taking on Google's Adwords and adding more than a few billion dollars in revenue should have increased the price?
      - patmcc 13y ago
        You're layering a lot of your believe about Facebook onto that assumption - maybe people saw that news and concluded Facebook would fail in external ads and lose a bunch of money, which decreased the stock price. It's hardly unambiguously good news (like record profits or something).
  - Sssnake 13y ago
    Irony overload. What was wrong with that paper again? Correlation != causation?
  - jaynos 13y ago
    Pretty much every stock was down yesterday.
- nostromo 13y ago
  Wait... didn't we just learn that correlation != causation? :)
  - yid 13y ago
    Apparently not all of us did :)
  - frostmatthew 13y ago
    http://xkcd.com/552/ http://xkcd.com/552/
- gfodor 13y ago
  S&P was down a lot too. Just one of an infinite number of reasons to not assume price action on a company can be traced back to a given news story.
  - chronic25 13y ago
    No, you're wrong. The article definitely caused price movements.
- richardwhiuk 13y ago
  In the short term, stocks on Wall Street are essentially a random walk.

DaniFong 13y ago

Big data: 0. Smart analysis: 1. Snark: 100. :-)

augustocallejas 13y ago

One of the top comments on that page is that they loved the tone of article. Personally I hate sarcastic responses (no matter how correct you are), which sound very unprofessional. I hope to see less of these types of responses in the future.

deleted 13y ago
[deleted]
heyheyhey 13y ago
Why do you expect them to be professional? Obviously, the response was in jest and wasn't some formal response by FB's PR group. This was just done by a couple FB workers who laughed at Princeton's methodology. To expect a certain level of professionalism for someone's FB post seems a bit absurd and pedantic.
soup10 13y ago
I also hate sarcastic responses, america, and cake.
elwell 13y ago
/s
seanhandley 13y ago
You hate sarcasm? That's terrible!

curiousAl 13y ago

Google Trends Extrapolation: The Nostradamus of our times.

stretchydeath 13y ago
http://www.google.com/trends/explore#q=google%20trends http://www.google.com/trends/explore#q=google%20trends
- tethis 13y ago
  Google Trends predicts own demise?

smoyer 13y ago

Hmmmm ... I think I believe both articles! Higher education and social media are both in decline. Who needs critical thinking and the ability to gossip 24/7? I'm going to go rewatch Idiocracy (http://www.imdb.com/title/tt0387808/ http://www.imdb.com/title/tt0387808/).

johnjlocke 13y ago
Finally...a rational statement about what is really going on. I do think Facebook will survive, as infrastructure for the internet, but the history of all social networks says they have a time limit before they are replaced by something cooler. Right now, there is nothing better, so no one is jumping ship en masse.
ballard 13y ago
For anyone that hasn't seen it, Idiocracy goes after the present by exaggerating the future. It's a sad but true trend that people are too busy playing World of Warcraft to understand much less resist what is going on around them: automating of warfare[0], climate change[1], police militarization[2], sources of terrorism (including JSOC ("American Taliban") killing American citizens without a trial in countries without a declared war)[3] and peak oil [4]. That's a lot of heavy shit that that isn't being addressed, which defers consequences to be much more painful sooner or later. With commodities, often the consequences are spread out gradually (epic rise in cost of gasoline) whereas others are not so gradual. That seems far more important than whether some pre-press study is important or not. References: [0] http://www.ted.com/talks/daniel_suarez_the_kill_decision_shouldn_t_belong_to_a_robot.html http://www.ted.com/talks/daniel_suarez_the_kill_decision_sho... [1] California is having the worst drought in five decades. North Texas is having the worst drought in over a century. http://droughtmonitor.unl.edu/ http://droughtmonitor.unl.edu/ [2] https://www.aclu.org/militarization https://www.aclu.org/militarization [3] http://dirtywars.org/ http://dirtywars.org/ [4] http://www.theguardian.com/environment/peak-oil http://www.theguardian.com/environment/peak-oil

joelgrus 13y ago

This is pretty funny, although they ruined it with their "HEY GUYS THIS WAS A JOKE" at the end.

ultimatedelman 13y ago
agreed, but they have to put that in there to cover their asses in case some moron doesn't get it and gets offended.
- elwell 13y ago
  agreed, but they wouldn't have done that when they were still a startup.
GBond 13y ago
Judging from some of the comments here, that line at the end was apparently necessary.
- afhsfsfdsss88 13y ago
  FATAL: sense_of_humor.dll not found.

debacle 13y ago

> not all research is created equal You mean "not all linkbait is created equal?"

hooande 13y ago

I hope this becomes a bigger story than the original. I can't tell you how many times someone has thrown a junk science article in my face, thinking that the issue in question was now settled. A survey with a small sample size, outsized extrapolations and numbers that don't match the accompanying conjecture. There should be consequences for the people who publish these things. People have a tendency to believe anything that someone in a lab coat says, especially if it supports their point of view or anecdotal experience. In many cases the people who do the research present it with few qualifications while not standing behind assumed implications. If someone publishes sensational and link baity findings they should say, unequivocally, "I'm willing to stake my reputation on the idea this trend is real and will continue" or "These are just data and I'm not willing to say that they have any bearing on reality". Facebook may not have been right to dignify the initial post with a response, but I hope it works for the best. They say that some attention is better than no attention at all. It's important that this applies to self promotion and persona creation and not science. If somebody has something crazy to say, they should start a personal blog. Those who want to intentionally attract media attention should present themselves as such, instead of pretending to be doing any kind of meaningful experiment and hypothesis testing.

argumentum 13y ago
> There should be consequences for the people who publish these things. The question you pose here isn't whether or not something is "good science", it's about who gets to decide what constitutes "good science" and "bad science". Science has been able to handle itself quite well over the centuries since it has come into existence. Publishing something, in your name, as these authors have done, is an implicit declaration of what you phrased as "I'm willing to stake my reputation on the idea this trend is real and will continue". The spirit of free inquiry, on which science is based, demands that we hold nothing sacred and immune from questions. Good science, almost by definition, requires "bad science". Be careful what you wish for, because you just might get it.
Fomite 13y ago
Technically, this isn't even published, it's a preprint service, and it's hardly the worst thing I've ever seen come out of arXiv. If you want to blame someone, blame an uncritical media.

trollingineer 13y ago

Couldn't take this seriously given the writer is just mad he is a Data Scientist at Facebook and not doing research at an academic powerhouse.

btown 13y ago

What drove Cannarella and Spechler, two mechanical engineering Ph.D. students, to prerelease a paper about network theory and epidemiology, without coauthoring/consulting with epidemiologists or people experienced in viral communications theories, is beyond me. But the title "Debunking Princeton" seems to suggest this should be considered representative of all of the quality of research Princeton outputs, which is certainly not the case. Many posts have been made questioning the strength of the paper in question, and the fact that one with such a link-bait title is rising on HN is unfortunate in my view. Once again, a reminder to everyone that there are no peer review requirements for papers posted to arXiv. There is no evidence that the original paper was ever accepted by any journal or conference, and not surprisingly given the speculative nature of the study, the advisor of the two Ph.D.-candidate coauthors declined to place his name on the paper. So as an institution, Princeton is no more responsible for this paper than Obama is responsible for that drink machine being broken down the hallway (thanks Obama), even though it happens to be a drink machine affiliated with his country. In that regard, with all due respect towards Mr. Develin, I'm going to have to "debunk" "Debunking Princeton." [Full disclosure: I am a Princeton alumnus.]

yetanotherphd 13y ago
I don't think that an expert is viral communications theory would have done any better. Almost everything involving networks is a fad. Networks are an interesting mathematical structure, but on their own, they rarely enable one to build a compelling statistical model. E.g. people join Facebook for very complex social reasons, that go beyond local interactions between individuals. The image of Facebook has changed significantly since it started, and new products have entered the market, like snapchat and kik.
lbrandy 13y ago
Is this a clever meta-troll? Or are you actually taking the title seriously? It should be obvious that the title itself is part of the self-parody. The whole thing is a giant exercise in reductio ad absurdum. In fact, the 'joke' only works based on Princeton's reputation as an excellent university, if that makes you feel any better. [Full disclosure: I am a facebook employee]

dmazin 13y ago

That was... unprofessional.

yetanotherphd 13y ago

Very nice. I think the bigger issue is that there is a huge amount of academic research in which a fancy model (usually involving the latest academic fads) is considered a substitute for serious statistics. I think Facebook's models with linear time trends are actually much more believable than the original paper.

raverbashing 13y ago

Oh and wasn't the Facebook research based on Google trends? Meaning: people that can't type "facebook.com" on the address bar and type it into google then click? So... based on that, we can assume people are getting more familiar with the internet maybe?

Sssnake 13y ago
The original article isn't very long. If you want to discuss it, you could, I dunno, try reading it? Yes, it is from google trends. Just like the data on yahoo and google itself that they compared it to. So no, there is no rational basis to assume that the data indicates people are getting more familiar with the internet.

jjcm 13y ago

Sassy, but I feel like Princeton was deserving of a response like this. It was a pretty poor study that had a lot of lazy data and speculation behind it.

S4M 13y ago

Call yourself "Data Scientist" and publish graphics without units. Congrats!

owenversteeg 13y ago
It's clearly a satirical post. I'm sure that if he was publishing a nonsatirical article he would also not have implied that correlation is causation.
minimaxir 13y ago
The exact values for the axes are proprietary to Facebook. (in fact, in ggplot2 which he used for the first 2 graphs, you explicitly have to tell it to not show the units on the axes, and it's very unintuitive)

md224 13y ago

So, if this all snark, does that mean there's no "strong correlation between the undergraduate enrollment of an institution and its Google Trends index"? Because that would be a pretty interesting correlation. I'm also curious why Princeton's search volume seems to have declined. Obviously it doesn't mean Princeton is going to disappear, but what does it mean? Could be statistically insignificant, perhaps. EDIT: Here are some comparisons: http://www.google.com/trends/explore#q=%2Fm%2F05zl0%2C%20%2Fm%2F08815%2C%20%2Fm%2F01bm_%2C%20%2Fm%2F01w5m%2C%20%2Fm%2F01w3v&cmpt=q http://www.google.com/trends/explore#q=%2Fm%2F05zl0%2C%20%2F... Interesting how the initial spread seems to narrow.

hammock 13y ago
>I'm also curious why Princeton's search volume seems to have declined. Obviously it doesn't mean Princeton is going to disappear, but what does it mean? http://www.google.com/trends/explore#q=harvard%2C%20princeton%2C%20yale%2C%20uva http://www.google.com/trends/explore#q=harvard%2C%20princeto... Google trends represents share of search volume, not absolute volume [1]. As the higher education bubble continues to grow, ivy league schools have not grown as fast as other schools, and there are new schools being created as well. All of this activity contributes to Princeton's (for example) share of search decrease within their sector, and overall. [1]en.wikipedia.org/wiki/Google_Trends
jules 13y ago
In other news: http://glengilchrist.co.uk/wp-content/uploads/2012/06/pirateChart1.png http://glengilchrist.co.uk/wp-content/uploads/2012/06/pirate... It is really easy to find something that correlates with your data, when your data follows a simple trend. Now if princeton enrollment was going up and down a lot, and the google trends followed the same shape, that would be something.
bhouston 13y ago
md224 wrote: "I'm also curious why Princeton's search volume seems to have declined. Obviously it doesn't mean Princeton is going to disappear, but what does it mean? Could be statistically insignificant, perhaps." I have seen this in most nerdy topics associated with early internet adoption in Western English speaking countries -- or at least that is my theory. Basically as other countries come online and late adopters come online, they do not search for the same types of topics earlier adopters looked for. This trend for academic or technical topics decreasing as a percentage of Google trends is universal, see here: http://www.google.com/trends/explore?hl=en-US&q=Science,+/m/05qfh&cmpt=q&content=1 http://www.google.com/trends/explore?hl=en-US&q=Science,+/m/...

snowwrestler 13y ago

The article jokes about "the scientific principle 'correlation equals causation,'" but I feel like I see people make this mistake all the time, particularly with statistical concepts like expected value, and epidemiological risk factors. Statistics is descriptive--it's not predictive. It tells you about the data you have. It doesn't tell you why the systems produced that data, and whether they might produce very different data under different conditions.

dynamic99 13y ago

This is just awesome

yuvadam 13y ago

Hey, Facebook, wanna prove Princeton wrong? How about publishing actual metrics proving that user engagement is at an all-time high? (Oh, you don't have metrics to prove that? Is that because users are jumping ship?)

ajju 13y ago
You mean the actual metrics linked to in the 2nd sentence of the post? https://www.facebook.com/photo.php?fbid=10101235005815241&set=a.766892728661.2325161.4599&type=1&theater https://www.facebook.com/photo.php?fbid=10101235005815241&se... (Oh, you missed that? Is that because you didn't actually read the post?)
- RBerenguel 13y ago
  Personally I didn't even check what the FB link was since it was clearly a link to a Facebook album. For anyone interested in data munching, a graphic is worthless, you need the numbers. And more data, always
- yuvadam 13y ago
  Those look like numbers from Facebook's SEC filings, which says all there is to say about which metrics they're choosing to publish. (Read the post, didn't miss it.)
  - ajju 13y ago
    Not sure what you expect them to publish beyond MAUs and DAUs by region. Those are pretty standard ways to measure engagement and if I were Facebook I'd be loathe to publish more granular data just to "quiet the critics".
    - deleted 13y ago
      [deleted]
- lotso 13y ago
  I'm not sure DAU is the best stat for looking at engagement.

devindotcom 13y ago

Seems like Facebook is responding to the media interpretation of the Princeton paper rather than the paper itself. I saw no problems with the paper: it applied an epidemiologically-inspired statistical technique to Facebook using a defunct precursor as exemplar. The limitations of this technique are obvious, but it's an interesting idea. People do "research" like this all the time - you throw shit at the wall and see what sticks. Most of the stuff that ends up on the floor never receives any attention at all, so you don't hear about it on first-tier news sites. But when it's about Facebook, it goes viral, and suddenly is the subject of intense scrutiny. They didn't bring this to the UN for a call to action. They didn't start a company around it. They just applied an idea to some data and wrote it up. And now the entire Internet is making fun of some exaggerated version of their idea, summarized by Huffington Post hit-mongers. Personally, I applaud these guys for putting in the work to test out a theory. If it's not correct, it will go in the bin with the other ten million papers with flawed theories, premises, methods, or other aspects that have been published in the last day or two.

ellyagg 13y ago
Well, sure. They have a right to explore it, just as Facebook has a right to make fun of it. Maybe it's not the original authors' fault it blew up, but that doesn't mean Facebook has to ignore the consequences.
rogerbinns 13y ago
Alex St John has good details on why using existing epidemiologically formulas don't work: http://www.alexstjohn.com/WP/2014/01/23/forecasting-demise-facebook/ http://www.alexstjohn.com/WP/2014/01/23/forecasting-demise-f... Example: "The amount of time you spend sick with the flu is constant no matter how many of your friends ALSO contract the flu"
- 1stop 13y ago
  Not if they contract a different flu... then there is a higher chance you'll get that too, and be sick longer.
chaz 13y ago

elwell 13y ago

I wonder if this article would exist if the first one had been by Harvard instead of Princeton (given that Harvard was the origin of Facebook).

username223 13y ago
The apparent author is also a Harvard alum, fwiw.

dapvincent 13y ago

"What's beef?"

lucb1e 13y ago

> every Like for this post counts as a peer review. Start reviewing! Wait did Facebook just start likewhoring themselves too?

nonconfermist 13y ago

Maybe all sites on the Internet are getting less attention as the number of things to do on the Internet increases.

abus 13y ago

I see this as Facebook's willingness to use its unfair influence to discredit anyone who dares challenge it. In other words, "if you have a page here, we won't hesitate to use it against you".

scholia 13y ago
Facebook takes a lot of flack: it's today's standard whipping boy. In this case, it did respond to a fundamentally flawed research paper, but I doubt it would have bothered if that paper hadn't been so widely and so shoddily reported. I though it was quite generous of Facebook to resort to gentle mockery rather than shred its numerous stupidities.

nilkn 13y ago

For anyone curious about the author of this post... http://en.wikipedia.org/wiki/Mike_Develin http://en.wikipedia.org/wiki/Mike_Develin

tedsumme 13y ago

Not to support the princeton conclusion but to initiate your debunk by pointing at "like' trends first is silly.

adharmad 13y ago

Nice comeback by facebook. Instead of defending facebook against the claims in the paper (facebook is not dying! - a very hard-to-defend position) they turned it around and used the same techniques used in the paper to show that Princeton is dying. Offense is the best defense!

lafar6502 13y ago

Sciencey stuff didn't work? Time to call lawyers

teaneedz 13y ago

Isn't Facebook a carrier of bugs?

charleswalter 13y ago

The fact that Facebook even responded reeks of fear. If the original article had been completely ridiculous, Facebook could've laughed it off and wouldn't have had to respond at all. It's like if someone tells me I'm fat. Because I'm in good shape, I wouldn't react and just think the person is weird for telling me that. But if I was anywhere close to overweight, you'd see a strong reaction of some kind from me. What the reaction is doesn't matter so much as the fact that there is a reaction.

protez 13y ago

What is causation exactly? There's no way to prove any instance of causations at all, if we delve deep into the problem of causation. Even though all data, or correlations, support F=ma, a single exceptional case can disapprove the "correlation," therefore we cannot assert force causes accelerations, and so on, since no amount of correlations is ever enough to prove causation.

jokoon 13y ago

no, I still hate facebook.

crm416 13y ago

As a Princeton student, it's been pretty frustrating to watch this devolve into (drawing from comments on this page) a "sham study by Princeton", with this response being a criticism of "Princeton's methodology", and that "Princeton was deserving of a response like this". It's been pointed out by a few other HNers, but this type of logic does a massive injustice and disservice to all the institution's undergraduate and graduate students, as well as its professors, who work hard to produce some of the highest-quality research in the world. To say that this is a "Princeton study" is to present this as if it were endorsed or produced by the administration or some department or even a tenured professor. Instead, let's remind ourselves that this was a pre-peer reviewed paper posted on _arXiv_ by two PhD students (who have likely been at the university for a few years, tops). To paint this as the Princeton community getting together as a collective and putting forth their best attempt to "debunk" Facebook is just hilariously unfair. Look, there's a thick anti-higher education slant on HN. People love referencing the higher education bubble and the 'demise' of the current university system or whatnot. But it'd be nice if we could keep things in perspective here and at least do better than the media, who can't wait to pounce on a Princeton vs. Facebook feud.

aristus 13y ago
Let's try a thought experiment. What if two Princeton students published an un-peer-reviewed paper on arXiv that said something useful, like revealing the origins of life, or the perfect cookie recipe? Would you argue quite as hard about how unfair it all is to call it a "Princeton study?" Or would you smile and take the reflected credit? Note that all the press now is about how "Facebook" responded to the study. It was actually a single person, Mike, who posted a note to his personal wall with some help from friends. Being a member of an institution means you represent the institution in all your actions. That's what being a member of an institution means.
- judk 13y ago
  Mike is employed by Facebook as an audience researcher, which makes him more of an official representative of his institution (in regard to the topic at hand) than a mechanical engineering student is of theirs.

logicallee 13y ago

If they weren't being satirical, this response would be a logical error, this one: http://en.wikipedia.org/wiki/Tu_quoque http://en.wikipedia.org/wiki/Tu_quoque (The logical error is that it could very well be true that Princeton is declining, and proving this would not affect the results of the Princeton study that Facebook is declining.) However, the fact that they did so satirically sounds like when a politician says, "that's preposterous" instead of "that's false." In addition, the data they use is pretty weak. The first chart shows that Princeton "crashed" or died between 2010 and 2011, since that's when that graph tanked. Since that data isn't good to make their Tu Quoque argument (that Princeton "will die") they are being satirical and ignore it. The second graph shows that more and more Non-Princeton articles are published. But this is due to more and more non-Princeton publications. Princeton has a fairly static amount of output, as the world's universities started outputing scholarly articles in English, you would expect Princeton's share to drop. More interesting would be if its share of what it is trying to go for, Nobel Prize Laureates, publication in Science and Nature, whatever - were on the decline. This isn't addressed, just a global proportion of all scholarly articles: not Princeton's aim. However, Facebook's goal is to get a majority of daily active users. The second to last graph actually shows a pretty good case that larger institutions (by enrolment) correlate with search relevancy. But that is not a case study of an institution whose enrolment fell, which is what it would take to make a parallel case with the Princeton paper. They would have to pick an institution whose enrollment fell with its relevancy, and then show that Princeton is on the same track. As it stands, it is not "longitudinal" but just a static cross-section of enrollment and mentions. Perhaps enrolment is static at all major institutions, regardless of search relevancy, and their enrollment remains full even if they become irrelevant? This small switcheroo is a major one, and shows why the article has to be satirical. Of course they did respond quickly :) It seems to indicate that Facebook did some research, but then found their results too weak to publish straight.

bhartzer 13y ago

I think there is a big difference here between Princeton, Harvard, and Yale. Harvard and Yale clearly bought "likes" to their pages, while Princeton kept it clean and thus doesn't have any fake followers or fake page "likes".

Ind007 13y ago

Leaving all these articles, facebook has to die - this is my intuition.

xtc 13y ago

Rather than refute the negative prediction for Facebook we're presented with a sarcastic appeal to those who don't think Facebook could ever dissipate. This only speaks more towards an argument that Facebook isn't being serious enough with its own statistics. Of all organizations Facebook should be the first to spot a trend especially with esteemed data scientists like Mr. Develin.

waylandsmithers 13y ago

Yikes. I found this to be shockingly childish and the fact that facebook got this defensive makes me think they believe the article has at least a kernel of truth to it.

xordon 13y ago

There are three kinds of lies: lies, damned lies, and statistics.

joshvm 13y ago

Good fun reading that, but.. an R^2 of 0.54? Not a convincing fit.

lockone 13y ago

My guess is that the real research is the response to the paper and not the conclusion presented in the paper. Kind of a shoot in the air to see which way people run. The information gained by observing Facebook's response could prove useful to other big companies that take a hit in today's media driven society(internet). What works and doesn't work in their PR response. They could also be tracking other data, such as stock prices of other social media companies. It's what makes research so much fun.