4 ms·

"For this campaign, we surveyed 930 Americans to explore their retirement plans. Among them, 16% were retired, 22% were still working, and 62% were retirees who

"For this campaign, we surveyed 930 Americans to explore their retirement plans. Among them, 16% were retired, 22% were still working, and 62% were retirees who had returned to work." So, 149 of those surveyed were retired. Of those 149, 25 (1 in 6) are considering returning to work. 13 of those want remote positions.

PragmaticPulp 4y ago
Thank you for extracting the actual numbers. Constructing underpowered surveys with loaded questions is a common tactic for pushing narratives like this. The headline confirms a lot of people’s beliefs that economic conditions are forcing people out of retirement and that remote work is the way of the future, but it really just a survey of a couple dozen people who are “considering” and “wanting” something, which ultimately has little bearing on what people actually choose to do. These underpowered survey tactics are most frequently used in “__% of Americans are living paycheck to paycheck” headlines. That’s why all of those headlines have drastically different numbers, some as high as 70-80% (which is obviously untrue). It really just means that someone was able to get a few random people on the phone to answer some questions in the middle of the day, which isn’t representative of the population at all.
- tqi 4y ago
  I think the standard error for this survey is only like +/- 3%, which imo doesn't change the narrative THAT much. That said, without more details on the survey design, it's hard not to think issues like sampling and response bias make the results difficult to generalize...
- hammock 4y ago
  I work in quantitative research. A cell with 13 or 25 respondents in it ought to be used with caution, but is not necessarily underpowered. My "rule of thumb" is that, given a well-represented and properly weighted sample, having 30+ in a given cell when crosstabbing like this is healthy, and 10+ per cell is the bare minimum I will accept - with caution.
  - Retric 4y ago
    Well-represented and properly weighted samples are unusual. As often as it’s used, phone surveys don’t qualify.
- kdfjgbdfkjgb 4y ago
  why is it obvious that that is untrue?
  - SantalBlush 4y ago
    On this site, the null hypothesis is whatever we already believe. When we see evidence of an alternative hypothesis, we find ways to throw that evidence in the trash. Small sample sizes, replication crisis, misleading headlines, research on mice, etc. are all ways to ignore whatever evidence we want. In short, we are always right, research be damned.
    - deleted 4y ago
      [deleted]
    - tqi 4y ago
      There is evidence that people are pretty good at predicting whether or not a study is going to replicate, so they're not necessarily off Ba(ye)se: https://www.theatlantic.com/science/archive/2018/08/scientists-can-collectively-sense-which-psychology-studies-are-weak/568630/ https://www.theatlantic.com/science/archive/2018/08/scientis... https://www.newscientist.com/article/2218843-common-sense-can-predict-if-a-psychology-study-will-ever-be-replicated/ https://www.newscientist.com/article/2218843-common-sense-ca... (in this particular case, I think the fact that this is content marketing is what is causing people to react so skeptically)
      - SantalBlush 4y ago
        This pretty much highlights what I'm saying. Regardless of how strong or weak this research is, it won't be scrutinized by viewers here, because it confirms what they want to believe.
  - michaelt 4y ago
    In the parlance of my country, if someone is "living paycheck to paycheck" they're a member of the working poor, people with no savings, desperately fighting to keep their head above water. Here is a survey saying 36% of Americans making >$200,000 are living paycheck to paycheck: https://www.lendingclub.com/asset/ls/cms/media-center/12th-paycheck-to-paycheck-report.pdf?v=7499f609-777c-4fc4-89a8-b50ed1ccb8d2 https://www.lendingclub.com/asset/ls/cms/media-center/12th-p... There might be a handful of people earning that much who are in a difficult financial position - divorced airline pilots with expensive alimony payments, perhaps? - but it is extremely hard to believe that 36% of people earning 4x the US median income are the working poor. On the other hand, the 36% number would make much more sense if the survey had asked people "Do you depend on your salary to pay for your living expenses?" or "Do you have enough savings to pay for 3-6 months of basic living costs?" i.e. much more expansive definitions of what it means to live paycheck-to-paycheck.
- spywaregorilla 4y ago
  No, you just don't understand how statistics works. If you get 80% on sample of 100 people, your 95% confidence interval is +/- 8%. This assumes a random sampling method.
  - deleted 4y ago
    [deleted]
  - kristjansson 4y ago
    > This assumes a random sampling method. I would encourage you to explore how difficult it is to actually get 100 responses from 100 randomly selected Americans, and how non-response/self-selection affect your confidence interval.
    - spywaregorilla 4y ago
      It is difficult. That is irrelevant to the bad logic I was responding to.
- raisedbyninjas 4y ago
  I think this discounts the finding that 575 people, the majority, were retirees that already returned to the workforce. This 2011 study said the rate for this was 15%. https://www.bls.gov/opub/mlr/2011/article/reentering-the-labor-force-after-retirement.htm https://www.bls.gov/opub/mlr/2011/article/reentering-the-lab...
  - kgwgk 4y ago
    That's not a "finding". It was their choice to survey mostly retired people who had returned to work.
- kristjansson 4y ago
  IMO it's less about power than the self-selection of the responders. Naive MoE estimates that assume random sampling completely miss that respondents are just a difference species, esp. w.r.t. financial attributes. It would be shocking if some large fraction of high-earners really were living paycheck-to-paycheck. That a large fraction people who chose to respond to a survey and who stated they earn $xxxK/yr also said they're living paycheck-to-paycheck is ... somewhat less surprising. There should be half a dozen "among survey respondents" comments to headlines like this, the same way there are a dozen "in mice" comments to every breathless report of medical progress in this-or-that.
inductive_magic 4y ago
...And we don't know their respective jobs. The only indicator we get is that more than 50% of those who consider going back need money, which probably correlates negatively with jobs that have a low entry barrier for remote work. Add to this that people above 50 tend to have serious problems getting jobs in tech (and likely adjecent industries).
bgirard 4y ago
Are we using "considering" with a stronger definition? Otherwise this is meaningless. On any given week I consider changing job, quitting, early retiring, changing teams, changing projects, etc... My brain will just rapid fire different scenarios and I'll typically conclude that staying put is my optimal move. What's more surprising to me is that 5 in 6 are NOT considering alternate scenarios.
- ChrisMarshallNY 4y ago
  I "consider" running away to join the circus, at least, once a day.
  - bgirard 4y ago
    "Software engineers on Hacker News are discussing joining the circus after mass layoffs" - paychex, probably
    - throwaway8689 4y ago
      That's the research proposal I want a grant for. My work will be going to the circus all the time and asking the acrobats if any of them used to work for Adobe.
sokoloff 4y ago
I wonder why they reported 53% interested in remote (rather than 52%, which is 13/25).
asymmetric 4y ago
Is your point that the sample size is too small? Beyond a certain threshold, sample size doesn’t matter as much as you think.
- throw_pm23 4y ago
  That's true if the sample is random.
  - deleted 4y ago
    [deleted]
abledon 4y ago
There should be a law where you have to state this sample size stuff at the start of every statistical article.
skeeter2020 4y ago
25 out of almost 1000 doesn't seem like much to me. My parents are both retired and financially comfortable but would consider employment with a lot of flexibility. The 62% who had already returned to work IS surprising to me. I'd bet most of those are financially motivated
- ghaff 4y ago
  Yeah. I imagine there are (broadly speaking) two groups that are conflated. There's the group that has returned or is considering returning to full-time work because they found they need the money. And there's a group who might do something interesting on the side for a bit of money and because it's interesting. I would absolutely consider the latter in retirement but would hope not to have to do the former.
deleted 4y ago
[deleted]

PragmaticPulp 4y ago

Thank you for extracting the actual numbers. Constructing underpowered surveys with loaded questions is a common tactic for pushing narratives like this. The headline confirms a lot of people’s beliefs that economic conditions are forcing people out of retirement and that remote work is the way of the future, but it really just a survey of a couple dozen people who are “considering” and “wanting” something, which ultimately has little bearing on what people actually choose to do. These underpowered survey tactics are most frequently used in “__% of Americans are living paycheck to paycheck” headlines. That’s why all of those headlines have drastically different numbers, some as high as 70-80% (which is obviously untrue). It really just means that someone was able to get a few random people on the phone to answer some questions in the middle of the day, which isn’t representative of the population at all.

tqi 4y ago
I think the standard error for this survey is only like +/- 3%, which imo doesn't change the narrative THAT much. That said, without more details on the survey design, it's hard not to think issues like sampling and response bias make the results difficult to generalize...
hammock 4y ago
I work in quantitative research. A cell with 13 or 25 respondents in it ought to be used with caution, but is not necessarily underpowered. My "rule of thumb" is that, given a well-represented and properly weighted sample, having 30+ in a given cell when crosstabbing like this is healthy, and 10+ per cell is the bare minimum I will accept - with caution.
- Retric 4y ago
  Well-represented and properly weighted samples are unusual. As often as it’s used, phone surveys don’t qualify.
kdfjgbdfkjgb 4y ago
why is it obvious that that is untrue?
- SantalBlush 4y ago
  On this site, the null hypothesis is whatever we already believe. When we see evidence of an alternative hypothesis, we find ways to throw that evidence in the trash. Small sample sizes, replication crisis, misleading headlines, research on mice, etc. are all ways to ignore whatever evidence we want. In short, we are always right, research be damned.
  - deleted 4y ago
    [deleted]
  - tqi 4y ago
    There is evidence that people are pretty good at predicting whether or not a study is going to replicate, so they're not necessarily off Ba(ye)se: https://www.theatlantic.com/science/archive/2018/08/scientists-can-collectively-sense-which-psychology-studies-are-weak/568630/ https://www.theatlantic.com/science/archive/2018/08/scientis... https://www.newscientist.com/article/2218843-common-sense-can-predict-if-a-psychology-study-will-ever-be-replicated/ https://www.newscientist.com/article/2218843-common-sense-ca... (in this particular case, I think the fact that this is content marketing is what is causing people to react so skeptically)
    - SantalBlush 4y ago
      This pretty much highlights what I'm saying. Regardless of how strong or weak this research is, it won't be scrutinized by viewers here, because it confirms what they want to believe.
- michaelt 4y ago
  In the parlance of my country, if someone is "living paycheck to paycheck" they're a member of the working poor, people with no savings, desperately fighting to keep their head above water. Here is a survey saying 36% of Americans making >$200,000 are living paycheck to paycheck: https://www.lendingclub.com/asset/ls/cms/media-center/12th-paycheck-to-paycheck-report.pdf?v=7499f609-777c-4fc4-89a8-b50ed1ccb8d2 https://www.lendingclub.com/asset/ls/cms/media-center/12th-p... There might be a handful of people earning that much who are in a difficult financial position - divorced airline pilots with expensive alimony payments, perhaps? - but it is extremely hard to believe that 36% of people earning 4x the US median income are the working poor. On the other hand, the 36% number would make much more sense if the survey had asked people "Do you depend on your salary to pay for your living expenses?" or "Do you have enough savings to pay for 3-6 months of basic living costs?" i.e. much more expansive definitions of what it means to live paycheck-to-paycheck.
spywaregorilla 4y ago
No, you just don't understand how statistics works. If you get 80% on sample of 100 people, your 95% confidence interval is +/- 8%. This assumes a random sampling method.
- deleted 4y ago
  [deleted]
- kristjansson 4y ago
  > This assumes a random sampling method. I would encourage you to explore how difficult it is to actually get 100 responses from 100 randomly selected Americans, and how non-response/self-selection affect your confidence interval.
  - spywaregorilla 4y ago
    It is difficult. That is irrelevant to the bad logic I was responding to.
raisedbyninjas 4y ago
I think this discounts the finding that 575 people, the majority, were retirees that already returned to the workforce. This 2011 study said the rate for this was 15%. https://www.bls.gov/opub/mlr/2011/article/reentering-the-labor-force-after-retirement.htm https://www.bls.gov/opub/mlr/2011/article/reentering-the-lab...
- kgwgk 4y ago
  That's not a "finding". It was their choice to survey mostly retired people who had returned to work.
kristjansson 4y ago
IMO it's less about power than the self-selection of the responders. Naive MoE estimates that assume random sampling completely miss that respondents are just a difference species, esp. w.r.t. financial attributes. It would be shocking if some large fraction of high-earners really were living paycheck-to-paycheck. That a large fraction people who chose to respond to a survey and who stated they earn $xxxK/yr also said they're living paycheck-to-paycheck is ... somewhat less surprising. There should be half a dozen "among survey respondents" comments to headlines like this, the same way there are a dozen "in mice" comments to every breathless report of medical progress in this-or-that.

inductive_magic 4y ago

...And we don't know their respective jobs. The only indicator we get is that more than 50% of those who consider going back need money, which probably correlates negatively with jobs that have a low entry barrier for remote work. Add to this that people above 50 tend to have serious problems getting jobs in tech (and likely adjecent industries).

bgirard 4y ago

Are we using "considering" with a stronger definition? Otherwise this is meaningless. On any given week I consider changing job, quitting, early retiring, changing teams, changing projects, etc... My brain will just rapid fire different scenarios and I'll typically conclude that staying put is my optimal move. What's more surprising to me is that 5 in 6 are NOT considering alternate scenarios.

ChrisMarshallNY 4y ago
I "consider" running away to join the circus, at least, once a day.
- bgirard 4y ago
  "Software engineers on Hacker News are discussing joining the circus after mass layoffs" - paychex, probably
  - throwaway8689 4y ago
    That's the research proposal I want a grant for. My work will be going to the circus all the time and asking the acrobats if any of them used to work for Adobe.

sokoloff 4y ago

I wonder why they reported 53% interested in remote (rather than 52%, which is 13/25).

asymmetric 4y ago

Is your point that the sample size is too small? Beyond a certain threshold, sample size doesn’t matter as much as you think.

throw_pm23 4y ago
That's true if the sample is random.
- deleted 4y ago
  [deleted]

abledon 4y ago

There should be a law where you have to state this sample size stuff at the start of every statistical article.

skeeter2020 4y ago

25 out of almost 1000 doesn't seem like much to me. My parents are both retired and financially comfortable but would consider employment with a lot of flexibility. The 62% who had already returned to work IS surprising to me. I'd bet most of those are financially motivated

ghaff 4y ago
Yeah. I imagine there are (broadly speaking) two groups that are conflated. There's the group that has returned or is considering returning to full-time work because they found they need the money. And there's a group who might do something interesting on the side for a bit of money and because it's interesting. I would absolutely consider the latter in retirement but would hope not to have to do the former.

deleted 4y ago

[deleted]