4 ms·
As a data scientist it is very frustrating to see a consistent lack of error bars on these numbers or any discussion about the magnitude of uncertainty around t
by astro1234 2mo ago
As a data scientist it is very frustrating to see a consistent lack of error bars on these numbers or any discussion about the magnitude of uncertainty around them by all news reports every time these numbers get released
- renegade-otter 2mo agoThe numbers are as good as they can get with hard manual work that goes into compiling them. The problem is that small businesses lag by 1-2 months reporting them, so the initial number being reported is almost always irrelevant, as it mostly applies to numbers coming out of larger corporations.
- jstanley 2mo ago> The numbers are as good as they can get That doesn't mean they don't warrant error bars. Error bars aren't there to show that you didn't do a good job collecting the data, they're there to show your level of uncertainty.
- vlovich123 2mo agoI would say the initial numbers should be embargoed until the number is reliable. People don’t understand error bars and reporting can more easily manipulate the current situation without outright lying.
- twosdai 2mo agoBoth? I don't think error bars and waiting to release data until more accurate are mutually exclusive.
- vlovich123 2mo agoThat’s what I meant. There should be error bars to reflect uncertainty of the inherent data, but they should wait until all the data that could change things comes in rather than using projections which always drastically change
- jmalicki 2mo agoNo, they shouldn't. Timely estimates as to the best knowledge of our current situation is extremely important for decision making under uncertainty. If the error bars are large, that is important information to know.
- vlovich123 2mo agoWhat decisions are so important and time sensitive they depend on these numbers right after a month is over? And also we at one point were making decisions without these numbers yet the quality of the decision making isn’t clearly improved.
- jmalicki 2mo ago"What kind of stocks should my mutual fund invest in?" "What sort of startups should I fund?" "Should my business aggressively hire this quarter anticipating growth, or hold out for a possible recession?"
- vlovich123 2mo agoA few months of extra no data won’t impact the stock investments if no one has that data. Startup funding is similarly not so time sensitive. As for businesses hiring, they should hire based on their own growth. Using an external signal that itself gets adjusted drastically is similarly problematic; I don’t hire because I think no one else is hiring and the numbers get majorly adjusted up. Similarly I hire a bunch of people and it turns out no one else hired. No one benefits from faster numbers that are majorly shifted in a few months anyway. You wouldn’t make any different decision if no one had the numbers. The only thing you’re doing differently is when the numbers come out your gambling whether or not you think the true number will be revised in some major way in a few months. That’s not helpful for the economy writ large.
- win311fwg 2mo ago> If the error bars are large, that is important information to know. The data is available. If it were important to the causal reader, however, then those casual readers would have already demanded its presence. But they don't care. What difference does it make to Hilary Homemaker? She just wants to get to say "Did you hear about the bad economic news?" when gossiping with her friends. Those who need to understand the uncertainty will already be looking at more than a news headline anyway.
- theptip 2mo agoNo, just report them with error bars. We know the 95th percentile revision size of the distribution. Moot point as the people who actually act on the data understand how to roll their own anyway.
- jfengel 2mo agoError bars would be misleading, because this isn't a sampling error. These start with actual counts. The more important errors are systematic: the definitions of terms, the reporting mechanisms, etc. Except that those things are constant from month to month, so deltas largely cancel them out. The problem isn't in the size of the possible errors, but in understanding what those numbers mean and how they are gathered. That requires far more background reading.
- parineum 2mo agoAh yes, lie to them and withold information because the public is too stupid to understand. Then, when they don't trust you anymore, throw up your hands and say it's not your fault.
- pgwhalen 2mo agoI think you say “as good as they can get with [the] hard manual work that goes into compiling them” as a way to respect the challenge of the BLS and say they’re doing the best with their resources, but it’s worth pointing out they could be better with more resources.
- jmalicki 2mo agoDOGE hurt them pretty hard: "The Bureau of Labor Statistics posted a notice on Wednesday stating that it stopped collecting data in three not-so-small cities (Lincoln, Nebraska; Buffalo, New York; and Provo, Utah) and increased “imputations” for certain items" https://www.cnn.com/2025/06/05/economy/cpi-data-bls-reductions https://www.cnn.com/2025/06/05/economy/cpi-data-bls-reductio...
- renegade-otter 2mo agoThat is exactly right. This is a short pod episode where a former BLS director explains all this: https://www.youtube.com/watch?v=EhsLjmqse98 https://www.youtube.com/watch?v=EhsLjmqse98
- pgwhalen 2mo agoIndeed this is my source for the claim!
- jm4 2mo agoWhat makes you think it's small businesses? I would assume most businesses doing any reporting at all are using a payroll system. Mine handles the reporting automatically. The kinds of payroll systems used by small businesses can be set up in a day. If anything, small businesses that make even a small effort to stay on top of things have an easier time than big ones with complicated systems and processes.
- jmalicki 2mo agoThat's true for the establishment survey, but the population survey, where they contact individuals, is what is actually used to calculate these numbers. It is literally calling/writing 60,000 people and asking "are you employed? did you look for a job? Etc." "How the government calculates unemployment" [0] | Does anyone in this household have a business or a farm? | Last week, did you do any work for (either) pay (or profit)? | If the answer to question 1 is "yes" and the answer to question 2 is "no," the next question is: | Last week, did you do any unpaid work in the family business or farm? | For those who reply "no" to both questions 2 and 3, the next key questions used to determine employment status are: | Last week, (in addition to the business) did you have a job, either full or part time? Include any job from which you were temporarily absent. | Last week, were you on layoff from a job? | What was the main reason you were absent from work last week? | For those who respond "yes" to question 5 about being on layoff, the following questions are asked: [0] https://www.bls.gov/cps/cps_htgm.htm https://www.bls.gov/cps/cps_htgm.htm
- amanaplanacanal 2mo agoYou have to wonder, since they are doing two different surveys trying to measure the same reality, which one more accurately measures the facts on the ground? Asking businesses how many people they are employing, or asking households whether or not their folks are working? The numbers don't really match up between the two.
- jmalicki 2mo agoThis is very common for economic statistics. GDP too. That's how you can get more accurate information, and you have actual numbers on the discrepencies. They rely on the household data more because it doesn't have as much systemic error, even though it does have more sampling error. Asking businesses gives you a more complete picture, but it has much more bias. Hell, this is why even in accounting, you have double-entry bookkeeping, and even there, the numbers don't always match up. Or "my bank account doesn't match up with my cashflow statement". It's not that one is better and you should use that, calculating multiple ways is central to accounting so you can detect errors.
- europoorean 2mo agoAs a physicist, I always shed a tear when looking at data without error bars. As a data scientist/generic programmer now, I don't really see them anywhere. I remember a colleague during my postdoc saying 'don't worry, no error bars in industry' when I asked about regression with ML lol
- natsucks 2mo agohah. it's true. no interest in uncertainty except when forced.
- Noaidi 2mo agoOur fake economy needs even more fake certainty now. They are all hiding from the obvious issue of stagflation because it is the most difficult economic problem to solve. The Fed know this which is why they did nothing with rates.
- cousinbryce 2mo agohttps://www.bls.gov/jlt/jltreliability.htm https://www.bls.gov/jlt/jltreliability.htm Read the docs
- astro1234 2mo agoOh I’m well aware the BLS knows what they’re doing and publish error bars, they are great. The problem is news articles, no one ever reads BLS reports but they do watch Fox News, CNBC, etc. and those outlets never ever get into uncertainty.
- mjburgess 2mo agoIf honest, the error bars would be so large as to make drawing conclusions from these numbers seem pointless. Certainly, both jobs growth and jobs loss are within the CI. What's actually being tracked by people using this data is the delta from previous years -- which, conditioning on the same collection methodology etc -- is more reliable. The actionable piece of info is sign(delta)
- hrhdhdh 2mo ago[flagged]
- rented_mule 2mo agoSomething the best data scientist I ever worked with was adamant about... there is almost never an inherent "speed bump" at 0. If error bounds expand on both sides of 0, we have no idea if the true sign is positive or negative. There is no "directional" read in that case. There are some areas where 0 has a special meaning, but that's rare and the cause of that needs to be understood before assuming that's the case. Everywhere else, 0 is just another point on the number line. > the error bars would be so large as to make drawing conclusions from these numbers seem pointless That is often the case, and being truly data based requires acknowledging that. We can then act based on other data, or admit that we're falling back to intuition and abandoning data based decision making for now.
- biophysboy 2mo agoThe bureau of labor statistics publishes confidence intervals in their reports
- jmalicki 2mo agoNot in their primary news release. A footnote says it is "generally conducted at the 90% interval", but the individual numbers do not have their actual computed error bars. Yes, they do publish them, but it is disappointing their primary releases do not have them. https://www.bls.gov/news.release/pdf/empsit.pdf https://www.bls.gov/news.release/pdf/empsit.pdf
- biophysboy 2mo agoIt’s common in scientific/stats publishing to have the headline state the avg/median, the main text plot the errors, and the supplementary give the explicit numbers. They have a big page about std errors.
- jmalicki 2mo agoThe main text doesn't plot the error bars, and the full data tables in their release don't have error bars either. You have to really dig into their website to find their std errors.
- biophysboy 2mo agobls.gov/ces is the main page for this survey and has a plot. The error updates annually. Their handbook has a Reliability section where they detail sampling/nonsampling error.
- throwaway27448 2mo agoNBC wouldn't know how to navigate bls.gov if you put a gun to their head. And it's all downhill from there
- 2mo ago
- amelius 2mo agoYes they should have said 23.0 * 10^3.
- inigyou 2mo ago2.30 × 10⁴ or bust
- mrloopex 2mo agoThis should have nothing to do with data science but basic math literacy. Quantities without context are meaningless. Reporters should know better but if when headlines have to catchy day in day out … shrugs. Stock market reporters are the worst: a 1000 point drop in the Dow meant a lot more when it was 7000 than it does at 40,000.
- astro1234 2mo agoOh my god do not trigger me about the news reporting “points” for the DOW or the S&P, I have seriously considered calling news stations about this. Or not providing inflation adjusted values when referring to historical financial reference points. I have to go find a happy place now
- selicos 2mo agoThe BLS has been gutted. It's surprising anything negative is escaping in these reports.
- splatter9859 2mo agoExactly. I'm somewhat surprised someone's give-a-damn wasn't broken enough to do a cursory filter. Not like it isn't plainly obvious in other statistics, however.
- jewslop 2mo ago[dead]
- seanhunter 2mo agoThe actual source this comes from has a 90% confidence interval in the published number. When people talk about the economy gaining or losing jobs they are talking about the bureau of labor statistics "nonfarm payroll" number. The methodology and a lot more are on the bls website https://www.bls.gov/ces/ https://www.bls.gov/ces/
- anigbrowl 2mo agoWait until you see the weekly reports for the Strategic Petroleum reserve. Go ahead, try to highlight that text. https://www.spr.doe.gov/dir/dir.html https://www.spr.doe.gov/dir/dir.html