5 ms·
Bad data vs no data at all? I would think no data would put you out of a job while bad data would require more hires to filter the data.
by ipaddr 5y ago
Bad data vs no data at all? I would think no data would put you out of a job while bad data would require more hires to filter the data.
- klyrs 5y agoI prefer employees who say "I don't know" over confident bullshitters.
- judge2020 5y agoBad data is often taken as good data, because sifting through it incurs 100x more friction than taking it at face value. When you ultimately get bad results you can just blame the bad data, and you still end up with a paycheck for the month(s) you wasted.
- kevin_thibedeau 5y agoAuthoritarian countries collapse because everyone is lying about reality. Same thing happens with metric driven management.
- dredmorbius 5y agoNo data clearly indicates that there is no data. A statement of "I don't know' clearly indicates a lack of knowledge. A statemnt of "I have no opinion" clearly indicates that the speaker has not formed an opinion. In each case, a spurious generated response: 1. Is generally accepted as prima facie evidence of what it purports. 2. Must be specifically analysed and assessed. 3. Is itself subject to repetition and/or amplification. With empirical evidence suggesting that falsehoods outcompete truths, particularly on large networks operating at flows which overload rational assessment. 4. Competes for attention with other information, including the no-signal case specifically, which does very poorly against false claims as it is literally nothing competing against an often very loud something. Yes: bad data is much, much, much, much worse than no data.
- inkblotuniverse 5y agoData that's had data censored from it is bad data.
- dredmorbius 5y agoFalse. Outlier exclusion is standard practice. It's useful to note what is excluded. But you exclude bad data from the analysis. Remember that what you're interested in is not the data but the ground truth that the data represent. This means that the full transmission chain must be reliable and its integrity assured: phenomenon, generated signal, transmission channel, receiver, sensor, interpretation, and recording. Noise may enter at any point. And that noise has ... exceedingly little value. Deliberately inserted noise is one of the most effective ways to thwart an accurate assessment of ground truths.
- ineedasername 5y agoDefining terms here is important, so let's avoid the word bad for a moment because it can be applied in different ways. 1) You can have an empty dataset. 2) You can have an incomplete dataset. 3) you can have a dataset where the data is wrong All of these situations, in some sense, are "bad" What I'm saying is that, going into a situation, my preference would be #2 > #1 > #3. Because I always assume a dataset could be incomplete, that it didn't capture everything. I can plan for it, look for evidence that something is missing, try to find it. If I suspect something is missing but can't find it then I at least know that much, and maybe even the magnitude of uncertainty that adds to the situation. Either way, I can work around it understanding the limits if what I'm doing or if there's too much missing, make a judgement call and say that nothing useful can be done with it. If I have what appears to be a dataset that I can work with, but the data is all incorrect, I may never even know it until things start to break or, before that if I'm lucky, I waste large amounts of time to find out that the results just don't make sense. It's probably important to note that #2 and #3 are also not mutually exclusive. Getting out of the dry world of data analysis, if your job is propaganda & if you're good at your job, #2 and #3 combined is where you're at.
- inkblotuniverse 5y ago
- ALittleLight 5y agoAs a metaphor, you can imagine a blind person in the wilderness who has no idea what is in front of him. He will proceed cautiously, perhaps probing the ground with a stick or his foot. You could also imagine a delusional man in the same wilderness incorrectly believing he's in the middle of a foot race. The delusional man just run forward at full speed. If the pair are in front of a cliff... As the saying goes, it's not what you don't know that gets you into trouble. It's what you know for sure that just ain't so.
- ineedasername 5y agoNot quite: if you have no data, you get new hires and news systems to collect and track it. You may be ignorant, but you know it, and can deal with it. Let's call is starting from 0. When you have bad data, you frequently don't know that you have bad data until things go very very wrong. You aren't starting from 0. 0 would be an improvement.
- dredmorbius 5y agoThis seems like extending the "known knowns" concept to an additional dimension, involving truth. In the known-knowns model, you have knowledge and metaknowledge (what you know, what you know you know): K U -- What you know K KK KU U UK UU \ What you know you know If we add truth to that, you end up with a four-dimensional array with dimensions of knowledge, knowledge of knowledge, truth-value, and knowledge-of-truth-value. Rather than four states, there are now 16: TT TF FT FF (Truth & belief of truth) ---- ---- ---- ---- KK | KKTT KKTF KKFT KKFF KU | KUTT KUTF KKFT KKFF UK | UKTT UKTF UKFT UKFF UU | UUTT UUTF UUFT UUFF False information is the FT and FF columns. In both the TF and FT columns, belief of the truth-value of data is incorrect. In both the KU and UU columns, there is a lack of knowledge (e.g., ignorance), either known or unknown. (I'm still thinking through what the implications of this are. Mapping it out helps structure the situation.)
- ineedasername 5y agoReminds me of epistemology, where this is a distinction between certainty and truth.
- dredmorbius 5y agoYes, there's an element of that in this.