30 ms·
> Your scientists would prefer you keep all measurements, even the bad ones. Sometimes those "bad" data actually have information. A constraint can allow "bad"
by midev 9y ago
> Your scientists would prefer you keep all measurements, even the bad ones. Sometimes those "bad" data actually have information.
A constraint can allow "bad" data if the "bad" data is still semantically valid for that column.
A freeform text column where the scientist writes notes might not need any constraints. But a column used to send measurements to a robot to manufacture something, must have some protections around what type of data it can store. And no, the scientist doesn't get to input "bad" data that makes the robot lose its mind.
Sometimes yes, you drop data on the floor.
- xapata 9y agoI've seen it go the other way plenty of times -- the dev thinks a system should only receive numbers, but occasionally a string is valid as well.
- always_good 9y agoThen the data model was wrong and they migrate their scheme appropriately. I don't think you understand that this is a core part of why you want a strict database: so you can perceive mismatches between your assumptions and reality and then respond precisely. Do you think it's superior to be in a situation where you thought all of your IDs were numbers, you built your system with that expectation, and it turns out strings have been silently making their way into your data? The difference between the two scenarios is that the person that used constraints knows when and why and where they got unexpected data and they can take more decisive, informed action. Meanwhile the other guy is desperately writing queries to find unexpected data, shooting around in the dark, and wondering what else slipped through.
- hinkley 9y agoUnfortunately he sounds like about half the Node developers I work with. Nobody wants to make concrete assertions. To decide anything. Usually this count field contains a number, but sometimes... It’s a miracle our stuff works as well as it does. A more patient man could probably create a good deal of programming literature trying to unpack that paradox. Instead I’m hunting for something where people are a little less concerned about job security and a little more open to mentoring, robustness, and low drama development processes.
- xapata 9y agoYou want openness to being mentored, but you're rejecting the possibility you're incorrect?
- hinkley 9y agoThe possibility that the big ball of mud people are right? That hero worshipping is better than team building? That the people who still write software the way it was done 25 years ago know something that the rest of us don’t? That all of the techniques and tools I pressed my face against the glass for 12 years ago that are now considered de rigeur are bad? That hoping for the other half of things I wish for will happen in the next 12 is a pipe dream? No, I’m not open to that kind of education. I’ve had plenty of volunteers to teach me if I ever change my mind. In the meantime I’ll try to work in the half (quarter?) of the industry that strives for something better. As someone once told me, I don’t need to be a good fit for every job. I just need to be a good fit for one at a time.
- xapata 9y agoNot sure where all those rhetorical questions came from. They seem like non-sequiturs.
- hinkley 9y agoMy point is that there is only so much you can learn from someone exhibiting a raft of bad actor behaviors. Life is short. Find better mentors instead of trying to eek out a little wisdom from bad ones.
- xapata 9y agoHow do you know who is a "bad one"? If you discount the "little wisdom" of people you disagree with, you're creating a a confirmation bias problem. A good scientist tries to find evidence that contradicts her theory. Evidence that is consistent with the theory proves nothing.
- xapata 9y agoIf the data model was wrong and we must migrate, then I hope we were storing all the records that were falsely labeled incorrect. It's a constant fight between analyst and application developer. The analyst wants a history of all states. The application developer often only cares about having a correct current state.
- midev 9y ago> It's a constant fight between analyst and application developer. The analyst wants a history of all states. The application developer often only cares about having a correct current state. Again, this is totally and completely wrong. Actually, it doesn't even make sense, to say it's wrong. If I'm building an application to be used by an analyst, I want to hit all their requirements. They tell me what sort of data they want to be collected. If requirements change, or the data isn't as valuable as originally thought, or it's badly formatted, or whatever, then we work together to change it. I don't fight with anyone.... They don't want shit data, and I don't want to store shit data. So who is there to fight with?
- xapata 9y agoThe case I'm considering isn't an application to be used by an analyst, but the analysis of an application used for some other purpose. It's often the case that an application only stores the current state. When there's historical data, it's often snapshots at regular intervals instead of an event log. Anyway, we're going way down a rabbit hole. All I wanted to say is that if the user, sensor, or application logic emits something weird, sometimes it's correct.