7 ms·

I want to believe you, but there are just so many complicated problems IN REALITY that we have to model in software that I don't really see complexity going dow

I want to believe you, but there are just so many complicated problems IN REALITY that we have to model in software that I don't really see complexity going down. Just an example: around here, most people have a first (given) and a last (family) name. If I don't model that as separate, I have trouble interfacing with other software. If I do, I have trouble with people from other cultures that don't follow that convention. Storing both risks the data going out of sync. What's the "right" way to store person names? There doesn't seem to be a simple solution. Another example: we model physical cables (both for power grid and for data transmission) in our CMDB. All works fine, until you suddenly have a Y-shaped cable with three connectors that doesn't fit into your data model. The real world always has these 1% of cases that don't fit the general pattern; if you focus on the 99%, the 1% make trouble. If you focus on modeling every case, you have 10x the complexity, even for the simple case. And then there are things that are moderately complex and security critical, like password recovery workflows. We haven't really found a way to reuse these among different technologies. Like, if you once figured out the perfect password reset workflow with Ruby on Rails, and your next job uses Python + Django, you're back to square one. If somebody has a good idea for how to tackle these problems, please let me know!

nightski 4y ago
As you implied, I don't think the answer is to make the perfect abstractions that can handle any scenario. But where software does fail in my humble opinion is making it easy to pull in tried and true tested solutions to the problems that we do face, even if they are not as common. Because even though they may not seem common, I'm absolutely certain many face the same scenario. The amount of duplication solving the same problems is insane. But this is not an easy problem to solve and I don't intend to trivialize it. I think we need to come up with better tools. code sharing through libraries/repositories (like npm) is great, but it can't be the final solution. Back in the day in Haskell I dreamed of a system where you could type out a type signature and a fully tested rated implementation would be imported from an "open source" service. You could import modules, functions, data structures, anything. But that vision is still a long ways off.
- feoren 4y ago
  The problem is "pulling in a tried and tested solution" means pulling in a slice of the whole stack: CSS, Javascript, client/server communication, API endpoints, data-flow logic, and database schema. People are very very bad at this because they believe that software engineering principles apply only to little tiny portions of this stack. If you actually apply your software engineering principles to the whole stack, you end up with something that looks very different than what everyone else is doing (so there's not a lot of tooling to help with). I have a better way to do this than anything else I've ever seen, but even with that I struggly very strongly with the JS/API side.
  - 7thaccount 4y ago
    Kinda reminds me of what they do in some areas of the finance world with Kx System's Kdb+. That entire install is tiny tiny and you get a high performance array language for querying your SSD database with either the main language "k" or a syntactical sugar dialect sitting on top called "q" that is more like SQL. I think of that as optimizing the entire stack for what the customer wants of rapid/high-speed analytics.
    - retcon 4y ago
      KDB is unique for translating "in house" tech to marketable CV entry status. Having Haskell and CS(FB) experience requires educating the recruitment agencies, for comparable recognition. Personally I consider every ill in computing as symptoms of the intellectual property system, from making reading potential prior art (aka learning) verboten to non disclosures making the Peter Principle look like the common cold to covid.
      - 7thaccount 4y ago
        I had trouble following your message. You've stated that Kdb+ looks good on a resume, Haskell requires explanation, and IP is bad? I'm not a big fan of closed source myself (and Kx Systems sounds like it would have a lot of red tape), but the technology is supposedly very fast and I find the language to be very simple and charming. I'd love for our analytics database to migrate , but I think the technology would really confuse our IT folks and it would be difficult to describe why it would make our jobs easier.
pphysch 4y ago
> Names Don't try to force schemas onto schema-less data. Store the "name" as a JSON string/blob representing the various possible attributes (given, middle, family, title, etc) and provide a variety of functions for representing that data. IF you really need to do this at all (for an internal app, you probably don't). > Physical links Include an Hardware Asset FK in your Link M2M table. Model each binary link explicitly, so a Y cable = 2-3 different Links that point to the same cable Asset. Or you can have single Links with M2M inputs/outputs. But definitely don't model Y cables explicitly. > Password recovery What is so complicated about this, specifically? Python and Ruby are completely different languages with no guarantees for interop.
- deelly 4y ago
  I could be wrong but using JSON as format for name is not a solution, its abstraction over possible solution. Suppose we store all names as JSON structure. Question - how people will create such structure from their names? Using form with dozens of fields and checkboxes? Second question: how we will display this structure? We have to find a way to convert this JSON into string to show name. And as result we will surely need function to convert this shown strings into JSON for sure. And how we can solve issue when some names could be presented as a bit different JSON structures? And how we can validate that these slightly different JSON structures are equal?
- shkkmo 4y ago
  Your solution assumes atleast a dozen incorrect things from the classic https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-...
  - pphysch 4y ago
    That's strange, because my one (1) assumption is that the name can be represented as a schema-less dictionary of zero or more Unicode strings. Heck, one of those strings could be an ID for a bitmap or SVG of their non-textual "name". If you can't do that, you have more pressing issues. Where are you getting "a dozen assumptions" from?
corrral 4y ago
> We haven't really found a way to reuse these among different technologies. Like, if you once figured out the perfect password reset workflow with Ruby on Rails, and your next job uses Python + Django, you're back to square one. C and C++ libraries with light language-specific wrappers largely serve this purpose, in practice. It's plausible that, say, a PHP postgres client lib and a Node postgres client lib will share nearly all their code—probably as C or C++. This doesn't get leveraged much aside from interfacing with daemons and sometimes for extremely complex e.g. media or crypto libraries, though. No-one's doing this for high-level software workflow building-blocks, like a password reset flow. [EDIT] as for the "why", I suspect it's because the things it's used for are far simpler than the things it isn't. A password reset flow can potentially need to interface with lots of different things, some of which may be custom to the project, some of which may vary with run-time input, et c. If you cut it down to only the parts that could truly be re-used anywhere, with all kinds of points where you can hook in as needed... you've only made about 5% of the work re-usable, so it's just about pointless.
- rubyskills 4y ago
  I'd say this is more true for C vs C++. It has been my experience that the interop between C and languages like Ruby is much easier, the tooling is usually much better (just use ffi!), and the complexity is less (since that is what we're aiming for right?)
- retcon 4y ago
  >> We haven't really found a way to reuse these among different technologies. The Common Language Environment on VMS, or its lesser(but infinitely more available hence virtuous by effect) imitation, dot net CLR . FFIs, if you bought a good environment. Sounds like if open source implementation of good foreign interfaces existed, the OP's problem wouldn't exist, let alone system level interoperability.
yellowapple 4y ago
> What's the "right" way to store person names? Store both a unified name field and separated given name / surname fields and let the user manage both. Yeah, this risks going out of sync, but that's the user's problem, not yours. Yeah, three fields are technically more complex than one or two, but it produces less complexity down the line.
- fragmede 4y ago
  So what happens when someone doesn’t have a surname? Falsehoods programmers believe about name: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-...
  - CRConrad 4y ago
    > So what happens when someone doesn’t have a surname? Then they leave that field empty.
nradov 4y ago
Falsehoods programmers believe about names: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... As a practical matter, the HL7 FHIR data model for names can work well enough for the vast majority of applications in almost any country. https://www.hl7.org/fhir/datatypes.html#HumanName https://www.hl7.org/fhir/datatypes.html#HumanName
a1369209993 4y ago
> What's the "right" way to store person names? struct Screenname { Matches<String,regex"\w+"> _; // TODO: ensure it is illegal to ask for users' real names // for now we'll just do the right thing unilaterally }
- fmakunbound 4y ago
  Works for email, phone numbers and addresses, too.
  - a1369209993 4y ago
    Actually, it doesn't. All of those are, well, addresses, and have some mechanical structure necessary for routing (admittedly, for phone numbers, the structure amounts to "any sequence of 0-9, #, *, and possibly those four wierd extra symbols with the fourth DTMF column tone"). You probably don't (or least shouldn't) care about that structure as long as it routes correctly, but it does exist. (Ie, you'd still use a string for that data, but it's not inherently incorrect not to, just pointless and error-prone.)
    - tesseract 4y ago
      I think the point is that someone else cares about the structure and sets its rules - which may change without warning or have complexity you are unaware of - and therefore if you are not the telephone company or the post office you are likely overstepping if you try to enforce structure rather than merely passing along what you were given.
yourapostasy 4y ago
The rabbit hole on names goes deep [1]. Lots of other fun rabbit holes. [2] For how to model physical objects like those cables, go see how Grainger does it as a working practical model. To really handle names properly, you need more context than the name in the presentation layer that many schemas take their modeling from can obtain. Government health care or similar widely-adopted encoding is sometimes Good Enough. If you want non-lossy exactitude however, then that's a much bigger scope (I'd be investigating a first-pass classifier with contextual hints taken from various geolocations, age, etc., that implies soliciting the name comes after what you normally solicit for input, and refining from there). Password reset; this is why vendors like Okta exist to abstract authN away for us, and auth0 for authZ. Then there is the rabbit hole of what this abstraction leads to... [1] https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... [2] https://github.com/kdeldycke/awesome-falsehood https://github.com/kdeldycke/awesome-falsehood
- mudbug 4y ago
lmm 4y ago
> Just an example: around here, most people have a first (given) and a last (family) name. If I don't model that as separate, I have trouble interfacing with other software. If I do, I have trouble with people from other cultures that don't follow that convention. Storing both risks the data going out of sync. What's the "right" way to store person names? There doesn't seem to be a simple solution. The way past bureaucracies that didn't have the luxury of unlimited cheap complexity did: you set some rules and anyone who doesn't like them can either suck it up and follow them, or deal with the consequences themselves. Allowing users to specify arbitrarily complex requirements and never saying "no" is what gets us into this problem.

nightski 4y ago

As you implied, I don't think the answer is to make the perfect abstractions that can handle any scenario. But where software does fail in my humble opinion is making it easy to pull in tried and true tested solutions to the problems that we do face, even if they are not as common. Because even though they may not seem common, I'm absolutely certain many face the same scenario. The amount of duplication solving the same problems is insane. But this is not an easy problem to solve and I don't intend to trivialize it. I think we need to come up with better tools. code sharing through libraries/repositories (like npm) is great, but it can't be the final solution. Back in the day in Haskell I dreamed of a system where you could type out a type signature and a fully tested rated implementation would be imported from an "open source" service. You could import modules, functions, data structures, anything. But that vision is still a long ways off.

feoren 4y ago
The problem is "pulling in a tried and tested solution" means pulling in a slice of the whole stack: CSS, Javascript, client/server communication, API endpoints, data-flow logic, and database schema. People are very very bad at this because they believe that software engineering principles apply only to little tiny portions of this stack. If you actually apply your software engineering principles to the whole stack, you end up with something that looks very different than what everyone else is doing (so there's not a lot of tooling to help with). I have a better way to do this than anything else I've ever seen, but even with that I struggly very strongly with the JS/API side.
- 7thaccount 4y ago
  Kinda reminds me of what they do in some areas of the finance world with Kx System's Kdb+. That entire install is tiny tiny and you get a high performance array language for querying your SSD database with either the main language "k" or a syntactical sugar dialect sitting on top called "q" that is more like SQL. I think of that as optimizing the entire stack for what the customer wants of rapid/high-speed analytics.
  - retcon 4y ago
    KDB is unique for translating "in house" tech to marketable CV entry status. Having Haskell and CS(FB) experience requires educating the recruitment agencies, for comparable recognition. Personally I consider every ill in computing as symptoms of the intellectual property system, from making reading potential prior art (aka learning) verboten to non disclosures making the Peter Principle look like the common cold to covid.
    - 7thaccount 4y ago
      I had trouble following your message. You've stated that Kdb+ looks good on a resume, Haskell requires explanation, and IP is bad? I'm not a big fan of closed source myself (and Kx Systems sounds like it would have a lot of red tape), but the technology is supposedly very fast and I find the language to be very simple and charming. I'd love for our analytics database to migrate , but I think the technology would really confuse our IT folks and it would be difficult to describe why it would make our jobs easier.

pphysch 4y ago

> Names Don't try to force schemas onto schema-less data. Store the "name" as a JSON string/blob representing the various possible attributes (given, middle, family, title, etc) and provide a variety of functions for representing that data. IF you really need to do this at all (for an internal app, you probably don't). > Physical links Include an Hardware Asset FK in your Link M2M table. Model each binary link explicitly, so a Y cable = 2-3 different Links that point to the same cable Asset. Or you can have single Links with M2M inputs/outputs. But definitely don't model Y cables explicitly. > Password recovery What is so complicated about this, specifically? Python and Ruby are completely different languages with no guarantees for interop.

deelly 4y ago
I could be wrong but using JSON as format for name is not a solution, its abstraction over possible solution. Suppose we store all names as JSON structure. Question - how people will create such structure from their names? Using form with dozens of fields and checkboxes? Second question: how we will display this structure? We have to find a way to convert this JSON into string to show name. And as result we will surely need function to convert this shown strings into JSON for sure. And how we can solve issue when some names could be presented as a bit different JSON structures? And how we can validate that these slightly different JSON structures are equal?
shkkmo 4y ago
Your solution assumes atleast a dozen incorrect things from the classic https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-...
- pphysch 4y ago
  That's strange, because my one (1) assumption is that the name can be represented as a schema-less dictionary of zero or more Unicode strings. Heck, one of those strings could be an ID for a bitmap or SVG of their non-textual "name". If you can't do that, you have more pressing issues. Where are you getting "a dozen assumptions" from?

corrral 4y ago

> We haven't really found a way to reuse these among different technologies. Like, if you once figured out the perfect password reset workflow with Ruby on Rails, and your next job uses Python + Django, you're back to square one. C and C++ libraries with light language-specific wrappers largely serve this purpose, in practice. It's plausible that, say, a PHP postgres client lib and a Node postgres client lib will share nearly all their code—probably as C or C++. This doesn't get leveraged much aside from interfacing with daemons and sometimes for extremely complex e.g. media or crypto libraries, though. No-one's doing this for high-level software workflow building-blocks, like a password reset flow. [EDIT] as for the "why", I suspect it's because the things it's used for are far simpler than the things it isn't. A password reset flow can potentially need to interface with lots of different things, some of which may be custom to the project, some of which may vary with run-time input, et c. If you cut it down to only the parts that could truly be re-used anywhere, with all kinds of points where you can hook in as needed... you've only made about 5% of the work re-usable, so it's just about pointless.

rubyskills 4y ago
I'd say this is more true for C vs C++. It has been my experience that the interop between C and languages like Ruby is much easier, the tooling is usually much better (just use ffi!), and the complexity is less (since that is what we're aiming for right?)
retcon 4y ago
>> We haven't really found a way to reuse these among different technologies. The Common Language Environment on VMS, or its lesser(but infinitely more available hence virtuous by effect) imitation, dot net CLR . FFIs, if you bought a good environment. Sounds like if open source implementation of good foreign interfaces existed, the OP's problem wouldn't exist, let alone system level interoperability.

yellowapple 4y ago

> What's the "right" way to store person names? Store both a unified name field and separated given name / surname fields and let the user manage both. Yeah, this risks going out of sync, but that's the user's problem, not yours. Yeah, three fields are technically more complex than one or two, but it produces less complexity down the line.

fragmede 4y ago
So what happens when someone doesn’t have a surname? Falsehoods programmers believe about name: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-...
- CRConrad 4y ago
  > So what happens when someone doesn’t have a surname? Then they leave that field empty.

nradov 4y ago

Falsehoods programmers believe about names: https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... As a practical matter, the HL7 FHIR data model for names can work well enough for the vast majority of applications in almost any country. https://www.hl7.org/fhir/datatypes.html#HumanName https://www.hl7.org/fhir/datatypes.html#HumanName

a1369209993 4y ago

> What's the "right" way to store person names? struct Screenname { Matches<String,regex"\w+"> _; // TODO: ensure it is illegal to ask for users' real names // for now we'll just do the right thing unilaterally }

fmakunbound 4y ago
Works for email, phone numbers and addresses, too.
- a1369209993 4y ago
  Actually, it doesn't. All of those are, well, addresses, and have some mechanical structure necessary for routing (admittedly, for phone numbers, the structure amounts to "any sequence of 0-9, #, *, and possibly those four wierd extra symbols with the fourth DTMF column tone"). You probably don't (or least shouldn't) care about that structure as long as it routes correctly, but it does exist. (Ie, you'd still use a string for that data, but it's not inherently incorrect not to, just pointless and error-prone.)
  - tesseract 4y ago
    I think the point is that someone else cares about the structure and sets its rules - which may change without warning or have complexity you are unaware of - and therefore if you are not the telephone company or the post office you are likely overstepping if you try to enforce structure rather than merely passing along what you were given.

yourapostasy 4y ago

The rabbit hole on names goes deep [1]. Lots of other fun rabbit holes. [2] For how to model physical objects like those cables, go see how Grainger does it as a working practical model. To really handle names properly, you need more context than the name in the presentation layer that many schemas take their modeling from can obtain. Government health care or similar widely-adopted encoding is sometimes Good Enough. If you want non-lossy exactitude however, then that's a much bigger scope (I'd be investigating a first-pass classifier with contextual hints taken from various geolocations, age, etc., that implies soliciting the name comes after what you normally solicit for input, and refining from there). Password reset; this is why vendors like Okta exist to abstract authN away for us, and auth0 for authZ. Then there is the rabbit hole of what this abstraction leads to... [1] https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-believe-about-names/ https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-... [2] https://github.com/kdeldycke/awesome-falsehood https://github.com/kdeldycke/awesome-falsehood

mudbug 4y ago

lmm 4y ago

> Just an example: around here, most people have a first (given) and a last (family) name. If I don't model that as separate, I have trouble interfacing with other software. If I do, I have trouble with people from other cultures that don't follow that convention. Storing both risks the data going out of sync. What's the "right" way to store person names? There doesn't seem to be a simple solution. The way past bureaucracies that didn't have the luxury of unlimited cheap complexity did: you set some rules and anyone who doesn't like them can either suck it up and follow them, or deal with the consequences themselves. Allowing users to specify arbitrarily complex requirements and never saying "no" is what gets us into this problem.