4 ms·

Looking back, I find it hard to offer concrete advice. So many of the "things I know to be true" changed over time, while so many of the "doing x to allow for f

Looking back, I find it hard to offer concrete advice. So many of the "things I know to be true" changed over time, while so many of the "doing x to allow for future y" didn't lead to anything. On the other hand, I have systems being worked on and still shipping 20 years old because of fortuitous decisions in the 90s. Most of those taken because of bad experiences with earlier systems. For example, I'm a big fan of surrogate keys and UUIDs, in database design. Mostly because of experience where they weren't used which caused problems later. (By contrast, performance costs of using them become lower every year.) Conversely deciding to build code libraries parallel to application development means long-term gains in performance and reliability. It's easy to view all this in hindsight, -much- harder with foresight. Today I try and see everything through 2 rules. 1) beware of "always" statements, especially especially in regard to client requirements. IME "always" is used as a synonym for "mostly" and "at the moment". 2) choose paths now that keep options open later. Will this system ever use distributed data? Probably not, but choose a primary key that leaves that option open. Lastly, and this applies to data design, remember "nothing is universal". Using Social Security number as an ID is great, until they outsource. Phone number is always 8 digits? Yeah right. Nobody works night shift... you get the idea. If in doubt, err on the side of "variant". You'll be right more often than not.

yafetn 3y ago
> 2) choose paths now that keep options open later. Not sure about this one. I’ve seen teams that are too noncommittal in their architecture decisions because “what if [insert unlikely thing that’d not actually be the end of the world] changes?” Then the project ends up only using surface level features of their chosen tooling because people want to keep too many options open and are afraid to commit to something where it matters.
- bruce511 3y ago
  There's no good idea or pattern that some team, somewhere, can't warp to create a bad outcome. Generally speaking there are lots of well-established best-practices, that are best in most contexts, but which fall over in other contexts. Knowing what to do where is part of the intangibleness that separates the great from the good from the competent. So some things are good to defer. -if- this product works, we -may- need to scale, so let's choose a database, and design, that allows for multiple servers. Picking say Postgres over SQLite. Other times saying "this is just a proof of concept, do it quick in SQLite". Is our API gonna be XML or JSON? Not sure, could go either way. Let's design so that part can be trivially replaced. With data design especially, IMO, it's worth -planning- for success. Starting with uuid will end up fine for 99% of the tables which will have "not many records". When we identify that 1%, which are growing by millions of records a minute, we can adjust for that table as desired. [On a side note, regarding clustering, it seems some people aren't aware that the primary key and clustered key don't have to be the same.]
beachy 3y ago
> For example, I'm a big fan of surrogate keys and UUIDs, in database design. You would think this (using surrogate keys) would be such well worn wisdom by now that discussions about it wouldn't be a thing. But somehow new generations of developers, weaned on nosql and on not using the god-given gifts of databases like integrity constraints, seem to love bike shedding key design and arguing vociferously that natural keys can be used as primary (and foreign) keys.
- ThereIsNoWorry 3y ago
  You wouldn't believe how often I have to fight for UUIDs instead of sequencing. UUIDs are great. For all practical purposes 0 possibility of collision; you can use it as an ID in a global system, it just makes so much fucking sense. But the default is still a number natural number sequence. As if it matters for 99% of all cases that stuff is "ordered" and "easily identifiable" by a natural number. But then you want to merge something or make double use and suddenly you have a huge problem that it isn't unique anymore and you need more information to identify. Guess what, an UUID does that job for you, across multiple databases and distributed systems, a UUID is still unique with 99.9999% probability. The one counter-example every 10 years can be cared for manually.
  - beachy 3y ago
    As far as I know the only practical downside with UUIDs in the modern age - unless you like using keys for ordering by creation time, or you have such enormous volumes that storage is a consideration - is that they are cumbersome for humans to read and compare, e.g scanning log files. And in any case the German tank problem means that you often can't use incrementing numbers as surrogate keys if they are ever exposed to the public e.g in urls.
    - mrkeen 3y ago
      > they are cumbersome for humans to read and compare, e.g scanning log files. I think the positive still outweighs the negative here: you can search your whole company's logs for a UUID and you won't get false positives like you would with serial integers.
Terr_ 3y ago
I like to say "design for deletion", meaning the priority is making sure code with outdated assumptions or fit can be found, deleted, and replaced easily. This is in contrast to my how my younger-self would instead focus on making code that "extensible" or "flexible" or "configurable". (With some overtones of impressing people, leaving a long-term mark upon the project, etc.) Nope! Go for "delete-able."
- vsnf 3y ago
  I've never heard this framing before. Can you offer any examples of what you mean?
  - Terr_ 3y ago
    A lot of it overlaps with Ya Ain't Gonna Need It and avoiding strong coupling, but I think the framing makes it easier to stay on target: A developer is less-likely to end up going: "Hey guys, I created a decoupling framework so we can swap anything for anything whenever we want!" If you design thinking "X years from now business-requirements or staff will have changed enough that this will need to be ripped out and rewritten, and I am not good enough at predicting the future to solve it cleverly", then it sets up a different expectation of how you invest your time. One might focus on code where it's easy to trace dependencies, even if that means a bit more boilerplate and avoiding layers of indirection that aren't (yet/ever) needed. "Greppability" of code also becomes important, especially if your language/tools aren't very good at tracing usages and caller/callee relationships.
    - whstl 3y ago
      Yep, you nailed it. Often the problem is exactly this magic "decoupling framework", which ends up being exactly the problematic part that that one eventually wants to swap out.
      - Terr_ 3y ago
        I've been trying to think of a micro-example from some refactoring I've been doing lately. Long ago somebody made a Thingy.find(...) method where you can pass all sorts of optional arguments to filter which Thingy comes back, like Thingy.find(primary_key=1) or Thingy.find(owner=123, active=true) etc. That complexity made it flexible for people writing caller-code, because they wouldn't need to alter the method to satisfy their future arbitrary needs, they can just pass whatever key-values they want to match. However now some original invariants have changed, and now I need to go find all the places it's being used ("flexibly") and reconstruct the caller's intent, because in some scenarios the correct return value is going to be a list of multiple results instead of 0-or-1. In contrast, the task would be easier if there was a named method that captured the intent for the 3-4 distinct use-cases.
magicalhippo 3y ago
> beware of "always" statements, especially especially in regard to client requirements. Along with its complimentary statement: Beware of "never", especially in regards to client requirements. Been on too many projects where it turned out "never" meant once or twice per year once we'd gone into production...
- bluGill 3y ago
  I put it a different way: programmers should gain domain knowledge instead of reading requirement only. If you have an understanding of the customers you can make a good call 90% of the time as to what customers will demand in the future without marketing misunderstanding. Marketing often wants todays features now and so will tell you "never" when what they really mean is don't make the next release take longer to add that future feature, but if you can design for it without making the schedule take too much longer do it.
  - magicalhippo 3y ago
    Having domain knowledge is a superpower of sorts. We're a small team doing B2B software, and especially us senior programmers and the head of the programming team has a lot of domain knowledge. Thus as you say we can very often immediately figure out when something is missing from the requirements, or when we feel things might be changing soon down the line. We can also detect when the customer is requesting something suboptimal or similar. We've received a lot of positive feedback from customers due to us devs pushing back, helping them to improve their processes and workflow. But it also leads to less work for us, or more generic code which can be reused by more customers. Our support team also has extensive domain knowledge, many have been hired from our customers. It makes our support excellent, often the customer's issue is a combination of how to do X and how to do X in our program, and for new devs this is a great resource to tap into. Together we're punching well above our weight, dominating our niche with a quite small team. And domain knowledge has helped a lot in that.
xxs 3y ago
>For example, I'm a big fan of surrogate keys and UUIDs, in database design. Why I fully support surrogate keys and versioning, I do not support UUIDs as primary keys - they are trivial to generate, not so much to place in B-trees. It's very common to need/process recently added data together, UUID make that much harder as they inherent placement is all over the place w/o any locality. UUIDs are ok/fine as external references, of course.
- antonyt 3y ago
  This is a solved problem. See UUIDv6 and higher.

yafetn 3y ago

> 2) choose paths now that keep options open later. Not sure about this one. I’ve seen teams that are too noncommittal in their architecture decisions because “what if [insert unlikely thing that’d not actually be the end of the world] changes?” Then the project ends up only using surface level features of their chosen tooling because people want to keep too many options open and are afraid to commit to something where it matters.

bruce511 3y ago
There's no good idea or pattern that some team, somewhere, can't warp to create a bad outcome. Generally speaking there are lots of well-established best-practices, that are best in most contexts, but which fall over in other contexts. Knowing what to do where is part of the intangibleness that separates the great from the good from the competent. So some things are good to defer. -if- this product works, we -may- need to scale, so let's choose a database, and design, that allows for multiple servers. Picking say Postgres over SQLite. Other times saying "this is just a proof of concept, do it quick in SQLite". Is our API gonna be XML or JSON? Not sure, could go either way. Let's design so that part can be trivially replaced. With data design especially, IMO, it's worth -planning- for success. Starting with uuid will end up fine for 99% of the tables which will have "not many records". When we identify that 1%, which are growing by millions of records a minute, we can adjust for that table as desired. [On a side note, regarding clustering, it seems some people aren't aware that the primary key and clustered key don't have to be the same.]

beachy 3y ago

> For example, I'm a big fan of surrogate keys and UUIDs, in database design. You would think this (using surrogate keys) would be such well worn wisdom by now that discussions about it wouldn't be a thing. But somehow new generations of developers, weaned on nosql and on not using the god-given gifts of databases like integrity constraints, seem to love bike shedding key design and arguing vociferously that natural keys can be used as primary (and foreign) keys.

ThereIsNoWorry 3y ago
You wouldn't believe how often I have to fight for UUIDs instead of sequencing. UUIDs are great. For all practical purposes 0 possibility of collision; you can use it as an ID in a global system, it just makes so much fucking sense. But the default is still a number natural number sequence. As if it matters for 99% of all cases that stuff is "ordered" and "easily identifiable" by a natural number. But then you want to merge something or make double use and suddenly you have a huge problem that it isn't unique anymore and you need more information to identify. Guess what, an UUID does that job for you, across multiple databases and distributed systems, a UUID is still unique with 99.9999% probability. The one counter-example every 10 years can be cared for manually.
- beachy 3y ago
  As far as I know the only practical downside with UUIDs in the modern age - unless you like using keys for ordering by creation time, or you have such enormous volumes that storage is a consideration - is that they are cumbersome for humans to read and compare, e.g scanning log files. And in any case the German tank problem means that you often can't use incrementing numbers as surrogate keys if they are ever exposed to the public e.g in urls.
  - mrkeen 3y ago
    > they are cumbersome for humans to read and compare, e.g scanning log files. I think the positive still outweighs the negative here: you can search your whole company's logs for a UUID and you won't get false positives like you would with serial integers.

Terr_ 3y ago

I like to say "design for deletion", meaning the priority is making sure code with outdated assumptions or fit can be found, deleted, and replaced easily. This is in contrast to my how my younger-self would instead focus on making code that "extensible" or "flexible" or "configurable". (With some overtones of impressing people, leaving a long-term mark upon the project, etc.) Nope! Go for "delete-able."

vsnf 3y ago
I've never heard this framing before. Can you offer any examples of what you mean?
- Terr_ 3y ago
  A lot of it overlaps with Ya Ain't Gonna Need It and avoiding strong coupling, but I think the framing makes it easier to stay on target: A developer is less-likely to end up going: "Hey guys, I created a decoupling framework so we can swap anything for anything whenever we want!" If you design thinking "X years from now business-requirements or staff will have changed enough that this will need to be ripped out and rewritten, and I am not good enough at predicting the future to solve it cleverly", then it sets up a different expectation of how you invest your time. One might focus on code where it's easy to trace dependencies, even if that means a bit more boilerplate and avoiding layers of indirection that aren't (yet/ever) needed. "Greppability" of code also becomes important, especially if your language/tools aren't very good at tracing usages and caller/callee relationships.
  - whstl 3y ago
    Yep, you nailed it. Often the problem is exactly this magic "decoupling framework", which ends up being exactly the problematic part that that one eventually wants to swap out.
    - Terr_ 3y ago
      I've been trying to think of a micro-example from some refactoring I've been doing lately. Long ago somebody made a Thingy.find(...) method where you can pass all sorts of optional arguments to filter which Thingy comes back, like Thingy.find(primary_key=1) or Thingy.find(owner=123, active=true) etc. That complexity made it flexible for people writing caller-code, because they wouldn't need to alter the method to satisfy their future arbitrary needs, they can just pass whatever key-values they want to match. However now some original invariants have changed, and now I need to go find all the places it's being used ("flexibly") and reconstruct the caller's intent, because in some scenarios the correct return value is going to be a list of multiple results instead of 0-or-1. In contrast, the task would be easier if there was a named method that captured the intent for the 3-4 distinct use-cases.

magicalhippo 3y ago

> beware of "always" statements, especially especially in regard to client requirements. Along with its complimentary statement: Beware of "never", especially in regards to client requirements. Been on too many projects where it turned out "never" meant once or twice per year once we'd gone into production...

bluGill 3y ago
I put it a different way: programmers should gain domain knowledge instead of reading requirement only. If you have an understanding of the customers you can make a good call 90% of the time as to what customers will demand in the future without marketing misunderstanding. Marketing often wants todays features now and so will tell you "never" when what they really mean is don't make the next release take longer to add that future feature, but if you can design for it without making the schedule take too much longer do it.
- magicalhippo 3y ago
  Having domain knowledge is a superpower of sorts. We're a small team doing B2B software, and especially us senior programmers and the head of the programming team has a lot of domain knowledge. Thus as you say we can very often immediately figure out when something is missing from the requirements, or when we feel things might be changing soon down the line. We can also detect when the customer is requesting something suboptimal or similar. We've received a lot of positive feedback from customers due to us devs pushing back, helping them to improve their processes and workflow. But it also leads to less work for us, or more generic code which can be reused by more customers. Our support team also has extensive domain knowledge, many have been hired from our customers. It makes our support excellent, often the customer's issue is a combination of how to do X and how to do X in our program, and for new devs this is a great resource to tap into. Together we're punching well above our weight, dominating our niche with a quite small team. And domain knowledge has helped a lot in that.

xxs 3y ago

>For example, I'm a big fan of surrogate keys and UUIDs, in database design. Why I fully support surrogate keys and versioning, I do not support UUIDs as primary keys - they are trivial to generate, not so much to place in B-trees. It's very common to need/process recently added data together, UUID make that much harder as they inherent placement is all over the place w/o any locality. UUIDs are ok/fine as external references, of course.

antonyt 3y ago
This is a solved problem. See UUIDv6 and higher.