4 ms·

The essay is a classic. > Shantytowns are squalid, sprawling slums. What is it that they are doing right? > Shantytowns are usually built from common, inexpen

The essay is a classic. > Shantytowns are squalid, sprawling slums. What is it that they are doing right? > Shantytowns are usually built from common, inexpensive materials and simple tools. Shantytowns can be built using relatively unskilled labor. This is a powerful metaphor, and I see it often in code. If you've ever seen an API gateway written using the same web framework as the other services, where in order to route "/external/order" to "/internal/order", the first step steps are e.g. "File | new | Controller", call it "ExternalController", make a GET method called "order", and in there code up a http call to "internal/order" ... you have a shantytown API gateway. It's terrible, it requires manual extending for any new endpoint. But it's not a specialised skill. Anyone on the dev team can extend it by just adding more of the same.

vikiomega9 7y ago
Well, maybe the good thing here is that at some point someone will notice that the specific gateway code can be extracted and then refactored? Idk there's probably a reason why they decided to do this, the most common answer I've gotten is, product requires feature out tomorrow. How does one reconcile this problem of competing priorities that aren't actually sized right (ie. short term feature development versus long term code health?)
- Jtsummers 7y ago
  You have to get management and customer support, and that's the hard part. They have to understand the idea of the capability trap [0][1]. A lack of maintenance is inviting disaster, this is true across domains. Software is cheap, though, and usually doesn't have terrible consequences (see the notable exceptions like Therac-25 or 737 Max). Who cares if your IM client is a POS, it's not going to cost lives. And even if it is junk, we can buy a new one (MS) or build a 17th one (Google). The way around this, and again you have to get management and customer support, is to bring maintenance activities into every iteration. Create systems that can detect issues and errors early (more comprehensive testing, new testing styles like fuzzing and property-based testings; moving critical portions to statically typed languages or adding type annotations; etc.). Make addressing those issues a priority, some portion of your time in each iteration should be spent addressing concerns or increasing the ability to detect issues. Over time, when I've seen this done, it's made making changes much faster, even for larger changes. But without this, even "trivial" changes can end up taking months, instead of days or weeks. [0] https://web.mit.edu/nelsonr/www/Repenning=Sterman_CMR_su01_.pdf https://web.mit.edu/nelsonr/www/Repenning=Sterman_CMR_su01_.... [1] https://www.systemdynamics.org/assets/conferences/2017/proceed/papers/P1325.pdf https://www.systemdynamics.org/assets/conferences/2017/proce...
  - vikiomega9 7y ago
    Some what of a random question, how did you find [0] and [1]?
    - Jtsummers 7y ago
      I don’t recall now. I’d heard of Sterman from various other readings on business and system dynamics and a recommendation from here. I can’t recall now if I found these papers from googling his name or a recommendation from my sister. I think she’s the one that gave me the phrase “capability trap” and Google found these and other articles.
      - vikiomega9 7y ago
        Awesome thank you, I was just trying to figure what the current state of your mental model on the subject is :)
- SideburnsOfDoom 7y ago
  > the specific gateway code can be extracted and then refactored? I In this case it's generally an attractive nuisance for business logic, aggregation, data transformation etc, which means that after a while 1) extracting the proxying is hard and 2) you can't understand a service without reading the code in multiple layers, since some of it's business logic is in the proxy layer. Big balls of mud can be big, distributed balls of mud.
droithomme 7y ago
> If you've ever seen an API gateway written using the same web framework as the other services, where in order to route "/external/order" to "/internal/order", the first step steps are e.g. "File | new | Controller", call it "ExternalController", make a GET method called "order", and in there code up a http call to "internal/order" ... you have a shantytown API gateway. That's not a shantytown. That's an architecture astronaut utopia, creating according to the latest methodologies.
mc3 7y ago
The advantage is when a new person starts. "How does this work?", oh here is a link to a well explained document on this thing called "Rails". Oh you know "Rails" already, of course you do, that's why we hired you. Where as with a customized "elegant" meta-programming DRY NIH solution unique to ThisCorp. "How does this work?". "Try to figure it out yourself. Jack has left and Ryan is too busy fixing production issues to help you". Obviously if you have a team of Haskell compiler writing geniuses YMMV.
- SideburnsOfDoom 7y ago
  > Where as with a customized "elegant" meta-programming DRY NIH solution unique to ThisCorp. In this specific case, I wouldn't recommend that either. e.g. there are plenty of products in the API gateway space that do that thing well. e.g. NGINX, Kong, Istio, Apigee etc. They're not "unique to ThisCorp" but neither are they well-known to many Ruby / Node / Java / C# coders. It's a separate, specialised knowledge. I think that's the "relatively unskilled labor" part: i.e. "yes, there are specialised tools that are excellent at solving this problem .... but we don't know them"

vikiomega9 7y ago

Well, maybe the good thing here is that at some point someone will notice that the specific gateway code can be extracted and then refactored? Idk there's probably a reason why they decided to do this, the most common answer I've gotten is, product requires feature out tomorrow. How does one reconcile this problem of competing priorities that aren't actually sized right (ie. short term feature development versus long term code health?)

Jtsummers 7y ago
You have to get management and customer support, and that's the hard part. They have to understand the idea of the capability trap [0][1]. A lack of maintenance is inviting disaster, this is true across domains. Software is cheap, though, and usually doesn't have terrible consequences (see the notable exceptions like Therac-25 or 737 Max). Who cares if your IM client is a POS, it's not going to cost lives. And even if it is junk, we can buy a new one (MS) or build a 17th one (Google). The way around this, and again you have to get management and customer support, is to bring maintenance activities into every iteration. Create systems that can detect issues and errors early (more comprehensive testing, new testing styles like fuzzing and property-based testings; moving critical portions to statically typed languages or adding type annotations; etc.). Make addressing those issues a priority, some portion of your time in each iteration should be spent addressing concerns or increasing the ability to detect issues. Over time, when I've seen this done, it's made making changes much faster, even for larger changes. But without this, even "trivial" changes can end up taking months, instead of days or weeks. [0] https://web.mit.edu/nelsonr/www/Repenning=Sterman_CMR_su01_.pdf https://web.mit.edu/nelsonr/www/Repenning=Sterman_CMR_su01_.... [1] https://www.systemdynamics.org/assets/conferences/2017/proceed/papers/P1325.pdf https://www.systemdynamics.org/assets/conferences/2017/proce...
- vikiomega9 7y ago
  Some what of a random question, how did you find [0] and [1]?
  - Jtsummers 7y ago
    I don’t recall now. I’d heard of Sterman from various other readings on business and system dynamics and a recommendation from here. I can’t recall now if I found these papers from googling his name or a recommendation from my sister. I think she’s the one that gave me the phrase “capability trap” and Google found these and other articles.
    - vikiomega9 7y ago
      Awesome thank you, I was just trying to figure what the current state of your mental model on the subject is :)
SideburnsOfDoom 7y ago
> the specific gateway code can be extracted and then refactored? I In this case it's generally an attractive nuisance for business logic, aggregation, data transformation etc, which means that after a while 1) extracting the proxying is hard and 2) you can't understand a service without reading the code in multiple layers, since some of it's business logic is in the proxy layer. Big balls of mud can be big, distributed balls of mud.

droithomme 7y ago

> If you've ever seen an API gateway written using the same web framework as the other services, where in order to route "/external/order" to "/internal/order", the first step steps are e.g. "File | new | Controller", call it "ExternalController", make a GET method called "order", and in there code up a http call to "internal/order" ... you have a shantytown API gateway. That's not a shantytown. That's an architecture astronaut utopia, creating according to the latest methodologies.

mc3 7y ago

The advantage is when a new person starts. "How does this work?", oh here is a link to a well explained document on this thing called "Rails". Oh you know "Rails" already, of course you do, that's why we hired you. Where as with a customized "elegant" meta-programming DRY NIH solution unique to ThisCorp. "How does this work?". "Try to figure it out yourself. Jack has left and Ryan is too busy fixing production issues to help you". Obviously if you have a team of Haskell compiler writing geniuses YMMV.

SideburnsOfDoom 7y ago
> Where as with a customized "elegant" meta-programming DRY NIH solution unique to ThisCorp. In this specific case, I wouldn't recommend that either. e.g. there are plenty of products in the API gateway space that do that thing well. e.g. NGINX, Kong, Istio, Apigee etc. They're not "unique to ThisCorp" but neither are they well-known to many Ruby / Node / Java / C# coders. It's a separate, specialised knowledge. I think that's the "relatively unskilled labor" part: i.e. "yes, there are specialised tools that are excellent at solving this problem .... but we don't know them"