14 ms·

A Codebase Is an Organism (2014)

115 points by Kortaggio 7y ago

taneq 7y ago
I've been waiting for a long time for people to start using the title "robopsychologist" in earnest. Especially for people who are called out to debug an unfamiliar system, in 'the wild' (ie. in production). You have to calm the system down, find out what it thinks is going on, gently correct any misconceptions, and guide it towards a better understanding of the world that lets it behave the way you need it to.
- andy_ppp 7y ago
  It’s commentary like this that makes me come back here and also I wish hacker news had a few more features, like being able to follow people who make interesting comments.
  - grawprog 7y ago
    You could always start a bookmarks folder for 'HNers I find Interesting' and save their profile to that. I bet there's a way to write a script that goes through them, checks for posts since the last update and display them.
- lioeters 7y ago
  Good one, yes I can imagine the field branching out and developing disciplines like Software Psychiatry, System Nursing, Geriatric Care for Legacy Codebases.. Coming soon: Diagnostic and Statistical Manual of Software Disorders.
gandalfgeek 7y ago
Not just code bases, but it is useful to think of large planet scale systems in organic terms. https://blog.vivekhaldar.com/post/6972614229/large-computer-systems-are-organic https://blog.vivekhaldar.com/post/6972614229/large-computer-...
- goldenkey 7y ago
  Might as well just ask the question: What is Life? And then realize that scale of space or time is irrelevant. A galaxy could be thinking like a brain, except thoughts travel light years, are mainly dictated by GR (Light and Gravity), and take eons to actually occur. I think it's clear that the only meaningful nature of scale in time and space for life is just the one we artificially put on it, because of our conscious experience. LOTR and other fantasy explore this with the idea of trees having much wisdom having lived so long.
  - dwaltrip 7y ago
    I once saw a comment similar to yours on HN, and there was a very good reply that critiqued the idea. I'll do my best to replicate something along the same lines. If we look at the total potential lifetime "clock cycles" of a system, and compare that number to systems that we know are considered intelligent and capable of thinking, it might shed light on the plausibility. It takes information 100,000 years to cross the galaxy, and the universe is 14 billion years old (the galaxy is younger, but not enough to matter for this), so roughly a maximum number of 100,000 round trips could have possibly occurred in this "galactic brain". Let's use a very conservative metric, such as how long it takes a human to blink in response to stimuli (about 100 ms), for our "human clock cycle". If a human lives for 80 years, that is about 2.5 billion seconds, or 25 billion clock cycles. This is roughly 7 orders of magnitude (10s of millions) more than in the galactic brain. This gives the a very loose impression that a such system would probably have not have enough information flow or feedback loop cycles to "think" anything interesting. Of course, this a very rough heuristic, but I think it is interesting and useful. The idea of unexpected or strange "thinking systems" is very cool and we should continue to explore it, but there are certainly some hard constraints defining the space of such systems.
    - nikki93 7y ago
      Wouldn't this just mean that we happen to be in a young universe?
      - 7y ago
preston4tw 7y ago
Two things: mods can we get the title updated to reflect the publish year of 2014? The second is that the codeswarm link in the article is dead. This project on github is the closest thing I can find to any remnant of the project: https://github.com/rictic/code_swarm https://github.com/rictic/code_swarm
- ivan_ah 7y ago
  Check out gource, which does something similar https://gource.io/ https://gource.io/ (git history visualization)
qes 7y ago
Not even just the source code. Our system, even with the source code static, has constantly changing characteristics. We integrate with dozens of 3rd party APIs, making over 10M external HTTP calls per day. That leads to a lot of variability in runtime characteristics.
- longcommonname 7y ago
  This is true for us too. The behavior of ours is due to how airline fare rules are filed which creates all sorts of silly things.
qntmfred 7y ago
Always liked the garden metaphor https://blog.codinghorror.com/tending-your-software-garden/ https://blog.codinghorror.com/tending-your-software-garden/
gfodor 7y ago
framing failing fast or not as a conflict of interest between computer and codebase is an interesting one. I’ve often seen it as a conflict of interest between the user and the developer: users want the code to muddle through (“PHP style”), but devs want highly visible obvious failures early so they are easily identified and fixed. How do people balance this? The “state of the art” still seems to be things like “have asserts turned on in the debug build” but a holistic approach where the application has different runtime contracts with regards to failures based upon who is using the app seems somewhat underutilized — I’ve done it piecemeal with application feature flags but has anything like this been done at the platform/language/framework level?
js8 7y ago
A codebase is an orgasm. People working on it get excited when it is starting, and when it's done, they become very tired of it.
- andy_ppp 7y ago
  Unless they really feel love for their users? Ugh, too far :-/
code_biologist 7y ago
Catabolism is another aspect of thinking about code as an organic entity that I find useful. Anabolic (growth) processes and catabolic (teardown) processes always accompany each other in biological systems. Too often we neglect to continually tear down and remove unused aspects of our systems. Both anabolism and catabolism are always operating, though at different rates depending on the environment and needs of the organism. The funny thing is that catabolic processes will happily tear down important parts of the system, forcing them to be anabolicly rebuilt. Things like bone and muscle mass. This seems wasted, but it has an extremely important purpose: uncontrolled anabolism is cancer. I think the analogy to code is obvious. We've all seen codebases that have gotten out of hand (gotten cancer), and part of the reason they got that way is because the growth pressures had no countervailing teardown pressure. The market provides selective pressure in nasty ways: codebases and systems with cancer slow and die, and are replaced by younger codebases without uncontrolled growth. How do we break that cycle? By encouraging catabolic activity culturally within our companies and codebases. Cleanup is incredibly valuable. Functionality that isn't being heavily used and is complicating implementation of new work should absolutely be on the chopping block. We don't have to be quite as aggressive as biological systems, but we shouldn't take their example lightly either. This also provides insight into when cleanup doesn't matter: when the survival horizon of the organism is sufficiently short that cancer frankly doesn't matter. Startups shouldn't worry too much until they have product market fit. Once it's clear they'll be around for years more though, catabolic work will help ensure future health.
- rojobuffalo 7y ago
  username checks out. that's a really thought provoking analogy. there's definitely something to be gained in the process of tearing down and building back up.
carapace 7y ago
A codebase is a mechanical approximation of a set of human intentions. It's a model of an organic phenomenon.
- lonelappde 7y ago
  A code bases that manages a data store is not a model, it is a phenomenon.
euske 7y ago
> In this way, building software isn't at all like assembling a car. In terms of managing growth, it's more like raising a child or tending a garden. I knew it! I always thought maintaining my project was like doing a bonsai tree! (Disclaimer: I have zero experiences of bonsai.)
- lonelappde 7y ago
  Bonsai stay small as they age. Sadly, software programs do not.
  - inimino 7y ago
    because the gardener trims them both above and below ground.
lonelappde 7y ago
A production system is like the TARDIS. You don't program the system, you negotiate with it.
kazlock 7y ago
> two modules will tend to grow ever more dependent on each other unless separated by hard ('physical') boundaries This is especially true in monorepos. Code resuse has it's benefits, but left unchecked the dependency graph can turn into a huge highly-connected glob. Modules end up accidentally importing code they have no business importing because of sneaky transitive dependencies. The blast radius of even small simple changes becomes enormous, and testing/debugging becomes more and more complex. A great way to keep this in check is to have apply code isolation in the test environment. When you checkout the entire repo for a build or test, its easy for these kinds of dependencies to grow unnoticed. But if you require build/test targets to explicitly declare what code they depend on (and only make that code present when running them), changes to the dependency structure must be explicitly acknowledged in code review. This is one of the core principles behind build tools like Bazel.
- sitkack 7y ago
  Even in monorepos, calls into a module should go through a versioned interface.
  - j88439h84 7y ago
    I'm not familiar with that, how does it look? Docs maybe
  - Groxx 7y ago
    In general I agree, because it allows gradual migration of users... but in essentially every case I've seen, removing versioned interfaces is billed as the primary feature of monorepos.
    - sitkack 7y ago
      I just realized monorepos are analogous to a flat network, everything can reach everything else, or that main memory is flat/linear/same cost for each read/write (not true, but it is the abstraction we believe). I now believe that monorepos can only work well when there is a mechanical tool for ensuring correctness and that refactorings can be done atomically across the whole tree. Infact, it might be necessary to _only_ commit the refactoring operation to the tree and the source itself. That the whole tree is a blockchain of tree edit operations.

115 points by Kortaggio 7y ago

I've been waiting for a long time for people to start using the title "robopsychologist" in earnest. Especially for people who are called out to debug an unfamiliar system, in 'the wild' (ie. in production). You have to calm the system down, find out what it thinks is going on, gently correct any misconceptions, and guide it towards a better understanding of the world that lets it behave the way you need it to.

andy_ppp 7y ago
It’s commentary like this that makes me come back here and also I wish hacker news had a few more features, like being able to follow people who make interesting comments.
- grawprog 7y ago
  You could always start a bookmarks folder for 'HNers I find Interesting' and save their profile to that. I bet there's a way to write a script that goes through them, checks for posts since the last update and display them.
lioeters 7y ago
Good one, yes I can imagine the field branching out and developing disciplines like Software Psychiatry, System Nursing, Geriatric Care for Legacy Codebases.. Coming soon: Diagnostic and Statistical Manual of Software Disorders.

gandalfgeek 7y ago

Not just code bases, but it is useful to think of large planet scale systems in organic terms. https://blog.vivekhaldar.com/post/6972614229/large-computer-systems-are-organic https://blog.vivekhaldar.com/post/6972614229/large-computer-...

goldenkey 7y ago
Might as well just ask the question: What is Life? And then realize that scale of space or time is irrelevant. A galaxy could be thinking like a brain, except thoughts travel light years, are mainly dictated by GR (Light and Gravity), and take eons to actually occur. I think it's clear that the only meaningful nature of scale in time and space for life is just the one we artificially put on it, because of our conscious experience. LOTR and other fantasy explore this with the idea of trees having much wisdom having lived so long.
- dwaltrip 7y ago
  I once saw a comment similar to yours on HN, and there was a very good reply that critiqued the idea. I'll do my best to replicate something along the same lines. If we look at the total potential lifetime "clock cycles" of a system, and compare that number to systems that we know are considered intelligent and capable of thinking, it might shed light on the plausibility. It takes information 100,000 years to cross the galaxy, and the universe is 14 billion years old (the galaxy is younger, but not enough to matter for this), so roughly a maximum number of 100,000 round trips could have possibly occurred in this "galactic brain". Let's use a very conservative metric, such as how long it takes a human to blink in response to stimuli (about 100 ms), for our "human clock cycle". If a human lives for 80 years, that is about 2.5 billion seconds, or 25 billion clock cycles. This is roughly 7 orders of magnitude (10s of millions) more than in the galactic brain. This gives the a very loose impression that a such system would probably have not have enough information flow or feedback loop cycles to "think" anything interesting. Of course, this a very rough heuristic, but I think it is interesting and useful. The idea of unexpected or strange "thinking systems" is very cool and we should continue to explore it, but there are certainly some hard constraints defining the space of such systems.
  - nikki93 7y ago
    Wouldn't this just mean that we happen to be in a young universe?
    - 7y ago

preston4tw 7y ago

Two things: mods can we get the title updated to reflect the publish year of 2014? The second is that the codeswarm link in the article is dead. This project on github is the closest thing I can find to any remnant of the project: https://github.com/rictic/code_swarm https://github.com/rictic/code_swarm

ivan_ah 7y ago
Check out gource, which does something similar https://gource.io/ https://gource.io/ (git history visualization)

qes 7y ago

Not even just the source code. Our system, even with the source code static, has constantly changing characteristics. We integrate with dozens of 3rd party APIs, making over 10M external HTTP calls per day. That leads to a lot of variability in runtime characteristics.

longcommonname 7y ago
This is true for us too. The behavior of ours is due to how airline fare rules are filed which creates all sorts of silly things.

qntmfred 7y ago

Always liked the garden metaphor https://blog.codinghorror.com/tending-your-software-garden/ https://blog.codinghorror.com/tending-your-software-garden/

gfodor 7y ago

framing failing fast or not as a conflict of interest between computer and codebase is an interesting one. I’ve often seen it as a conflict of interest between the user and the developer: users want the code to muddle through (“PHP style”), but devs want highly visible obvious failures early so they are easily identified and fixed. How do people balance this? The “state of the art” still seems to be things like “have asserts turned on in the debug build” but a holistic approach where the application has different runtime contracts with regards to failures based upon who is using the app seems somewhat underutilized — I’ve done it piecemeal with application feature flags but has anything like this been done at the platform/language/framework level?

js8 7y ago

A codebase is an orgasm. People working on it get excited when it is starting, and when it's done, they become very tired of it.

andy_ppp 7y ago
Unless they really feel love for their users? Ugh, too far :-/

code_biologist 7y ago

Catabolism is another aspect of thinking about code as an organic entity that I find useful. Anabolic (growth) processes and catabolic (teardown) processes always accompany each other in biological systems. Too often we neglect to continually tear down and remove unused aspects of our systems. Both anabolism and catabolism are always operating, though at different rates depending on the environment and needs of the organism. The funny thing is that catabolic processes will happily tear down important parts of the system, forcing them to be anabolicly rebuilt. Things like bone and muscle mass. This seems wasted, but it has an extremely important purpose: uncontrolled anabolism is cancer. I think the analogy to code is obvious. We've all seen codebases that have gotten out of hand (gotten cancer), and part of the reason they got that way is because the growth pressures had no countervailing teardown pressure. The market provides selective pressure in nasty ways: codebases and systems with cancer slow and die, and are replaced by younger codebases without uncontrolled growth. How do we break that cycle? By encouraging catabolic activity culturally within our companies and codebases. Cleanup is incredibly valuable. Functionality that isn't being heavily used and is complicating implementation of new work should absolutely be on the chopping block. We don't have to be quite as aggressive as biological systems, but we shouldn't take their example lightly either. This also provides insight into when cleanup doesn't matter: when the survival horizon of the organism is sufficiently short that cancer frankly doesn't matter. Startups shouldn't worry too much until they have product market fit. Once it's clear they'll be around for years more though, catabolic work will help ensure future health.

rojobuffalo 7y ago
username checks out. that's a really thought provoking analogy. there's definitely something to be gained in the process of tearing down and building back up.

carapace 7y ago

A codebase is a mechanical approximation of a set of human intentions. It's a model of an organic phenomenon.

lonelappde 7y ago
A code bases that manages a data store is not a model, it is a phenomenon.

euske 7y ago

> In this way, building software isn't at all like assembling a car. In terms of managing growth, it's more like raising a child or tending a garden. I knew it! I always thought maintaining my project was like doing a bonsai tree! (Disclaimer: I have zero experiences of bonsai.)

lonelappde 7y ago
Bonsai stay small as they age. Sadly, software programs do not.
- inimino 7y ago
  because the gardener trims them both above and below ground.

lonelappde 7y ago

A production system is like the TARDIS. You don't program the system, you negotiate with it.

kazlock 7y ago

> two modules will tend to grow ever more dependent on each other unless separated by hard ('physical') boundaries This is especially true in monorepos. Code resuse has it's benefits, but left unchecked the dependency graph can turn into a huge highly-connected glob. Modules end up accidentally importing code they have no business importing because of sneaky transitive dependencies. The blast radius of even small simple changes becomes enormous, and testing/debugging becomes more and more complex. A great way to keep this in check is to have apply code isolation in the test environment. When you checkout the entire repo for a build or test, its easy for these kinds of dependencies to grow unnoticed. But if you require build/test targets to explicitly declare what code they depend on (and only make that code present when running them), changes to the dependency structure must be explicitly acknowledged in code review. This is one of the core principles behind build tools like Bazel.

sitkack 7y ago
Even in monorepos, calls into a module should go through a versioned interface.
- j88439h84 7y ago
  I'm not familiar with that, how does it look? Docs maybe
- Groxx 7y ago
  In general I agree, because it allows gradual migration of users... but in essentially every case I've seen, removing versioned interfaces is billed as the primary feature of monorepos.
  - sitkack 7y ago
    I just realized monorepos are analogous to a flat network, everything can reach everything else, or that main memory is flat/linear/same cost for each read/write (not true, but it is the abstraction we believe). I now believe that monorepos can only work well when there is a mechanical tool for ensuring correctness and that refactorings can be done atomically across the whole tree. Infact, it might be necessary to _only_ commit the refactoring operation to the tree and the source itself. That the whole tree is a blockchain of tree edit operations.