5 ms·

And that's the thing, the code _is_ the blueprint. What is code if not a sufficiently detailed specification?

tonyarkles 6y ago
That has, until recently, been my philosophy for a long time. See my sibling comment for context. Another example of where modelling both saved time and helped make a system more reliable: about 6 months ago, I took on a project that involved integrating a Bluetooth-Iridium gateway into an existing app, so that the app could work in areas with no wifi or cell connectivity. There’s a lot of things going on here: the Iridium modem has a limited queue size, transmissions are not instant, connectivity to the constellation fades in and out, Bluetooth randomly disconnects, etc. My gut reaction is to slowly build it out. Start with the low-level BLE stuff and build layers on top. Which I did do to shake out the functionality. But before I went and built the whole thing, I stopped and tried to build a TLA+ model of the system. That modelling was only somewhat successful; based on the constraints applied by each part, I discovered that it would be impossible to make the entire system reliable. What the model did, though, was highlight specific failure cases that would need to be detected and reported to the user. I agree that, in the end, the source code ends up being a detailed and precise specification of what the system does. What I’m realizing more and more though is that putting together more abstract models of the system ahead of time can save a ton of time by forcing you to think through cases that would (in my experience) end up with dirty hacks when they’re discovered after 90% of the code has been written. It’s not the approach required for every project. I still love firing up a Python or Lisp REPL and hacking away. It’s way more fun! But for certain types of software, it’s very much not the appropriate way to work.
- hwayne 6y ago
  > That modelling was only somewhat successful; based on the constraints applied by each part, I discovered that it would be impossible to make the entire system reliable. "I discovered I can't satisfy all my requirements at once and need to plan my tradeoffs in advance" sounds like a pretty successful modeling result to me!
  - tonyarkles 6y ago
    You're right :). I sometimes look at that modelling exercise with disappointment; not because I didn't get any value out of it (I got a ton), but because I'm not yet good enough at TLA+ to have been able to successfully write a model that handled all the edge cases.
- RealityVoid 6y ago
  I like this reply, but my instinct is to ask, would you not have found out the same thing by doing the implementation? This is the first time I hear about TLA+ and I don't know how difficult it is to work with. Was it easier than doing the thing? It does sound interesting though. But I assume that there is quite a curve to learning something like it. And, my nitpicky self also says "but that's modeling! Surely, architecture and modelling is not the same thing!". But again, my lack of insight might lead me astray. I do agree, a little bit of thought beforehand is very useful. Usually, I do this by doing a sketch on paper beforehand, if the mental map is not clear enough, maybe do some simple flow chart that describes the core functionality, just to make sure I understand it well enough.
  - hwayne 6y ago
    I wrote this case study on using TLA+ for a business project: https://medium.com/espark-engineering-blog/formal-methods-in-practice-8f20d72bce4f https://medium.com/espark-engineering-blog/formal-methods-in.... This was a system we'd already spent a lot of time working on and consistently saw a direct correlation between making changes and new production fires. TLA+ caught issues with the planned changes before we actually implemented them.
    - tonyarkles 6y ago
      And I would like to thank you for the material you've written. You, personally, played a huge role in my choice to look into and eventually standardize on TLA+ as my "distributed systems modelling" tool as well as getting me going on it. You rock, thank you!
  - tonyarkles 6y ago
    > "would you not have found out the same thing by doing the implementation?" Yes, although there's a couple of important points to that: - A lot of it wouldn't have been discovered until late into the implementation. The project itself looked straightforward, and my estimates would have been brutally off if I hadn't done the modelling exercise ahead of time. - Some of it would likely not have been discovered until after the app shipped. Rare edge cases. To give you context, it's an Android app that talks to an Iridium (satellite) modem over BLE. I basically modelled an abstract Android Activity lifecycle with extra events for user actions, a BLE stack (both sides of the connection), the Iridium connection, and the server on the other side of it. Each of these state machines were relatively straightforward to model. The insight came from letting TLA+ find paths through the five parallel state machines that got me into a weird state. (For an easy example, we make a request that successfully makes it out the Iridium modem to the server, the server replies, the response is delivered to the modem, and there's a corrupt BLE packet that terminates the connection.) - You're right, architecture and modelling are different things. My flow is basically: 1) high-level requirements, 2) model of the system based on the HLRs, 3) low-level requirements derived from the HLRs and the insight gained from modelling, 4) architecture derived from 1,2,3. - The amount of thought I'm going to put into a project ahead of time depends drastically on the project. The Iridium/BLE project? On the safety spectrum of 1 (mobile game) to 10 (flight critical software for a commercial jet), that project was about a 6-7. The software didn't have the ability to kill someone, but if it failed in the wrong way, they might believe someone was coming to rescue them when no one was coming.
hwayne 6y ago
Which blueprint? Engineers I talked to often mentioned that they'd make several different blueprints and different layers of detail. Is code the "high level" or the "low level" blueprints?

That has, until recently, been my philosophy for a long time. See my sibling comment for context. Another example of where modelling both saved time and helped make a system more reliable: about 6 months ago, I took on a project that involved integrating a Bluetooth-Iridium gateway into an existing app, so that the app could work in areas with no wifi or cell connectivity. There’s a lot of things going on here: the Iridium modem has a limited queue size, transmissions are not instant, connectivity to the constellation fades in and out, Bluetooth randomly disconnects, etc. My gut reaction is to slowly build it out. Start with the low-level BLE stuff and build layers on top. Which I did do to shake out the functionality. But before I went and built the whole thing, I stopped and tried to build a TLA+ model of the system. That modelling was only somewhat successful; based on the constraints applied by each part, I discovered that it would be impossible to make the entire system reliable. What the model did, though, was highlight specific failure cases that would need to be detected and reported to the user. I agree that, in the end, the source code ends up being a detailed and precise specification of what the system does. What I’m realizing more and more though is that putting together more abstract models of the system ahead of time can save a ton of time by forcing you to think through cases that would (in my experience) end up with dirty hacks when they’re discovered after 90% of the code has been written. It’s not the approach required for every project. I still love firing up a Python or Lisp REPL and hacking away. It’s way more fun! But for certain types of software, it’s very much not the appropriate way to work.

hwayne 6y ago
> That modelling was only somewhat successful; based on the constraints applied by each part, I discovered that it would be impossible to make the entire system reliable. "I discovered I can't satisfy all my requirements at once and need to plan my tradeoffs in advance" sounds like a pretty successful modeling result to me!
- tonyarkles 6y ago
  You're right :). I sometimes look at that modelling exercise with disappointment; not because I didn't get any value out of it (I got a ton), but because I'm not yet good enough at TLA+ to have been able to successfully write a model that handled all the edge cases.
RealityVoid 6y ago
I like this reply, but my instinct is to ask, would you not have found out the same thing by doing the implementation? This is the first time I hear about TLA+ and I don't know how difficult it is to work with. Was it easier than doing the thing? It does sound interesting though. But I assume that there is quite a curve to learning something like it. And, my nitpicky self also says "but that's modeling! Surely, architecture and modelling is not the same thing!". But again, my lack of insight might lead me astray. I do agree, a little bit of thought beforehand is very useful. Usually, I do this by doing a sketch on paper beforehand, if the mental map is not clear enough, maybe do some simple flow chart that describes the core functionality, just to make sure I understand it well enough.
- hwayne 6y ago
  I wrote this case study on using TLA+ for a business project: https://medium.com/espark-engineering-blog/formal-methods-in-practice-8f20d72bce4f https://medium.com/espark-engineering-blog/formal-methods-in.... This was a system we'd already spent a lot of time working on and consistently saw a direct correlation between making changes and new production fires. TLA+ caught issues with the planned changes before we actually implemented them.
  - tonyarkles 6y ago
    And I would like to thank you for the material you've written. You, personally, played a huge role in my choice to look into and eventually standardize on TLA+ as my "distributed systems modelling" tool as well as getting me going on it. You rock, thank you!
- tonyarkles 6y ago
  > "would you not have found out the same thing by doing the implementation?" Yes, although there's a couple of important points to that: - A lot of it wouldn't have been discovered until late into the implementation. The project itself looked straightforward, and my estimates would have been brutally off if I hadn't done the modelling exercise ahead of time. - Some of it would likely not have been discovered until after the app shipped. Rare edge cases. To give you context, it's an Android app that talks to an Iridium (satellite) modem over BLE. I basically modelled an abstract Android Activity lifecycle with extra events for user actions, a BLE stack (both sides of the connection), the Iridium connection, and the server on the other side of it. Each of these state machines were relatively straightforward to model. The insight came from letting TLA+ find paths through the five parallel state machines that got me into a weird state. (For an easy example, we make a request that successfully makes it out the Iridium modem to the server, the server replies, the response is delivered to the modem, and there's a corrupt BLE packet that terminates the connection.) - You're right, architecture and modelling are different things. My flow is basically: 1) high-level requirements, 2) model of the system based on the HLRs, 3) low-level requirements derived from the HLRs and the insight gained from modelling, 4) architecture derived from 1,2,3. - The amount of thought I'm going to put into a project ahead of time depends drastically on the project. The Iridium/BLE project? On the safety spectrum of 1 (mobile game) to 10 (flight critical software for a commercial jet), that project was about a 6-7. The software didn't have the ability to kill someone, but if it failed in the wrong way, they might believe someone was coming to rescue them when no one was coming.

hwayne 6y ago

Which blueprint? Engineers I talked to often mentioned that they'd make several different blueprints and different layers of detail. Is code the "high level" or the "low level" blueprints?