3 ms·

I architect everything around queues and run at least two redundant, independent processors (written in different languages, even) for each queue. I try to hav

I architect everything around queues and run at least two redundant, independent processors (written in different languages, even) for each queue. I try to have as few services involved as possible, which basically means the web server.

t0astbread 6y ago
So you wrote your app twice?
- forgotmypw17 6y ago
  Yep, pretty much. Once you've written something once, it's easier to write it again. I try to use languages with similar syntax and only use a common subset of their syntax, so it's often almost entirely a copy-paste job. There are many advantages to doing it this way. One is that I thoroughly review each side of the codebase while writing the other. Another is that I get a complete coverage test suite for free out of the deal. Another is that if something goes sideways, it's easier to figure out where that is. It's also easier to discover any faults, because the outputs don't match up.
  - leesalminen 6y ago
    Honestly, this sounds so crazy to me that I must be missing something. I might try it out one day! Would it be a fair assumption that these were fairly simple apps in terms of business logic? There are like 2-3 subsystems of mine that the thought of having 2 of each makes me quiver in fear.
    - forgotmypw17 6y ago
      Yes, fairly simple.
    - forgotmypw17 6y ago
      In part I was inspired by what I learned about mainframes. I don't know that much about them, but I do know that every single component is hot-swappable and redundant, so that's how I tried to design my application. As a bonus, I have to design it simply, and the process of rewriting it several times helps that end.
plesiv 6y ago
Which languages do you use most often? Languages are not created equal, some are clearly superior to others for a given task. If you can develop effectively in a better programming language, then why wouldn't you invest more effort in making the implementation of your task in that language more foolproof? Write better tests, clean-up the interfaces, you can even try to use formal methods to ensure correctness etc.
- forgotmypw17 6y ago
  Perl is the language I use most often for processing, with bash and PHP for redundancy and glue. As others mentioned, I chose Perl and PHP because those are the languages I am most familiar with, and because they are "boring", meaning they've been stable enough that I could've written my scripts 20 years ago, and they would still work today. PHP to a lesser extent, but still true. Also, as I mentioned the "lowest common denominator" style of writing allows me to write Perl which I can copy and paste into PHP almost without changes, and vice-versa. To facilitate this, I ported several functions from one to the other, e.g. str_replace for Perl and and index() for PHP. I can't think of ways to make it more foolproof than writing two redundant systems, nor how much better tests I can have than full coverage of each process by a redundant one, except by introducing triple-redundancy, which is not out of the question. Is that what you had in mind, or something else?
  - plesiv 6y ago
    OK, I get it :)
    - forgotmypw17 6y ago
      Going back to your comment, I feel like I missed this part, or at least did not address it: >Write better tests, clean-up the interfaces, you can even try to use formal methods to ensure correctness etc. I definitely put effort into all these things, but my experience shows that no matter how much work goes into that, things will find a way to fail in a way I could not predict.

t0astbread 6y ago

So you wrote your app twice?

forgotmypw17 6y ago
Yep, pretty much. Once you've written something once, it's easier to write it again. I try to use languages with similar syntax and only use a common subset of their syntax, so it's often almost entirely a copy-paste job. There are many advantages to doing it this way. One is that I thoroughly review each side of the codebase while writing the other. Another is that I get a complete coverage test suite for free out of the deal. Another is that if something goes sideways, it's easier to figure out where that is. It's also easier to discover any faults, because the outputs don't match up.
- leesalminen 6y ago
  Honestly, this sounds so crazy to me that I must be missing something. I might try it out one day! Would it be a fair assumption that these were fairly simple apps in terms of business logic? There are like 2-3 subsystems of mine that the thought of having 2 of each makes me quiver in fear.
  - forgotmypw17 6y ago
    Yes, fairly simple.
  - forgotmypw17 6y ago
    In part I was inspired by what I learned about mainframes. I don't know that much about them, but I do know that every single component is hot-swappable and redundant, so that's how I tried to design my application. As a bonus, I have to design it simply, and the process of rewriting it several times helps that end.

plesiv 6y ago

Which languages do you use most often? Languages are not created equal, some are clearly superior to others for a given task. If you can develop effectively in a better programming language, then why wouldn't you invest more effort in making the implementation of your task in that language more foolproof? Write better tests, clean-up the interfaces, you can even try to use formal methods to ensure correctness etc.

forgotmypw17 6y ago
Perl is the language I use most often for processing, with bash and PHP for redundancy and glue. As others mentioned, I chose Perl and PHP because those are the languages I am most familiar with, and because they are "boring", meaning they've been stable enough that I could've written my scripts 20 years ago, and they would still work today. PHP to a lesser extent, but still true. Also, as I mentioned the "lowest common denominator" style of writing allows me to write Perl which I can copy and paste into PHP almost without changes, and vice-versa. To facilitate this, I ported several functions from one to the other, e.g. str_replace for Perl and and index() for PHP. I can't think of ways to make it more foolproof than writing two redundant systems, nor how much better tests I can have than full coverage of each process by a redundant one, except by introducing triple-redundancy, which is not out of the question. Is that what you had in mind, or something else?
- plesiv 6y ago
  OK, I get it :)
  - forgotmypw17 6y ago
    Going back to your comment, I feel like I missed this part, or at least did not address it: >Write better tests, clean-up the interfaces, you can even try to use formal methods to ensure correctness etc. I definitely put effort into all these things, but my experience shows that no matter how much work goes into that, things will find a way to fail in a way I could not predict.