4 ms·

- "comprehensive suite": you can't mock everything. If you have an algorithmic trading platform, you connect to many 3rd parties and their systems are very comp

- "comprehensive suite": you can't mock everything. If you have an algorithmic trading platform, you connect to many 3rd parties and their systems are very complex, poorly documented and they don't work according to the documentation. - "automated tests": traders don't know what they want. They think they know, but they don't. And they will tell you things they don't want, and when you deliver them they'll be angry. - you don't have direct access to production. If something fails, you won't get all the logs because regulatory reasons. Sometimes you won't even get anything. - etc. I see the value of continuous deployment, just wanted to mention that it's not applicable in many-many fields outside of the CRUD bubble.

crdoconnor 12y ago
This is profoundly ironic. Every excuse that you have made for not automating testing I have heard before ("mocking is hard. let's go shopping"). Most commonly, however, these excuses come from the pits of unprofessional software engineering, though - the "CRUD bubble" as you put it. I find it pretty funny that you think "customers who don't know what they want" is a problem that is unique to trading. Or poorly/wrongly documented APIs.
- sz4kerto 12y ago
  No, I have never said that I think these problem is unique to trading (that would have been a funny thing to say, indeed). "Every excuse that you have made for not testing I have heard before" -- does this fact render the excuses invalid? Yes, mocking is sometimes too hard and you should not do it because the effort never pays back it's cost. And these situations are very common.
  - crdoconnor 12y ago
    >does this fact render the excuses invalid The "mocking is hard" excuse is invalid, yes. Sometimes it can make sense to create a dirt simple adapter whose code you very rarely have to change, which you subject to heavy manual testing. Then you mock that and leave it alone. You can use this when converting a weird protocol into an easier to handle & easier to mock protocol. However, that just moves the point at which you have to mock back a little, and saves you from writing lots of harness code. It doesn't prevent you from doing continuous integration or continuous deployment. It also doesn't mean you shouldn't mock. It also only makes sense if mocking is harder than actually writing, manually testing and deploying this service. >Yes, mocking is sometimes too hard and you should not do it because the effort never pays back it's cost. Testing is hard. Let's go shopping.
- collyw 12y ago
  He said automated testing. You can still test (and are able to test a lot more things) when you do it manually. You know, like actually use the application as it is intended.
- xorcist 12y ago
  I don't read them as excuses for not testing. I read them as reasons why automated testing is not enough. Big difference, and worth taking seriously.
  - crdoconnor 12y ago
    They are not excuses for not testing, no. They are excuses for not automating a highly repetitive process.
    - sz4kerto 12y ago
      Nothing is farther from me than not automating a highly repetitive process. :) Please don't respond to strawmans, that's not useful for anyone. I'm saying that there are things that you can't mock efficiently because the thing you want to mock has a very complex behavior -- this means you can't think of all the possible combinations of how it can deviate from the spec. Let's not forget where the thread started from -- continuous, automated deployment to production. I agree that one should do a lot of automated tests, mock what makes sense to be mocked, etc. However, a fully automated testing means no manual testing, right? That's what is almost impossible in many environments -- and this makes continuous deployment extremely risky. Just to reiterate: I am not against mocking, I am not against automating repetitive processes. I just think that 100% automated testing is not feasible in many cases, and that makes 100% automated continuous deployment infeasible as well.
      - crdoconnor 12y ago
        >I'm saying that there are things that you can't mock efficiently because the thing you want to mock has a very complex behavior -- this means you can't think of all the possible combinations of how it can deviate from the spec. Mocking complex and unreliable behavior by 3rd party systems is part of writing good tests and, ultimately, delivering high quality software. If you can't think of all possible combinations of unpredictable behavior, that's fine. You keep testing until you find as many as you think there are and once you do, you mock it. Once you get to the point where you think you have found all of them, you can release. Continuously. If you never release that point, either you grit your teeth and release anyway, or you never release. If you can do this, you will probably score one over on the competition, because getting to this point is rare and valuable. >However, a fully automated testing means no manual testing, right? Absolutely not. It just means that you've minimized your reliance upon it, and the manual testing you do do does not have to be done at the end of each iteration - it can be done continuously. >That's what is almost impossible in many environments -- and this makes continuous deployment extremely risky. If you are minimally reliant upon manual testing then continuous deployment is not risky. If you are heavily reliant upon manual testing then continuous deployment is risky >Just to reiterate: I am not against mocking, I am not against automating repetitive processes. I just think that 100% automated testing is not feasible in many cases There's a difference between something being hard and something not being feasible. Getting to 100% automated testing is hard, but it is feasible.
ribs 12y ago
3rd party systems - since they're complex and poorly documented and don't work like they say they do, the obvious thing to do is to simulate their vagaries and test against that. Trader don't know what they want - you could substitute any kind of stakeholder for "traders". It is the point of stole, to handle evolving requirements. IT'S THE ENTIRE POINT OF AGILE. No access to production - I'm not sure what argument you're making here. And then your assertion about continuous deployment is unsupported, so I won't bother,

crdoconnor 12y ago

This is profoundly ironic. Every excuse that you have made for not automating testing I have heard before ("mocking is hard. let's go shopping"). Most commonly, however, these excuses come from the pits of unprofessional software engineering, though - the "CRUD bubble" as you put it. I find it pretty funny that you think "customers who don't know what they want" is a problem that is unique to trading. Or poorly/wrongly documented APIs.

sz4kerto 12y ago
No, I have never said that I think these problem is unique to trading (that would have been a funny thing to say, indeed). "Every excuse that you have made for not testing I have heard before" -- does this fact render the excuses invalid? Yes, mocking is sometimes too hard and you should not do it because the effort never pays back it's cost. And these situations are very common.
- crdoconnor 12y ago
  >does this fact render the excuses invalid The "mocking is hard" excuse is invalid, yes. Sometimes it can make sense to create a dirt simple adapter whose code you very rarely have to change, which you subject to heavy manual testing. Then you mock that and leave it alone. You can use this when converting a weird protocol into an easier to handle & easier to mock protocol. However, that just moves the point at which you have to mock back a little, and saves you from writing lots of harness code. It doesn't prevent you from doing continuous integration or continuous deployment. It also doesn't mean you shouldn't mock. It also only makes sense if mocking is harder than actually writing, manually testing and deploying this service. >Yes, mocking is sometimes too hard and you should not do it because the effort never pays back it's cost. Testing is hard. Let's go shopping.
collyw 12y ago
He said automated testing. You can still test (and are able to test a lot more things) when you do it manually. You know, like actually use the application as it is intended.
xorcist 12y ago
I don't read them as excuses for not testing. I read them as reasons why automated testing is not enough. Big difference, and worth taking seriously.
- crdoconnor 12y ago
  They are not excuses for not testing, no. They are excuses for not automating a highly repetitive process.
  - sz4kerto 12y ago
    Nothing is farther from me than not automating a highly repetitive process. :) Please don't respond to strawmans, that's not useful for anyone. I'm saying that there are things that you can't mock efficiently because the thing you want to mock has a very complex behavior -- this means you can't think of all the possible combinations of how it can deviate from the spec. Let's not forget where the thread started from -- continuous, automated deployment to production. I agree that one should do a lot of automated tests, mock what makes sense to be mocked, etc. However, a fully automated testing means no manual testing, right? That's what is almost impossible in many environments -- and this makes continuous deployment extremely risky. Just to reiterate: I am not against mocking, I am not against automating repetitive processes. I just think that 100% automated testing is not feasible in many cases, and that makes 100% automated continuous deployment infeasible as well.
    - crdoconnor 12y ago
      >I'm saying that there are things that you can't mock efficiently because the thing you want to mock has a very complex behavior -- this means you can't think of all the possible combinations of how it can deviate from the spec. Mocking complex and unreliable behavior by 3rd party systems is part of writing good tests and, ultimately, delivering high quality software. If you can't think of all possible combinations of unpredictable behavior, that's fine. You keep testing until you find as many as you think there are and once you do, you mock it. Once you get to the point where you think you have found all of them, you can release. Continuously. If you never release that point, either you grit your teeth and release anyway, or you never release. If you can do this, you will probably score one over on the competition, because getting to this point is rare and valuable. >However, a fully automated testing means no manual testing, right? Absolutely not. It just means that you've minimized your reliance upon it, and the manual testing you do do does not have to be done at the end of each iteration - it can be done continuously. >That's what is almost impossible in many environments -- and this makes continuous deployment extremely risky. If you are minimally reliant upon manual testing then continuous deployment is not risky. If you are heavily reliant upon manual testing then continuous deployment is risky >Just to reiterate: I am not against mocking, I am not against automating repetitive processes. I just think that 100% automated testing is not feasible in many cases There's a difference between something being hard and something not being feasible. Getting to 100% automated testing is hard, but it is feasible.

ribs 12y ago

3rd party systems - since they're complex and poorly documented and don't work like they say they do, the obvious thing to do is to simulate their vagaries and test against that. Trader don't know what they want - you could substitute any kind of stakeholder for "traders". It is the point of stole, to handle evolving requirements. IT'S THE ENTIRE POINT OF AGILE. No access to production - I'm not sure what argument you're making here. And then your assertion about continuous deployment is unsupported, so I won't bother,