3 ms·

Is it possible to use, say, S3, and meet these criteria? (Then, something more complex or special purpose, like say Amazon Elastic Transcoder or something).

theamk 6y ago
Sure, if you use a separate bucket for your teat instances, and reset it between tests. This can probably work with more complex services too.
- jrochkind1 6y ago
  to avoid conflict between two entities running tests at once (say via CI), I guess it would need to use Amazon API to construct the bucket and tear it down around each test? But I'm not sure that would comply with the requirements, they seem to forbid running against external services at all, no? '"calling out to a few hard-to-run external services" doesn't count.'
  - theamk 6y ago
    Yep, to use AWS S3, you'll need a lock to ensure a single job at a time, or create buckets programmatically for each test. I think every team needs to decide how much to follow the original advice, there is certainly a balance between simulating external service locally (with possible associated behavior changes) or using an external service. If you do want to run everything locally, there are servers which offer S3 API, like "minio" and "fake_s3". Probably would be very nice for localhost operation. But if you need a complex thing like Transcoder this is not going to work.
csharptwdec19 6y ago
I had to actually do something like this. Due to the hilarious nature of the business, getting an S3 account to use for 'local' (i.e. dev machine) development was a huge chore. We wound up creating an Interface to abstract our usage of S3, and wrote a 'local' provider that just used the local filesystem. Of course, many use cases that would not work.... But the big -advantage- of it was, we at least knew that if we ever wanted to switch away from S3, it wouldn't be hard. The Interface had a spec that could be used to ensure that any plugin would work with our application's usage. It wasn't -that- much more work.
- jrochkind1 6y ago
  > We wound up creating an Interface to abstract our usage of S3 Which violates these requirements, right? Using something that mocks S3 instead of actually being S3? (If I was going to do that, I'd just use open source Minio S3 clone, assuming it adequately cloned the S3 features I needed).
  - forgotmypw17 6y ago
    I think there is always an ideal case, but if you can't get there 100%, doesn't mean you shouldn't try, ending up with 95%.
rcfox 6y ago
I know it says "no mocks", but you could use Minio to have a local S3 alternative. Their client can even be used as a frontend to both S3 and a Minio instance.
- jrochkind1 6y ago
  That's available for S3, although I think it would still violate OP's requirements, but in spirit it'd be good. Not usually available for most other AWS services you might be using though. Say, Elastic Transcoder for transcoding A/V media.
dumbfounder 6y ago
Well, if you use S3 then it has to be, because what this person says HAS to be true. So, yes, you can use S3, so we need to carve out external services to say we don't need to run those services personally. So, what if you have a separate business unit that you don't share code with that created the service? Well, I guess we will make an exception for that too, but they REALLY need to make sure their stuff works. Ok, what about the group on the 4th floor? I don't really want to run their code because wow that's a hassle and it is on a completely different stack and test system. Well then just treat it as external. Same with J's service, they sit next to me. And it's really the same stack but it's still hard to run it as a complete system to test. Well that's ok too then I guess. Ok we are back to microservices and why they are useful. All that said, I agree with the sentiment of the post, testing the system as a whole can reveal issues before they get to prod. But maybe that means you need to harden your contracts and internal SLAs.

theamk 6y ago

Sure, if you use a separate bucket for your teat instances, and reset it between tests. This can probably work with more complex services too.

jrochkind1 6y ago
to avoid conflict between two entities running tests at once (say via CI), I guess it would need to use Amazon API to construct the bucket and tear it down around each test? But I'm not sure that would comply with the requirements, they seem to forbid running against external services at all, no? '"calling out to a few hard-to-run external services" doesn't count.'
- theamk 6y ago
  Yep, to use AWS S3, you'll need a lock to ensure a single job at a time, or create buckets programmatically for each test. I think every team needs to decide how much to follow the original advice, there is certainly a balance between simulating external service locally (with possible associated behavior changes) or using an external service. If you do want to run everything locally, there are servers which offer S3 API, like "minio" and "fake_s3". Probably would be very nice for localhost operation. But if you need a complex thing like Transcoder this is not going to work.

csharptwdec19 6y ago

I had to actually do something like this. Due to the hilarious nature of the business, getting an S3 account to use for 'local' (i.e. dev machine) development was a huge chore. We wound up creating an Interface to abstract our usage of S3, and wrote a 'local' provider that just used the local filesystem. Of course, many use cases that would not work.... But the big -advantage- of it was, we at least knew that if we ever wanted to switch away from S3, it wouldn't be hard. The Interface had a spec that could be used to ensure that any plugin would work with our application's usage. It wasn't -that- much more work.

jrochkind1 6y ago
> We wound up creating an Interface to abstract our usage of S3 Which violates these requirements, right? Using something that mocks S3 instead of actually being S3? (If I was going to do that, I'd just use open source Minio S3 clone, assuming it adequately cloned the S3 features I needed).
- forgotmypw17 6y ago
  I think there is always an ideal case, but if you can't get there 100%, doesn't mean you shouldn't try, ending up with 95%.

rcfox 6y ago

I know it says "no mocks", but you could use Minio to have a local S3 alternative. Their client can even be used as a frontend to both S3 and a Minio instance.

jrochkind1 6y ago
That's available for S3, although I think it would still violate OP's requirements, but in spirit it'd be good. Not usually available for most other AWS services you might be using though. Say, Elastic Transcoder for transcoding A/V media.

dumbfounder 6y ago

Well, if you use S3 then it has to be, because what this person says HAS to be true. So, yes, you can use S3, so we need to carve out external services to say we don't need to run those services personally. So, what if you have a separate business unit that you don't share code with that created the service? Well, I guess we will make an exception for that too, but they REALLY need to make sure their stuff works. Ok, what about the group on the 4th floor? I don't really want to run their code because wow that's a hassle and it is on a completely different stack and test system. Well then just treat it as external. Same with J's service, they sit next to me. And it's really the same stack but it's still hard to run it as a complete system to test. Well that's ok too then I guess. Ok we are back to microservices and why they are useful. All that said, I agree with the sentiment of the post, testing the system as a whole can reveal issues before they get to prod. But maybe that means you need to harden your contracts and internal SLAs.