2 ms·

First of all it is the test code, no matter whether it's implemented as a feature flag or in any other way. Test code and test data shall not be mixed with the

First of all it is the test code, no matter whether it's implemented as a feature flag or in any other way. Test code and test data shall not be mixed with the production one for many well-documented and well-known reasons: security, additional points of failure, additional memory requirements, impact on architecture etc. >But if you are testing an artifact, why isn’t the artifact testing part of your CI process? It is and it shall be part of the CI process. Commit gets assigned build number in tag, artifact gets the version and build number in it's name and metadata, deployment to CI environment is performed, tests are executed against specific artifact, so every time you deploy to production you have a proof, that the exact binary that is being deployed has been verified in its production configuration. >I’m assuming the use of a service discovery tool to determine what gets run. Service discovery is irrelevant to this problem. Substitution of mock can be done with or without it.

scarface74 8y ago
What exactly are you trying to accomplish? If you are testing a single microservice and don’t want to test the dependent microservice - if you are trying to do a unit test and not an integration test, you are going to run against mock services. If you are testing a monolith you are going to create separate test assemblies/modules that call your subject under test with mock dependencies. They are both going to be part of your CI process then and either way you aren’t going to publish the artifacts until the tests pass. Your deployment pipeline either way would be some type of deployment pipeline with some combination of manual and automated approvals with the same artifacts. The whole discussion about which is easier is moot. Edit: I just realized why this conversation is going sideways. Your initial assumptions were incorrect. you may want to test only A: with monolithic architecture you'll have to produce another build of the application, that contains mock of B (or you need something like OSGi for runtime module discovery). That’s not how modern testing is done. https://www.developerhandbook.com/unit-testing/writing-unit-tests-with-nunit-and-moq/ https://www.developerhandbook.com/unit-testing/writing-unit-...
- ivan_gammel 8y ago
  > What exactly are you trying to accomplish? Good test must verify the contract on the system boundaries: in case of the API, it's verification done by calling the API. We are discussing two options here: integrated application, hosting multiple APIs, and microservice architecture. Verification on the system boundaries means running the app, not running a unit test (unit tests are good, but serve different purpose). Feature flags make it only worse, because testing with them covers only non-production branches of your code. > Your initial assumptions were incorrect. With nearly 20 years of engineering and management experience, I know very well how modern testing is done. :)
  - scarface74 8y ago
    Verification on the system boundaries means running the app, not running a unit test What is an app at the system boundaries if not a piece of code with dependencies? If you have a microservice - FooService that calls BarService. The "system boundary" you are trying to test is FooService using a fake BarService. I'm assuming that you're calling FooService via HTTP using a test runner like Newman and test results. In a monolithic application you have class FooModule that depends on BarModule that implements IBarModule. In your production application you use create FooModule: var x = FooModule(new BarModule) y = x.Baz(5); In your Unit tests, you create your FooModuleL var x = FooModule(new FakeBarModule) actual= x.Baz(5) Assert.AreEqual(10,actual) And run your tests with a runner like NUnit. There is no functional difference. Of course FooModule can be at whatever level of the stack you are trying to test - even the Controller.

scarface74 8y ago

What exactly are you trying to accomplish? If you are testing a single microservice and don’t want to test the dependent microservice - if you are trying to do a unit test and not an integration test, you are going to run against mock services. If you are testing a monolith you are going to create separate test assemblies/modules that call your subject under test with mock dependencies. They are both going to be part of your CI process then and either way you aren’t going to publish the artifacts until the tests pass. Your deployment pipeline either way would be some type of deployment pipeline with some combination of manual and automated approvals with the same artifacts. The whole discussion about which is easier is moot. Edit: I just realized why this conversation is going sideways. Your initial assumptions were incorrect. you may want to test only A: with monolithic architecture you'll have to produce another build of the application, that contains mock of B (or you need something like OSGi for runtime module discovery). That’s not how modern testing is done. https://www.developerhandbook.com/unit-testing/writing-unit-tests-with-nunit-and-moq/ https://www.developerhandbook.com/unit-testing/writing-unit-...

ivan_gammel 8y ago
> What exactly are you trying to accomplish? Good test must verify the contract on the system boundaries: in case of the API, it's verification done by calling the API. We are discussing two options here: integrated application, hosting multiple APIs, and microservice architecture. Verification on the system boundaries means running the app, not running a unit test (unit tests are good, but serve different purpose). Feature flags make it only worse, because testing with them covers only non-production branches of your code. > Your initial assumptions were incorrect. With nearly 20 years of engineering and management experience, I know very well how modern testing is done. :)
- scarface74 8y ago
  Verification on the system boundaries means running the app, not running a unit test What is an app at the system boundaries if not a piece of code with dependencies? If you have a microservice - FooService that calls BarService. The "system boundary" you are trying to test is FooService using a fake BarService. I'm assuming that you're calling FooService via HTTP using a test runner like Newman and test results. In a monolithic application you have class FooModule that depends on BarModule that implements IBarModule. In your production application you use create FooModule: var x = FooModule(new BarModule) y = x.Baz(5); In your Unit tests, you create your FooModuleL var x = FooModule(new FakeBarModule) actual= x.Baz(5) Assert.AreEqual(10,actual) And run your tests with a runner like NUnit. There is no functional difference. Of course FooModule can be at whatever level of the stack you are trying to test - even the Controller.