Achieving Software Testing Nirvana in a CI/CD environment with a sprinkling of Agile – Part V

Part V – Release Stage

Blue-Green Deployments

Blue green deployments are used when we want to swap one environment by pointing the load balancer to another environment such as the staging environment. The staging environment could be the green environment, when doing a release, once all has been given the OK, the load balancer can be switched to point at the green/staging environment. The old production environment, which will have the old version, can be used for the next staging promotion. In case of any issues with the deployment to production, the green environment can be switched back to the blue. Blue/Green deployments can be very tricky when dealing with state, such as Database deployments and handling transactions that may have been added during the new application deployment. A best practice for this is database versioning with tools like DBDeploy or DBUp etc and de-coupling your application from your database deployments. So one version of your application can be backward compatible with the previous version of the database as you continually upgrade.

Canary Releases

Canary releases is the act whereby a release of a feature may only be released to a subset of your users, for example, users in London would see the new code whereas users in the rest of the UK would still see the old production version, you may then slowly increase the volume as to which percentage or region of the UK will see the new version as confidence for the new feature grows. Another way of doing Canary Releases is A/B Testing, this way you can have a certain percentage of the population see version A of the new feature and another percentage see version B, and use this type of testing to gauge which version had better click through rates for example.

Emergency Fixes

If there are any issues with a release to production, then these changes should be rolled back using whichever rollback strategy is in place, and the fix should be put through the pipeline like every other feature, that way it is fully tested in every deployment stage and will not result in an introduction of more bugs. Emergency fixes should never by implemented in production, even in the case of something small like a misconfiguration of an application setting for example.

You Gotta Stop

A little note on test data

Using the correct state to facilitate a test can be a little tricky. There are three types of test data that should be used within the context of any test;

  1. Test specific data
  2. Test reference data
  3. Application reference data

Test specific data is only used within the test and not be any other test and must be torn down immediately after the test execution, so as to isolate it from other tests. Test reference data is data that can be used by more than one test and is there to act as an aid for all tests within the context of the reference data, things like a configuration setting, or a database connection. Application reference data is much more commonly used in acceptance tests, where data is setup to be used by tests in many contexts from the application start-up. When looking to setup data to be used by one or more tests, it is much more optimal to use the applications public API to get data from a data store and to tear it down once complete, if not the use of Mother classes can be used to setup static data-sets for use by the tests. It is extremely important to remove data state after each isolated test run, as when interacting with a database, tests that validate row count, when one test may have added a new entry to the same table will fail.


Thanks for reading and keep building awesome.

Until next time!

Ian B

P.S. You can also find me on Twitter: @ianaldo21

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s