Tuesday, March 14, 2017

Unit and End to End (e2e) Testing should be enough - the rest are intangible

I have learned from my years as full time developer that (those tests, that according to Google's classification are called medium), are a life saver. I have also learned from my years as a team leader that these tests are the perfect excuse to deliver tight coupled, AKA 3v1l untestable code.

If Unit tests are covering all the functionality of your loosely couple software and e2e tests are covering all common scenarios followed by your users, I would name the medium tests "intangible tests". They are useful as ad-hoc tests that aim at getting a proof of concept (POC) out of the door. However maintaining them, relying on them for application quality and delaying your delivery because of them will be nothing more than paying a big opportunity cost.

Quick Glossary:
  1. Small Tests: "Local Memory" Tests. Hopefully your current Unit tests.
  2. Medium Tests: "Local Host" Tests. Hopefully you don't maintain them.
  3. Large Tests: "Any Host" Tests. Hopefully your current e2e tests.
The local host tests should not be run as part of your pipeline. They are intangible. I would keep them in the project just for historical reasons, if I would ever keep them.