Approximately TestDriven
As it might shine through in my previous posts – I have a pragmatic view on TDD. I understand why tests are important – I do tests, but I don’t test everything – maybe I approach 15% test-coverage. I chose to make ERP-systems my speciality – and I land at lousy 15% coverage?
Many would say that a SOA-based ERP-system has so much complexity and is so intricate that it should be near 100% covered in tests. I disagree (not surprisingly). Having built these monsters (albeit not distributed, but as a standard rich client) for three years has taught me a lot. We have gone through various stages of TestDriven:
We started out doing 100% TestDriven – we set the mark at 80% test coverage – and all interfaces should be built using TestDriven methodologies. We also decided that all classes should uphold SOLID principles – solid (haha) reasons should be present if one of the principles were violated. This period lasted about 6 months – and then came the first large round of re-factorings…
It showed us a few things – first, don’t test simple properties – they catch no errors, we had none regressions in that area – and they only caused us problems when re-factoring as the tests were brittle.
Then we proceeded doing fewer tests and instead focused on integration tests – and we tried to do acceptance testing from UI down to the database. We spent a lot of time here… and learned a few more things – first, UI tests are expensive and they are extremely brittle – a change in UI automation keys would break it, a simple change in docking could break them and most importantly: Complex WPF datatemplating is almost impossible to test in a meaningful way.
Then we introduced the Model-View-ViewModel pattern in all UI-centric functionality – and now the tests started paying off. All the ‘goup’-code collected in one place and testable. Hooray!
That left us with doing 3 kinds of tests:
- Integration tests – and only when there was actual complexity involved
- ViewModel tests – all bugs seemed to stem from here, and were easy to test away.
- IoC tests – we created a simple test: Can all presenters be resolved?
The last one might seem curious – but presenters are the ‘mother’-objects in these scenarios – they tie all the other classes together – and if they resolve – all other elements have been registered correctly. And about 20% of the few bugs we had were forgetting to register one of the dependencies correctly.
The problem is that a new developer cannot do it the same way. Remember, this is the result of three years of experience and we now develop in a way that is much like TestDriven in that classes are introduced in the same way (“oops, not a part of this class’ responsibility – new class”). And we wouldn’t be able to do it this way if we hadn’t felt the pain early on.
The guy who introduced patterns, principles and TDD/BDD/DDD to me once said something I think is truth: “If you haven’t felt the pain of doing things the wrong way – you’ll see the right way as unnecessary friction”. And to learn to do things better – you need to approach it with in a certain way – which is all about taxonomy levels:
When beginning – you need to adhere strictly to the rules.
When advancing – you can start applying the rules when necessary.
When competent – you don’t need the rules.
When omnipotent – you make the rules.
If you skip one of the levels – you are bound to f#¤% up. At my current programming level – I’m somewhere around competent in the areas I specialize in. I still have epiphanies when I look at other peoples code going “Ahh, of course – that’s so simple that I should be spanked for not seeing that right away”, but I no longer think of the principles and patterns as rules. I apply some of the logic of the patterns to my classes when it is necessary – but sometimes the solution is a hybrid between two patterns – or only parts of a pattern.
So, I accept the rules of Test Driven Development, and I accept that you have to walk the path to find the balance, but I disagree in that everything has to be driven by tests. However, your development should be testable – thus, the classes need to be built the same way as they would have been if you had done TDD.