Schizophrenia Driven Design - Unit Testing
It starts with choosing a framework which especially supports unit testing (NanoContainer, PicoContainer,Spring, Guice, EJB 3 etc.). No one can argue against that. A framework always introduces additional complexity and dependency on the infrastructure, but the increase of test coverage is good for the quality on the other hand and outweights the bloat in general.
After the introduction of such a framework, developers tend to foget about the actual reason of its introduction - the testability. Although every single layer could be easily replaced, mocked and unit tested, it actually happens only rarely. In fact the majority of projects do not use the testing capabilities of the particular framework at all. Not only the test coverage is poor, but especially the quality as well:
- Only service / transaction boundaries are tested. The classes behind the facades are not. It is actually functional testing, not unit testing. The actual benefits of the "test first" frameworks are not leveraged.
- In continuus integration environments the majority of the tests is even commented out. With this approach sun will always shine in hudson land :-).
- Only trivial, CRUD-like use cases are tested at all - the hard stuff not. It is mostly driven by the QAs.
- Mocks are only used for trivial cases.
- Assert statements are only used rarely - sometimes only a subset like Assert.true. System.out.println still rules.
I recognized the discrepance between articles, conferences and the reality several years ago. At the conferences the gurus told you how to practice "test first", in reality I was happy about enforcing usable tests at functional level.
At the beginning I thought that it is only my unlucky experience, but I began to ask during conferences, workshops and trainings about the mismatch. I saw only few projects with good test coverage and pragmatic tests so far. These were actually not business projects, rather than frameworks or technical libraries, which are far easier to test (the developer is the domain expert at the same time). Actually no one answered the survey positively. But: is your real world, business project (=frequently changing requirements, lot of pressure, not academia) well unit-tested? If yes - drop me an email - you will be the first :-). But then I would like to see the tests as well.
Knowing the gap it is actually better to prescribe the testing of only the crucial things and expecti high quality here. 80%+ test coverage with crappy unit tests and green bars cause more damage, than use.
Why Schizophrenia Driven Design? Look at the definition: "...a mental disorder characterized by abnormalities in the perception or expression of reality. It most commonly manifests as auditory hallucinations, paranoid or bizarre delusions, or disorganized speech and thinking with significant social [teams] or occupational dysfunction. Onset of symptoms typically occurs in young adulthood, with approximately 0.4–0.6% [this is not true in software engineering - I would say about 80% of projects are affected so far] of the population affected. Diagnosis is based on the patient's self-reported experiences and observed behavior [or retrospectives :-)]. No laboratory test for schizophrenia currently exists.... [except greenbars without asserts, but with 5MB System.out.println :-)]"
[see also "Real World Java EE Patterns - Rethinking Best Practices" book for examples and more detailed - pattern specific unit test discussion]
I think you are being too harsh. I am currently working for a company producing real, customer-facing, software. Test-Driven Development (TDD) is practiced and enforced by peer pressure from within the development team.
In particular, tests are expected to run "clean" and produce no console output. Where console output is a significant part of the application (such as logging) it is captured during the test and asserted.
This is not the first job I have had where test-first, and good unit-testing practice generally, have been a key part of practice. Such companies certainly do exist!
Posted by Frank Carver on January 27, 2009 at 11:47 AM CET #
I tried to be explicit not harsh. I'm not doubting the existence of such projects, but I'm not sure about the mainstream.
The message should be - If you really want to do TDD - do it whole heartly.
Partial testing causes a real damage.
For clarification: consequent testing with continuus integration is a really good thing.
really thank you for your feedback,
so there is still some light out there :-),
Posted by Adam Bien on January 27, 2009 at 01:11 PM CET #
Associations that come to my mind:
Testing is overrated:
Paranoia is close to Schizophrenia when looking in the trenches of IT-development:
Posted by Jonas on January 27, 2009 at 03:15 PM CET #
Paranoia is very close. I actually planned to name of the next posts Paranoia Driven Design - I hope it's o.k. :-).
Can you share my experience?
Thank you for the pointers!,
Posted by Adam Bien on January 27, 2009 at 07:35 PM CET #
"Yo no creo en brujas, pero que las hai, hai!"
Think again, you are a luck man...
I am the architect (the former just left to start another project..) of a project based on all those evil J2EE patterns, which means I simply can't touch it without starting a very heavy portal platform on top of a very heavy (EJB 2.x) application server platform...
How should I call this kind of "design" ?
Posted by Rodolfo on January 29, 2009 at 02:46 AM CET #
in case the patterns were applied consistently - you can easily switch to "weightless" :-) EJB 3 applications and kill about 80% of the old infrastructural code. I'm actually involved in such a project.
I will call your design: "Legacy / Heritage Design"... :-)
What is: "Yo no creo en brujas, pero que las hai, hai!"?
Why I am a lucky man?
Posted by Adam Bien on January 29, 2009 at 09:44 AM CET #
Actually my first thought was to start refactoring all the code to keep it cleaner and focused on it's business domain instead of focused on patterns.
But the project has a long and problematic history (I can imagine why..) so the customer is always pessimistic and will not allow any architectural or design change so I will have to add features and fix bugs on top of this mess.
I called you a lucky man since I got the impression that at least you has been trying to write better tests (by not using what u call Schizophrenia driven design) and to me this is just a dream given the design (and political issues) I usually have to deal with.
Finally, that phrase I guess is a spanish expression. It means something like "I dont believe in witches, but they are out there !". By this I meant I also have been looking for business projects with good unit test practices but actually never saw it..but they are out there ! ;)
Posted by Rodolfo on January 29, 2009 at 02:52 PM CET #
"...Actually my first thought was to start refactoring all the code to keep it cleaner and focused on it's business domain instead of focused on patterns..."
Yes, kill the procedures and bring the objects back! :-).
I know a song for us: "Out Demons, Out" from Edgar Broughton Band :-).
I'm a freelancer - in every bigger project I was confronted with lot of politics and unlogical behavior. But this is just business as usual - lot of golf course and hype driven decisions.
Posted by Adam Bien on January 29, 2009 at 02:59 PM CET #