My experience with unit testing

I am not new to programming but have had bad programming habits for too long. "Enough is enough", I said to myself one day and decided to use a test-first approach for the next project.

Here's what I experienced.

1. Although I created the test cases before writing the classes, I didn't keep up with the policy for long. I found it a little distracting to think of and write a test case before any change to my code. The result was that soon my test cases were totally out of sync with the code.

2. It is difficult to be disciplined enough to keep test cases up to date specially when you are working with tight deadlines. Effort estimation must take into account this fact.

3. I had to continuously refactor my code to make it more testable. Writing test cases turned out to be a great way to write well formed APIs and good code in general. I had to make several modifications to my initial design to allow dependency injection. Moral of the story - automated unit tests can't be implemented as an after thought.

4. I found it difficult to test private methods and had to change their visibility to test them.

5. I found it difficult to automate everything. There were times when I just wanted to print the values of a hashmap and inspect manually. I found it too time consuming to completely automate the testing in such cases. Perhaps it's a case of old habits. But I realized how much of our testing is based on manually inspecting the results. It is difficult to break this habit and to learn to rely on automated test results.

6. Mocks are confusing (jmock) but very very helpful.

7. Seeing the unit tests run successfully didn't give me much confidence except for the first time when I ran them with fresh code. I can think of several reasons for it. Firstly, as I mentioned before, manually verifying the results of a testcase gives one a confidence that's difficult to achieve by running automated tests. There is something reassuring about seeing the results with your own eyes. Secondly, my test cases were not comprehensive enough to test full functionality. So after some time, I didn't see much value being added by running these tests. And lastly, I was coding for a database intensive CRUD application; my code was heavily dependent on file i/o and database access, both of which needed extensive integration testing. After a while, the unit tests were rendered completely useless and I was hardly running them.

8. There is only so much that unit tests can achieve. Most of the bugs in my code were caught during integration testing. But I must admit that at least some of these could have been caught during unit testing if I had better unit tests.

10. There should be a set of integration tests that check essential functionality. These tests may not run during the continuous build but must run during the nightly build.