You don't know if you don't measure

I'm excited about two new team members that we have, Thomas Voß and Allan LeSage, who are working to increase our quality infrastructure. One of the first things they've been working on is getting per-commit code coverage measurements of our unit tests in to Jenkins. I think that this is great because, while we have tests, we have no real way to know if we have enough tests. Code coverage isn't a magic bullet there, but it does give us some idea of where we stand.

If you had asked me (as soon as yesterday) about dbus-test-runner's test suite I would have described it as good. It's a small amount of code, and it has quite a few tests. So it must be good. Allan submitted a merge request to add code coverage measurements. Here's what came out on the summary:

Line Coverage Functions Branches
79.4 % 239 / 301 88.0 % 22 / 25 62.0 % 67 / 108

Ouch! I don't think we could really describe that as good. Not awful, but it should be better, especially for a test tool!

This story does have a happy ending. I took a little bit of time to make a new branch that adds tests, but also makes the utility more testable so that the code can be hit in a reasonable test. Overall, a big win for dbus-test-runner. Here's the results after that branch:

Line Coverage Functions Branches
95.2 % 317 / 333 96.2 % 25 / 26 84.4 % 103 / 122

Comments: Identi.ca | Twitter

posted on Fri, 09 Dec 2011 at 23:31 | permanent link