The areas covered in the survey were test automation, source control, continuous integration, and code quality assurance. Nothing too fancy, but a collection of practices that I consider important, and which are all quite well known. I was curious to see how wide spread these practices are.
In this post I'll cover the results of the first part. I got 62 responses to the survey. Not enough for statistical significance. And there's probably also some bias in the group, since they come mainly from my network. So don't take the results for more than they are.
Test Automation Practices
So 71% of respondents use automated tests. That's pretty high. And then again it's sort of low. The benefits of test automation are so widely accepted, that did actually expect this number to be even higher. The next few questions where only asked of the respondents that use automated tests. Here's how many respondents use each type of test automation:
| Unit tests | 95% |
| Integration tests | 80% |
| Systems tests | 20% |
| UI tests | 14% |
| Acceptance tests | 16% |
| Performance tests | 20% |
| Load tests | 11% |
So unit tests and integration tests are wide used, while the rest are only sporadically used. No big surprise, I'd say.
Next I asked how much of the code is covered. This is an important question; there's a lot of difference in what tests gives you between having low coverage percentages, having high percentage and having 100%. So how much do respondents cover?
| 0-25% | 14% |
| 25-50% | 34% |
| 50-75% | 36% |
| 75-100% | 20% |
| I can't estimate that | 2% |
Not too high. So who writes those tests?
| I do | 98% |
| The testers do | 9% |
| The business people do | 0% |
...responds writes their own tests. That's good. So when are the tests written?
| Before the production code (test first) | 3% |
| Along side the production code (sometimes before, sometimes right after) | 47% |
| Shortly after the production (after production code, but in the same workflow) | 19% |
| After the production code for a complete feature is finished | 2% |
| During a separate test phase | 0% |
...so mainly alongisde or shortly after. But not before. This probably explains why the coverage percentages seem to be a bit low.
OK, I'll the rest of the interpretation to you.
Next post I'll go through the result for the source control questions.
Hey Christian, out of plain curiosity, in what part of the world and from how many people did you take that survey? Some facts are new to me, Like the downfall from a shiney 80% integration to the 20% system tests. I did not see that one coming...
ReplyDeleteMost respondents are from Denmark, but a few are from around the world - including Canada, Thailand, Switzerland and some others.
ReplyDeleteAs stated its based om 62 responses, and they are all reached through my network, so don't take this as being too statistically sound.