How useful are code coverage reports?

1

Rollbar

This week, as a relevant anecdote, we saw some notifications from Rollbar that showed us something was wrong with a new feature. We managed to get a coverage report for a single module of our code (took a little while), and we saw that the bugs identified in Rollbar were actually not covered by tests.

#Error Tracking #Bug Trackers #Crash Reporting 18 social mentions

2

PHPUnit

Back in 2010, I was working for Plusnet, an ISP in the UK (Referral link). The engineering team were still getting to grips with unit testing our code. What we saw time and time again was many tests being written, high coverage reports, and management was happy. However, if you checked out a test, we saw that tests were running, and executing code, but they were not asserting anything. This meant the tests were more of a linter. So I wrote the --assert-strict feature in PHPUnit (which has subsequently become fail-on-incomplete). This helped us raise the quality of our tests at Plusnet.

#Development #Online Services #Automated Testing 30 social mentions