NCover allows you to measure how much of your code is covered by your testing. Measuring code coverage is critical if you need to deliver high quality code quickly. Repeatedly, I have seen the addition of unit testing and code coverage reporting to increase developer productivity.
NCover produces actionable information showing you which lines of code are not tested via your unit tests.
I’ve been using NCover since 2004. Since that time, I’ve installed and configured it for hundreds of developers and build servers. I have a very defined SDLC which allows developers to get coverage locally before publishing changes. Once changes are published, the build server publishs coverage results. This way, a developer who sees a reduction in coverage on their local machine knows they are not finished with assignment. I’ve done all this with a coverage tool which is 10+ years old. The longevity of the tool and process has been extraordinary.
NCover moved to version 5 in 2014 and I felt it was time to upgrade. To be honest, NCover has been poking me to upgrade for a while now! This post documents the upgrade of my local development process to NCover 5 Desktop.
My Current Process
My SDLC process allows for a single developer to run one command (testfast.bat) to perform the following:
- Drop the local test database
- Create the local test database
- Populate the local test database with test data
- Get all project dependencies
- Compile all the code
- Run coverage unit tests with coverage
- Return test and coverage results
- Fail if any unit tests fail
- Fail if coverage thresholds are not met
That is a lot! It’s all driven from one command which calls an MSBuild script. (I can on-board a new developer in minutes)
NCover 5 has a different model than 3. It runs a service which will collect coverage on any running .NET process. You configure an NCover project to monitor only the executables and dlls which you’re interested in. It’s all very GUI driven which some users may like. This requires some changes to my local developer process.
My first job to migrate from NCover 3 to NCover 5 Desktop was to see what I could do from the command-line. Luckily, NCover has decent documentation. I needed to change my local build process to use the new tools.
The New Process
- Drop the local test database
- Create the local test database
- Populate the local test database with test data
- Get all project dependencies
- Compile all the code
- Add/Update the NCover service with my project definition
- ncover project-import –file=<path to file> –overwrite
- Enable my NCover project
- ncover enable <project name>
- Run unit tests
- Return test results
- Fail if any unit tests fail
- Get coverage results from service and display
- ncover summarize –project=<project name>
- Fail if coverage thresholds are not met
Adding or removing test dlls requires one of the following:
- A manual update of project definition (step 6 above)
- Update the project definition via the UI, then export the new definition
- ncover project-export –project=<project name> –file=<path to file>
The Downsides
There are three problems with this:
- The coverage configuration used to reside inside my MSBuild script. Now it’s’ contained in an NCover project-specific file.
- The coverage reporting tool is slow.
- The same process will not work on a build server. Yikes!
I can work with #1 fairly easily. This can be handled with a little bit of patience and learning.
In version 3, the report system generated static html which was very fast to run locally. Also, I could zip up the report pages and send to someone for review. In version 5, The reporting tool only runs on a local webserver process. I’m running an i7 with 8 logical processors and 12G of Ram. It’s a bit slow compared to the lightning fast static html files (of course). Hopefully, they”ll improve this in the future.
#3 is a problem. NCover 5 Desktop allows you to view results via a locally running webserver through the loopback address (127.0.0.1). NCover doesn’t support changing this address in Desktop. You need Code Central + Collector for this. ($2,700). There is a somewhat crippled output report you can include in a build server process. However, it doesn’t allow for drill down of information. This is a mistake in my view. NCover 5 Desktop is already expensive ($658).
The beautiful part of my original process is the same process developers use is used by the build server. There are no differences. It solves the “…well… it works on my machine” discussion. In my mind, my build server is just another developer compiling code and publishing results. Asking me to spend an extra $2,700 simply to modify a hostname is… um… curious. As a work around for this, I setup a reverse proxy to serve the local content to a public IP. It works but I shouldn’t have to do this for something so simple as changing a hostname.
The Upsides
There are many upsides to the upgrade however:
- The reports are much more accurate.
- NCover 3 didn’t handle things like anonymous types and methods, generics, and LINQ well.
- The new metrics are insightful.
- Although branch coverage is still the most important to me, the others provide excellent supporting information. What’s your CRAP score?!
- It’s faster. A lot.
- It will be easier build new projects. There is less setup required with the new version.
Final thoughts
I wish I could change the hostname on the process. Also, NCover should help small teams and startups with pricing cuts. That said, NCover is the best quality tool any development team can buy. Period.
