Sunday, 24 February 2013

Using Delphi code coverage

I couldn't find a good self-explanatory example for the usage of Delphi-code-coverage tool so here you will find a simple example from top to bottom. The idea is really simple; I have a set of units which I want to know whether they are covered by my unit tests or not. Even though there is a lot of information about the project, I couldn't find a really good example explaining all the details about How to properly set up the project, folders, flags, etc. I have used the Delphi-code-coverage-wizard which is great to generate the initial structure (batch file + list of paths + units) but I went manual from there as I couldn't get any output.
I must say that both projects are really good and that's why I'm giving them an opportunity as I want to test the coverage of my projects under Delphi XE.
First step is to correctly identify where your source code is and your unit tests are. This step is really important as to correctly set-up the code coverage project. All my testing code is under my repository and available here: Delphi-repositoty. For this example I'm using a very simple example where I have a unique unit test which tries to cover two units (thundax.AbstractFactory.pas and thundax.Prototype.pas).

My unit test project is called thundax.DesignPatternsTest.exe which has 2 tests:
Now I want to know if those two unit tests cover my previously defined units (thundax.AbstractFactory.pas and thundax.Prototype.pas).

Download Delphi-Code-Coverage and leave it under your bin\ folder or where your test.exe application is (in my case thundax.DesignPatternsTest.exe).
Create two text files called dcov_paths.lst and dcov_units.lst. You can use Delphi-code-coverage-wizard to set up the initial structure, but I would rather go manual as I have more control of the execution.

- The first file (dcov_paths.lst) will contain the path to your source code:
C:\ThundaxDelphiRepository\thundax-delphi-personal-repository\src\Design Patterns\

- The second file (dcov_units.lst) will contain the list of unit tests you want to check whether they cover your code or not.
thundax.Prototype
thundax.AbstractFactory

Your projects needs to have debug information generating a detailed Map file:

Now we can create a small batch file (dcov_execute.bat) or run the following command from the command line tool:
CodeCoverage.exe -e "thundax.DesignPatternsTest.exe" -m "thundax.DesignPatternsTest.map" -ife -uf dcov_units.lst -spf dcov_paths.lst -od ".\" -lt -html -xml -emma -meta

This is the configuration that works for me, where:
  • -e identifies the executable to run (in this case the Unit test framework).
  • -m identifies the map file (the map file contains all the debugging symbols).
  • -ife include file extension (if you are using generics.collections use this option).
  • -uf list of your units (source code, not your unit tests code).
  • -spf list of source directories.
  • -od Output directory (".\" gets  the current).
  • -lt generates log file. (Delphi-Code-Coverage-Debug.txt).
  • -html Html output
  • -xml Xml output
  • -emma EMMA output
  • -meta Generate separate meta and coverage files when generating emma output - 'coverage.em' and 'coverage.ec' will be generated for meta data and coverage data. (Needs -emma as well). These are the important files to run EMMA.
Once the line has been run, you should get the correct output files:
  • CodeCoverage_summary.html (Html output)
  • CodeCoverage_summary.xml (xml output)
  • coverage.es (EMMA output)
  • coverage.em (EMMA output)
  • coverage.ec (EMMA output)
  • Delphi-Code-Coverage-Debug.txt 
  • thundax.AbstractFactory(thundax.AbstractFactory.pas).html (Your report)
  • thundax.Prototype(thundax.Prototype.pas).html (Your report)

If you open the summary you will see the total of lines and %age covered by your test:

And you can inspect the files by doing click on the links:
As you can see, one of the units is only 69% covered. Once we inspect the file we can see that there are lines (highlighted in blue) which are not covered by my unit tests and the green ones which are.

If you want to use EMMA, is as easy as this (once the code coverage tool has outputted the .ec and .em files):

java -cp emma.jar emma report -r html -in coverage.em,coverage.ec

Emma component will generate a coverage folder with the following html file:

The entire example can be found in my Delphi repository.

I hope you find this useful and start using it.
Jordi

4 comments:

  1. You can also use -dproj switch to specify path to your unit test project. In this case it will take list of units to cover directly from project file, it is easier because you don't need to maintain list of units in separate file.

    ReplyDelete
    Replies
    1. Thanks ekot, but sometimes you don't want to include all the units as it could be for example that some of your units are related to the user interface and can't be tested using unit tests, hence you should remove those units and use other techniques to cover those test, via automatic testing or some other UI tools.

      Jordi

      Delete
  2. Hi to all, how is everything, I think every one is getting more from this web page, and your views are fastidious for new users.

    Drupal developer London

    ReplyDelete
  3. What's up to all, the contents existing at this site are really awesome for people knowledge, well, keep up the good work fellows.

    PHP developer London

    ReplyDelete