Code Coverage for End Users

CarlosRoggan · ‎12-12-2012

Background 0

This Blog is about measuring the code coverage of an installed java product.

It is based on Eclipse as usage example and assumes some basic knowledge about Eclipse and its launch configurations.

Background 1

The typical use case of (java) code coverage measurement is the following:

-> During the automated build,

-> the automated tests are executed automatically

-> the lines of code, which are covered by the tests, are automatically detected

-> a report is automatically generated

-> which might be automatically pushed to a monitoring tool.

-> At the very end of the chain, a human eye might have a close look at the results.

->-> What we can see: there's a strong focus on the automation <-<-

Background 2

A more Developer-centric use case is to have the possibility to check the covered code right during development (e.g. using EclEmma in Eclipse):

->Implement the automated test

-> run it ("JUnit" launch configuration in Eclipse)

-> check EclEmma-plugin for the code which has been covered.

This ability is provided by the popular Eclipse-plugin EclEmma.

Background 3

A use case, which is Developer-centric, but not automated, is recently being supported by EclEmma:

Instead of using EclEmma for a junit-run, it can be used as well for a "normal" Eclipse application.

-> Implement the application-code in Eclipse

-> run the dev-instance in eclipse ("Eclipse Application" lauch configuration in Eclipse)

-> click through the coded application

-> check EclEmma-plugin for the code which has been touched

Foreground

Now, what this Blog actually wants to focus on is a rather seldom use case: the end user centric use case.

The user uses a java application and wants to check the used code.

->-> Here we have a strong focus on the end user <-<-

Which means:

-> no java sources

-> no build process

-> no development environment

Typically, the code-coverage tools create instrumented byte-code during the build. This instrumented code contains extra code that records that the statement (or branch or line of code) has been executed.

Since in our use case we're dealing with a ready product, there's no possibility of instrumenting the code during build.

Just an application which should be measured.

=> How can the underlying code been measured?

Underground

The java code-coverage tool JaCoCo has solved this requirement with the so-called on-the-fly byte-code instrumentation.

JaCoCo hooks directly into the Java VM and instruments class files during class-loading.

This is done using the Java VM-parameter -javaagent which has to specify an agent-implementation as defined in package java.lang.instrument.
JaCoCo provides an implementation of such agent.

Basically, such an agent is invoked prior to the main-method of the java-application and runs in the same Java VM.

Note that this VM-parameter is supported since JDK 5.

JaCoCo is a free Java code coverage library distributed under the Eclipse Public License.

More information can be found on the JaCoCo website (see below)

One additional advantage of JaCoCo is that its mechanism is not tool-specific, it can be used for any java program by configuring the VM.

JaCoCo stores the coverage result in a .exec file.

If desired, the .exec file can be imported into eclipse and be viewed in the EclEmma Coverage View.

JaCoCo provides Ant tasks for facilityting further activities like managing .exec files and creating reports.

The following sections provide examples based on Eclipse.

We assume that the application for which we want to measure the code coverage, is an eclipse-plugin.

Playground 1

The goal of this section is to measure the code coverage of an eclipse application.

Step 1: Install JaCoCo

Download the library jacocoagent.jar and store it on the local file system (see link section below).

Step2: Configure the Java VM

For Eclipse, we have to edit the eclipse.ini file, which can be found at: <installationPath>/eclipse/eclipse.ini
We have to add the mentioned parameter and provide the path to the downloaded jacocoagent.jar.

Thus, the most simple configuration would be the following:

Append the following line to eclipse.ini:

-javaagent:C:/temp/jacocoagent.jar

As result JaCoCo will write an execution file with default name to the same directory (as eclipse.ini)

It will append results and overwrite an existing file.

Further refinement possible, though:

-javaagent:C:/temp/jacocoagent.jar=includes=com.my.relevant*,excludes=org.not.relevant*,destfile=C:/temp/executions/myCoverage.exec

We can see that the name of the resulting execution file can be specified and that filtering is supported, such that not all loaded class files are measured.

One more example:

-javaagent:C:/jacocoagent.jar=append=false

Instructs JaCoCo to create a new file (overwrite existing)

The syntax is as follows:

-javaagent:[yourpath/]jacocoagent.jar=[option1]=[value1],[option2]=[value2]

More details can be found in the JaCoCo website (see below).

Step 3: Run Eclipse

Now we're ready to start Eclipse.

After using the plugin which we want to measure, we close Eclipse.

When exiting the JVM, the coverage file is written.

The attempt of reading such coverage file with human eye will result in disappointment: it is not human-readable.

We have to generate a report as described in the following section.

Playground 2

After we've collected the coverage data and stored it in a .exec file, we want to check the results. We want to have a report where we can see which java classes and methods have been covered.

In order to cover such topics, JaCoCo provides a bunch of Ant-tasks, which are provided within the library jacocoant.jar.

For our specific use case we need the Ant task jacoco:report.

Step 1: Download the JaCoCo Ant tasks

Download the jacocoant.jar file from the location specified in the last section and store it on the local file system.

Step 2: Install Ant

If not already available, follow the instructions on the Ant website.

Step 3: Create Ant script

JaCoCo provides the Ant task jacoco:report, but doesn't provide the Ant script, since it is easy to write such script.

Basically, it is just some Ant-xml which invokes the mentioned Ant-task and provides some required information to this task.

We have to specify the path to the .exec file as input for the report generation.

Furthermore, we have to provide the path to the class-files of the application which has been measured. In our example this is the Eclipse plugin directory.

Optionally, it is possible to specify a directory which contains the source files, but this is not our use case.

Additionally, the output format has to be specified (html, xml or csv) and optionally the path to the output directory.

Here's an example for such an Ant script, which can be saved as createReport.xml

<?xml version="1.0" encoding="UTF-8"?>

<project name="Example Ant Build to write JaCoCo Report" default="report" xmlns:jacoco="antlib:org.jacoco.ant">

 <property name="work.dir" location="C:/temp" />

 <property name="lib.dir" location="${work.dir}/lib" />

 <property name="exec.dir" location="${work.dir}/exec" />

 <property name="report.dir" location="${work.dir}/report" />

 <property name="eclipse.plugin.dir" location="C:/myEclipse/eclipse/plugins" />

 <property name="src.dir" location="C:/perforce/myproject"/>

 <taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">

  <classpath path="${lib.dir}/jacocoant.jar" />

 </taskdef>



 <target name="report">

  <jacoco:report>

   <!-- execution data as input for report generation -->

   <executiondata>

    <file file="${exec.dir}/myCoverage.exec" />

   </executiondata>

   <!-- the class files of the measured application -->

   <structure name="MyApplication">

    <classfiles>

   

     <!-- eclipse directory containing plugins as jar files-->

     <zipfileset dir="${eclipse.plugin.dir}" includes="com.my.relevant.plugins.*.jar" />

   

     <!-- needed if there are plugins which are not compressed -->

     <fileset dir="${eclipse.plugin.dir}" includes="com.my.relevant.plugins.*.class" />

    </classfiles>

   

    <!-- specifying the source files allows to view the covered code -->

    <sourcefiles encoding="UTF-8">

     <fileset dir="${src.dir}" />

    </sourcefiles>

   </structure>

  

   <!-- write HTML report -->

   <html destdir="${report.dir}" />

  

  </jacoco:report>

 </target>

 

</project>

Step 4: Execute the Ant script

Open the command prompt, step into the Ant installation directory and invoke ant -f createReport.xml.

Playground 3

One more interesting feature which is supported by JaCoCo: merge of multiple execution files.

This can be the case if several instances of the same application are used, or if different persons are using the application.

In such cases, the result will be a bunch of multiple .exec files.

These files can be merged into one global coverage file using an Ant taks provided by JaCoCo.

Step 1 and Step 2:

Same as above: download the jacocoant.jar file and install Ant.

Step 3: Create Ant script

JaCoCo provides the Ant task jacoco:merge

It requires the name and path of the (merged) result file.

Furthermore, it needs the directory of the .exec files which should be merged.

Here's an example for such an Ant script, which can be saved as merge.xml

<?xml version="1.0" encoding="UTF-8"?>

<project name="Example Ant script for merging multiple jacoco.exec files" default="merge" xmlns:jacoco="antlib:org.jacoco.ant">

 <property name="work.dir" location="C:/temp" />

 <property name="lib.dir" location="${work.dir}/lib" />

 <property name="merge.dir" location="${work.dir}/merge" />

 <property name="exec.dir" location="${work.dir}/executions" />

 <taskdef uri="antlib:org.jacoco.ant" resource="org/jacoco/ant/antlib.xml">

  <classpath path="${lib.dir}/jacocoant.jar" />

 </taskdef>

 <target name="merge">

 <!-- Invoke the ant task and provide the name of the merged result file -->

  <jacoco:merge destfile="${merge.dir}/merged.exec">

   <!-- specify the folder which contains all exec files which have to be merged -->

     <fileset dir="${exec.dir}" includes="*.exec"/>

  </jacoco:merge>

 </target>



</project>