In my previous blog CAP CI/CD using Azure DevOps - Part 1, I showed how to setup continuous integration for unit testing automation. From there, you were able to see the generated Unit Test report from Azure Build Pipelines. It is a nice looking report that show's a high level overview of the pass and failed tests and it's also nice to see that you can drill down to the details of the failed test cases. However, we can still add further reporting to this setup, i.e. code coverage, and this is the topic in this blog.
Unit tests themselves are not enough. It would be good if we have an idea of how much of our test cases are covering in terms of lines of code executed. By adding code coverage reporting, we are able to assess how many percent of our code has been tested. A high percentage of coverage will give us confidence that our code is free from bugs.
Note that it is not necessary to focus on achieving 100% code coverage as it takes away significant time from the developer and this approach is more focused on quantity. The best approach is to focus on the quality of the test cases by making sure that critical and important features have test cases written. Knowing what test cases to prioritise to write is the key!
If you follow through Part 1 of this blog series, then you already have a CAP Project that look exactly like the one in this repository -- cap-devops-part1
Let's continue on working on this project and setup the code coverage reporting.
This will export the code coverage report as cobertura in XML format, because this is one of the few supported code coverage report format that is supported by Azure DevOps.
If you followed the steps above, then the portion of your package.json file should look like below:
It is simply a task that will publish the generated xml file called cobertura-coverage.xml to Azure Build Pipelines.
Switch back to your Azure DevOps account and do the following:
As you can see, there is a summary section and there's also the coverage section where the Line coverage for the JavaScript file catalog-service.js is 100%.
For our demo CAP Model project, achieving 100% code coverage is relatively easy because the project is very simple. But this is not always the case for most developments, so don't feel bad if you don't hit 100% code coverage. Again, the important thing is the quality of your tests! Code coverage is merely a guideline and gives you the confidence of how significant your test coverage is.
Code Coverage is a nice addition to our DevOps CI/CD setup and it compliment our Unit Test reporting. And with the Azure DevOps tool, it fits nicely for the consumption of the DevOps Development Team. Not to mention on how easy it is to setup.
So far our DevOps setup is starting to look good, but we haven't really got into the main event yet, which is the continuous deployment stage. This will be the topic for my next blog.
~~~~~~~~~~~~~~~~
Appreciate it if you have any comments, suggestions, or questions. Cheers!~
Unit tests themselves are not enough. It would be good if we have an idea of how much of our test cases are covering in terms of lines of code executed. By adding code coverage reporting, we are able to assess how many percent of our code has been tested. A high percentage of coverage will give us confidence that our code is free from bugs.
Note that it is not necessary to focus on achieving 100% code coverage as it takes away significant time from the developer and this approach is more focused on quantity. The best approach is to focus on the quality of the test cases by making sure that critical and important features have test cases written. Knowing what test cases to prioritise to write is the key!
Prerequisites
- SAP Business Application Studio or Visual Studio Code
- SAP Cloud Platform Account
- GitHub Account
- Microsoft Azure DevOps Account
| Note: If you want to follow through the steps described in this blog, you can register for a free Microsoft Azure DevOps account if you don't have one yet. |
Setup Code Coverage
If you follow through Part 1 of this blog series, then you already have a CAP Project that look exactly like the one in this repository -- cap-devops-part1
Let's continue on working on this project and setup the code coverage reporting.
- 1. Go back to your code editor and execute below command in the terminal:
> npm install --save-dev nyc| This will install Istanbul node module -- a JavaScript test coverage reporter. If you want to know more about this tool, you can visit the official website https://istanbul.js.org/. |
- 2. In the test-azure npm script, add below script just before the mocha command:
nyc --temp-dir=./report/raw --reporter=cobertura --report-dir=./reportThis will export the code coverage report as cobertura in XML format, because this is one of the few supported code coverage report format that is supported by Azure DevOps.
If you followed the steps above, then the portion of your package.json file should look like below:
"devDependencies": {
"chai": "^4.2.0",
"chai-http": "^4.3.0",
"mocha": "^7.2.0",
"mocha-junit-reporter": "^1.23.3",
"nyc": "^15.1.0",
"sqlite3": "^4.2.0"
},
"scripts": {
"test": "mocha tests/test.js --timeout 15000 --exit",
"test-azure": "nyc --temp-dir=./report/raw --reporter=cobertura --report-dir=./report mocha tests/*.js --reporter mocha-junit-reporter --reporter-options mochaFile=./report/test-results.xml --timeout 15000 --exit",
"start": "npx cds run"
},- 3. In the azure-pipelines.yml file, add the task below at the very end of the pipeline task list:
- task: PublishCodeCoverageResults@1
condition: eq(variables['Build.SourceBranch'], 'refs/heads/master')
inputs:
codeCoverageTool: cobertura
summaryFileLocation: '**/cobertura-coverage.xml'It is simply a task that will publish the generated xml file called cobertura-coverage.xml to Azure Build Pipelines.
| For those who don't speak spanish language, cobertura is the spanish word for coverage. And for that matter, the filename cobertura-coverage.xml sounds repetitive now, isn't it? |
- 4. With all the necessary changes done, it is now time to commit and push the changes to your branch and let the Azure Build Pipelines get triggered.
Analyse the Code Coverage Results
Switch back to your Azure DevOps account and do the following:
- 1. Navigate to the build pipelines section and you will see the build execution which ended up in error. This is no longer a surprise to you if you have been following the series from Part 1.
- 2. View the code coverage report by clicking on the Code Coverage tab.
As you can see, there is a summary section and there's also the coverage section where the Line coverage for the JavaScript file catalog-service.js is 100%.
For our demo CAP Model project, achieving 100% code coverage is relatively easy because the project is very simple. But this is not always the case for most developments, so don't feel bad if you don't hit 100% code coverage. Again, the important thing is the quality of your tests! Code coverage is merely a guideline and gives you the confidence of how significant your test coverage is.
- 3. Additionally, you can drilldown to the individual JavaScript files and see the lines of codes that was covered by the line coverage report.
Closing
Code Coverage is a nice addition to our DevOps CI/CD setup and it compliment our Unit Test reporting. And with the Azure DevOps tool, it fits nicely for the consumption of the DevOps Development Team. Not to mention on how easy it is to setup.
So far our DevOps setup is starting to look good, but we haven't really got into the main event yet, which is the continuous deployment stage. This will be the topic for my next blog.
~~~~~~~~~~~~~~~~
Appreciate it if you have any comments, suggestions, or questions. Cheers!~
- SAP Managed Tags:
2 Comments
