It is good to have the pass/fail reporting in Slack. However, we can amp it up by providing the test results, too.
We can read the test results and coverage report and include the information in the output if it is available.
Test Parsing Script
We will create functions to add to our groovy script to parse the test results. These functions can go at the end of the file outside the pipeline block.
The function readTextFile will read a text file, handling a UTF-8 byte order marker (if present) at the beginning:
String readTextFile(String filePath) {
def bin64 = readFile file: filePath, encoding: 'Base64'
def binDat = bin64.decodeBase64()
if(binDat.size() >= 3
&& binDat[0] == -17
&& binDat[1] == -69
&& binDat[2] == -65) {
return new String(binDat, 3, binDat.size() - 3, "UTF-8")
} else {
return new String(binDat)
}
}
The following function requires an import for reading XML files at the top:
import groovy.xml.*
The gatherTestResults will read the test results generated and report the result:
String gatherTestResults(String searchPath) {
def total = 0
def passed = 0
def failed = 0
findFiles(glob: searchPath).each { f ->
String fullName = f
def data = readTextFile(fullName)
def trx = new XmlParser(false, true, true).parseText(data)
def counters = trx['ResultSummary']['Counters']
// echo 'Getting counter values...'
total += counters['@total'][0].toInteger()
passed += counters['@passed'][0].toInteger()
failed += counters['@failed'][0].toInteger()
}
if(total == 0) {
return "No test results found."
} else if(failed == 0) {
if(passed == 1) {
return "The only test passed!"
} else {
return "All ${total} tests passed!"
}
} else {
return "${failed} of ${total} tests failed!"
}
}
We’ll also want to gather the code coverage results:
To tie it all together, we’ll store the test results after running the tests at the top of the file:
def testResult = ""
We’ll add the call to the function to get the test results in our Publish Test Output stage:
stage ("Publish Test Output") {
steps {
script {
testResult = "\n" + gatherTestResults('TestResults/**/*.trx')
}
mstest testResultsFile:"TestResults/**/*.trx", failOnError: true, keepLongStdio: true
}
}
The script will fail for several runs as we work through script approvals. Jenkins will generate a script error using the static toDouble() method. To fix this:
- Go to Manage Jenkins ⇾ Script Approval (recommend doing this in another window as we will go back to this)
- Approve the use of toDouble:
Then run again. The next script approval will be for the use of round():
We should now get successful runs from Jenkins posting into Slack:
This does look odd, however, as there are 3 lines that should appear instead of 1. This is because we are building and testing against a release configuration instead of a debug configuration. If we want more precise information, we can change the Jenkinsfile.groovy to build and test debug, then build release, package, and potentially publish it.
First, change the Build Solution and Run Tests stages to look like this (note the configuration was changed from “-c Release” to “-c Debug”:
stage('Build Solution - Debug') {
steps {
echo "Setting NuGet Package version to: ${nugetVersion}"
echo "Setting File and Assembly version to ${version}"
bat "dotnet build --nologo -c Debug -p:PackageVersion=${nugetVersion} -p:Version=${version} --no-restore"
}
}
stage ("Run Tests") {
steps {
// MSTest projects automatically include coverlet that can generate cobertura formatted coverage information.
bat """
dotnet test --nologo -c Debug --results-directory TestResults --logger trx --collect:"XPlat code coverage" --no-restore --no-build
"""
}
}
Then, after publishing code coverage, add the following stages:
stage('Clean') {
steps {
bat "dotnet clean --nologo"
}
}
stage('Build Solution - Release') {
steps {
echo "Setting NuGet Package version to: ${nugetVersion}"
echo "Setting File and Assembly version to ${version}"
bat "dotnet build --nologo -c Release -p:PackageVersion=${nugetVersion} -p:Version=${version} --no-restore"
}
}
Running ‘dotnet clean’ will not delete the TestResults or the code coverage output, so we can continue to archive these results.
The new results in Slack now look like this:
The final Jenkinsfile.groovy can be seen on GitHub with the complete list of changes at Jenkinsfile.groovy.