C/AL Test Result Reporting Ideas

C/AL Test Result Reporting Ideas

In this article, I would like to share a few small things I made so I can get the most out of the C/AL test results and I can get my list of bugs/errors as quick as possible.

Let me start with saying thank you to Nav-Skills for hosting the webinars to discuss the topic of automated testing and I would definitely recommend everyone to watch this Webinar : Start With Automated Testing in Dynamics NAV. I’ve been thinking about how I would like to use the results of the test runs and what would be helpful to developers and to managers.

If you interested in setting up your environment for scheduled automatic testing check out this How Do I Video from Microsoft: How Do I Automate Daily Test Execution in Microsoft Dynamics NAV 2016.

Test Execution Errors

When I was setting up the testing I had some errors which made me think that I really need to know if this happens during a project so I can fix the testing quickly. I decided to use two codeunits to be able to send an email notification with the error message if my test execution would fail. I implemented this pattern

IF NOT CODEUNIT.RUN(CODEUNIT::"Automated Testing") THEN
  SendErrorNotificationEmail(GETLASTERRORTEXT);

(If you would like to know more about this check out this msdn article: GETLASTERRORTEXT Function)

As an example, I received a few emails with this error:
The test execution failed with the following error message: The CAL Test Enabled Codeunit already exists. Identification fields and values: No.=’1′.

Failed Tests

I would like to receive the errors from my tests overnight so when I go to work I can action them first thing in the morning. So, I implemented the Pass Rate Calculation from Microsoft’s How Do I video and attached the output of CALTestManagement.ExportTestResults(TRUE, TRUE) to the email that includes the stats. I decided to change the export format from XML to CSV (XML Port 130403 – CAL Export Test Result) so I can easily open the file in Excel from the Email.

In the file, I can see the Test Codeunit, Method, Error Message and the Call Stack which is useful enough for me to start fixing the issues.

ErrorLog

My email arrives like this (not always showing a 100% :)) with a bullet list of the pass rate and the attachment:

TestEmail

Statistics

I thought doing the automated tests and capturing the results would give us interesting numbers that we could look at over a period. I thought of a few things like using excel or NAV chart but then I realised that I have all the information in a table in NAV and it has a page as well so that means that I could use Power BI! I exposed the CAL Test Results (Page 130405) as a web service and used the OData web service in my Power BI Dashboard.

I added a custom field based on the “Start Time” field that only shows the date instead of date time.

I created two charts based on the same information because I wanted to see the different representations of the results. I think the line chart show the trend better whilst the bar chart is good to see the ratio per day.

PowerBiCharts

Of course, these charts are not enough without knowing what happened during the reported period in the project or sprint.

As a developer, I am curious to see my numbers after 6-12 months because I think it will help me with my own progression.

Summary

I hope you enjoyed this article, as you can see with little effort I added small customisations and created a simple Power BI dashboard. This now gives me my to-do list in the mornings automatically and it enables me analyse my performance from a testing perspective.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s