Testing with Analytics Unit Tests
At this year's dialog, there was a great presentation from Maike Mock, b.telligent, about Continuous Integration with Longview.
During her presentation, Mrs. Mock guided the audience through several approaches for Continuous Integration, discussing different tools and methods in that context. For the aspect of testing Analytics applications, she presented a Longview-internal approach.
After several requests during and after this presentation, I want to provide with this article the according material together with a description about how to use it.
One challenge for creating Analytics applications in Longview is to ensure that implemented functionality will not only work now in the intended way, but in the future as well. This could be done of course manually, by proving all existing functionality after each change of the application. Which easily sums up to a big effort. Or, another option, have a specific testing period for the whole application every time before rolling out a new release. Which is a certain effort as well.
These approaches are limiting the base concept of “Continuous Integration”. And, if agile methods like Scrum are in place, where after each sprint a potentially releasable product should be available, this processing is creating some troubles for the development team.
The solutions for such situations is mostly the same: automated testing. And there are myriads of options available for “classic” programmers, like Unit Testing, UI (User Interface) Testing, Black/Gray/White Box testing, etc… Within the Longview Analytics teams, we have many of these automated testing options in place. So e.g. for our Analytics applications, we are using Ranorex, MS CodedUI, our Webservice interface and others, for automated tests in nightly builds and in continuous integration builds.
All these tests work on UI level. Which is fine. But for enhanced application development, we learned that this approach is no longer sufficient. If an application consists on different logical modules, it is important to ensure that these modules will behave in the expected way. But in most cases, such modules are working internally and don’t expose an UI which can be tested in the classical way.
(Note: For such modules, it is helpful if functionality is executed by the EXECUTE function (instead of using formula references). In this way it is even possible to follow a test-driven approach during application development, which basically means to write the test first, and implement the functionality in a way that it meets the expected test results.)
For such a module-based testing, but for other functional testing as well, we have established a specific concept during the creation of “Spotlight”, our AdHoc analysis offering. We call this “Analytics Unit Tests”.
Testing with Analytics Unit Tests
The base idea behind the Analytics Unit Tests is to execute application functionality and compare the actual results with an expected result. This approach is very similar to classic unit tests on code level (hence the name).
Attached to this article, you will find a zip file containing two Analytics documents, testCaseTemplate.apd and testSuite.apd.
The testCaseTemplate.apd can be used to execute and proof a test case on a step-by-step base. Like, in the case of a report, set some preconditions, switch filters to a certain setting, execute a calculation, maybe some other user interaction, and proof the results visualized in the reports main table. That would be a test on UI level.
In the above described case of a module containing a certain function set, like different ways to spread planning data in a data sheet, the steps of the test case document would execute the exposed functions of the module. In order to ensure that the functionality is still working as expected after future application changes.
How to use Analytics Unit Tests
Follow these steps to include the Analytics Unit Tests into your application
- Place the folder AnalyticsUnitTest as a sibling to the application folder which you like to test.
- Open the file application.apa in the Analytics Designer and add your application folder as an external folder (over File/Application Settings…/External Folders).
- Open the file testCaseTemplate.apd and save it under a new name, like testCaseDataSpreading.apd.
- Open Layer 1 in this document and follow the steps to include the single steps of the testcase.
Once your test cases are defined, they can be executed at any time. Either manually, or by a certain trigger, like during the continuous integration process.
There are for sure more aspects to talk about in this context, and I want to encourage you to use the comments section of this posting to start an according conversation.
- 711 views
- 2 versions
- 0 replies
- 1 follower
- Posted By:
- Hans Peter Wolff
- September 22, 2017
About this forum
- 26,359 views
- 30 topics
- 3 followers
Your forum to share new discoveries, nifty solutions and helpful workarounds