In this blog post I would like to provide some details about using test plans for advanced test orchestration in SAP Cloud ALM. The blog will address the following questions:
- When do you need test plans in SAP Cloud ALM?
- How to use test plans in SAP Cloud ALM?
1. When do you need test plans in SAP Cloud ALM ?
In larger implementation or transformation projects you might want to run your test activities in iterations and so need setup multiple test cycles re-using the same or a similar set of test cases.
On top, you might want to roll-out the implemented solution or a global template to multiple organizations like subsidiaries, plants or markets which require dedicated tests per roll-out. Thus, you might want to re-use most of the test cases.
Although it was possible without test plans to run the same test case again while creating new test runs, the overall test status was always set to the status of the last test run.
With the following illustrations I would like to explain the underlying concept with the existing approach without test plans and the new approach while using test plans:
1.1. Lean Test Orchestration Without Test Plans
In the following example, the first test run was set to ‘Passed’ after all test actions were completed successfully and the test run was set to ‘Finished’. Then a second test run was started and test action 1.1 was set to ‘Fail’. In consequence, the overall test status is displayed as ‘Failed’.
After introducing test plans, this lean approach is still available, so you could continue without any overhead. In most cases, this is sufficient for smaller projects and public cloud implementations with an agile or lean implementation approach. Therefore, we still support executing test cases without the need for test plans.
1.2 Advanced Test Orchestration with Test Plans
For more complex projects, customers asked for the possibility to have a dedicated execution context for each test cycle.
To achieve this, you could create a test plan and assign relevant test cases
As the test plan status is still ‘In Preparation’, no occurrences of the assigned test cases are displayed in the Test Execution app yet.
After switching the test plan status to ‘In Testing’, all assigned test cases in status ‘Prepared’ get a new occurrence (execution context) and the test execution can be started.
The next picture shows the situation after Test Case 3 has been set to ‘Prepared’ and Test Plan B has been set to ‘In Testing’.
How to use test plans in SAP Cloud ALM?
New Test Plan App
There is a new tile in the launch pad called Test Plans:
When you open the Test Plan app, you will see a list of existing test plans with their status, time frame, and the aggregated preparation status and execution status of assigned test cases:
You can create new test plans choosing ‘Create’ to get this detail screen in draft mode:
To assign test cases choose ‘Assign Test Cases’ to get this simple assignment dialog:
You can either start typing in the field to get a prefiltered list of test cases, or you could open the value help to get an advanced assignment dialog with search and filter options:
After confirming the selection and the assignment dialog, you will come back to the test plan details in draft mode, showing the assigned test cases:
In draft mode, the test plan is stored but not yet available to work with. Only when you choose ‘Create’, there will be a ‘saved version’ that can be used for processing.
Whenever you open the test plan in edit mode, a draft version is created, which contains all your changes without saving. You can switch between the draft and saved version of your test plans next to the test plan title:
You could even close your browser without losing your changes.You can find more details on draft handling here.
Besides the test plan statuses ‘In Preparation’ and ‘In Testing’, there is a status called ‘Finished’, which indicates that the test phase / test cycle is finished. This status does not have any impact on the visibility of related test cases occurrences in the Test Execution app.
Enhancements in the Test Execution App
In the Test Execution app, you will find a new filter for test plans and the occurrences of test cases are grouped by test plans.
Without setting the filter for test plans, you will see at least one entry per prepared test case belonging to the group ‘(None)’ . In addition, you’ll get another entry for each test case per test plan that is in status ‘In Testing’ or ‘Finished’.
When planning to work with test plans, you could simplify the entry for your testers by setting the filter for the right test plan(s) and then sharing the URL.
Enhancements in the Test Preparation App
In the Test Preparation app, we have also added the filter for test plans to be able to reduce the list of test cases of the whole project to test cases that are not assigned yet to any test plan, or to list all test cases already assigned to a specific test plan.
Outlook on Planned Enhancements
We plan to enhance the test plan capabilities in several iterations. In one of the next itereations we plan to provide the following enhancements:
Analytics:
Filter on test plans in following dashboards:
- Test Execution Analysis and Defects Reporting
- Solution Process Traceability
- Requirements Tracebility
- User Story Tracebility
- Test-related cards in Overview app
Test Case Assignment to Test Plans
Advanced test case selection
- with additional filters in the Test Plan app
- by levaraging the Traceability apps to select related test cases for assignment to a test plan
Tester Assignment:
Assign testers to test cases
- Default assignment without test plan
- Dedicated assignment per test plan
Test Sequences:
Define the sequences of test cases within a test plan
