1. Why testing is increasingly important
Quality software assurance is an essential aspect in todays ever accelerating business process world. Order and delivery processes are integrated on vendors and customers IT sites and are more and more just in time to decrease storage cost and improve production speed. Major IT failures add up to millions and billions when business processes get interrupted. As a conclusion everybody needs to test to insure stability of the most important business processes. According to Cap Gemini’s world quality report (2019/20) the spending on software development process steadily grows whereas spending on human resources slowly increase and 23 – 28% of the development budget is allocated in testing:
On the other hand still most companies using ERP software in their daily business perform the necessary testing manually. Testing and tasks may be supported by Excel sheets with test data and instructions but are mostly done with high redundancy or meeting only 10-30% of the relevant, most-at-risk business processes.
Manual tests are very costly and time intensive and bind human resources to non-business tasks – losing pace in the worldwide competition.
Modern test automation tools can help significantly to improve this process and make testing much more efficient and effective at the same time.
2. A new Partnership in Testautomation
In autumn 2020 SAP decided to switch its SOLEX (Solution Extension) Partnership from Micro Focus to Tricentis in the area of software testing. The Tricentis solution portfolio has proven state-of-the-art modern test automation with high focus on efficiency and effectiveness in the testing process. In the last years Tricentis has always shown best positions in Gartners and Forresters magic quadrant and at the same time continuously grown their customer base worldwide.
SAPs top reasons to choose Tricentis as its Official testing solution are depicted in the following picture:
Many customers of the top10 market leaders in business branches like Banking, Financial Services, Energy/Utilities, Insurance, Telco’s and Global Services rely on Tricentis testing solutions and customer base has grown over 2000.
Tricentis products are part of the SAP pricelist and the P&T product portfolio (with different or adjusted product names).
3. The Tricentis Product Portfolio
The Tricentis product portfolio is mainly consisting of three solutions:
Enterprise Continuous Testing (ECT) – (product name at Tricentis: Tosca)
Change Impact Analysis (CIA) – (product name at Tricentis: Livecompare)
Enterprise Performance Testing (EPT) – (product name at Tricentis: Neoload)
At SAP the Tricentis products appear with a different names on the pricelist and in some cases are variations or bundles of Tricentis products with SAP interfaces to SAP Solution Manager and its Testing Suite ‘Focused Build’ or SAP ALM Cloud. Thus the products are not always 1:1 the same but mainly consists of the respective Tricentis solutions.
These three columns of software testing are the essential part of all software quality issues.
ECT (Tosca) covers the aspect of functional testing: “Does it work?”. Tosca is the original flagship product of Tricentis and was developed on the black-box testing approach with model based test automation to achieve better stability and flexibility on software changes and test case maintenance.
CIA (Livecompare) covers the aspect of selecting the software parts that have to be tested on a special upgrade/update or hotfix to the SUT (System under test) and answers the question: “What to test?”
EPT (Neoload) covers the aspect of performance testing and puts the SUT under load (load test). It can emulate hundreds and thousands of users accessing the same service at the same time, answering the question: “Does it scale?”.
All three products are well integrated and firmly fit together in the continuous development process of modern software design:
4. Enterprise Continous Testing (ECT)
ECT (Tosca) is the Tricentis solution for functional testing. Tosca can be named as the flagship product of Tricentis and was developed from 2007 by Tricentis founder Wolfgang Platz around the so-called black-box testing approach with model based test automation. In comparison to the older script based approach the model based test automation has many advantages in maintenance effort for test cases and flexibility.
ECT covers several aspects of functional testing and can be seen as a modern test automation suite in continuous development and software delivery process.
Model based test automation
The model based test automation approach abstracts from the implementation of the SUT by introducing a (simplified) model to test its behavior. In most cases the model can be derived from examining the behavior of the SUT namely from its GUI and can automatically be extracted by ECT via ‘ARA’ the Automated Recording Assistant in Tosca. In some cases the derived model has to be adjusted by the tester to adopt it to special testing aspects (improving the model). The testing approach itself is a “black-box testing” – meaning it is not directly related (and thus depending) on the actual code implementation of the SUT, but observes the behavior on the HMI (Human-Machine-Interface) or GUI (Graphical User Interface). So it detects which data goes into the machine and what results are coming from the machine. Other aspects happening in the coding of the software are not relevant for black-box testing approach.
Model based test automation must not be misunderstood or mixed up with model based testing, which is something completely different. Model based testing means a testing approach where a special (in most cases flow-model) model is derived from the software under test, which is simplified and can be better analyzed in the following process.
The model based test automation approach has lots of advantages over traditional approaches like scipt-based testing:
First of all it makes testing more easy for non-technical personal as it doesn’t require coding or scripting. It is also much more flexible to small program changes in the SUT as it abstracts from the implementation layer to a business layer. And it can be more easily adopted to changes in the SUT and the adopted models can easily be propagated across all depending test cases.
Thus model based test automation is more efficient and reusable compared to other techniques like scripting. It also can achieve high test automation rates of 90% and more.
Risk-based testcase design
Another important aspect of ECT is the risk-based testcase design. The focus of software testing should be on the most important and most-at-risk business processes so that time and resources are used efficiently when testing. This means that the parts of the SUT that should be tested have to be chosen according to their business importance and the risk of change under upgrades or updates of the SUT.
Also the test data has to be chosen and selected with respect to risk-based testcase design.
The variation in test data should be as complete as the underlying business process (resp. test model) demands but as compact as possible to perform the task without redundancy.
Tricentis Tosca (ECT) and Livecompare (CIA) play together in this aspect and provide a very efficient and cost effective approach to test automation with respect to most-at-risk business processes.
Three aspects are relevant and are mixed for optimization in risk-based testcase design:
- What to test (CIA derives the most-at-risk processes that should be tested)
- Manual adoption to introduce business importance to ECT/Tosca
- Test data selection and composition according to risk-based and efficient testing
Test data Management
Another important aspect of ECT/Tosca is the included Test data Management (TDM). More precisely this part consists of TDM (Test data management) and TDS (Test data Service). The TDS allows you to easily manage, store, and keep data stateful throughout your end-to-end testing.
TDM additionally allows you to mix test data in advanced combinatorics and generate synthetic test data to allow for most efficient testing with less duplicate testing. TDM can use and mix actual test data recorded from manual transaction processes and synthetically generated data to be most beneficial and efficient in the test automation process with high re-usability rates, thus covering most-at-risk business processes.
Orchestrated Service Virtualization (OSV)
With OSV you will be able to simulate services in the testing process which might be rarely available for testing in real life due to server outages or involved business impact. OSV is capable of emulating cloud applications and services that might need an external service provider and would put a performance impact on their machines. Thus OSV helps testing services that would otherwise hardly be available for testing.
ECT Integration in SAP Solution Manager, Special Offering
ECT integrates with SAP Solution Manager Test Suite and Focused Build in Solution Manager.
ECT for SAP comes in two ‘flavors’: the full ECT (functionality as in Tosca) or Tricentis Test Automation (TTA) only. TTA is offered free of license charge to SAP customers with Enterprise Support and can be used together with the Test Suite in SAP Solutions Manager to achieve modern test automation. Thus it can replace the CBTA (Component Based Test Automation) in Focused Build, but it needs the Solution Manager to integrate in the overall testing process with test case documentation and results as well as test data management.
The following picture describes the components and dependencies:
The components in grey will be only available in ECT, whereas the part in blue is TTA. For the full testing functionality SAP Solution Manager is needed with TTA to make up for the ‘missing parts’. TTA can only be used for SAP testing, whereas ECT can also be used for all other applications thus allowing for end-to-end tests consisting of multiple technologies (e.g. results are shown in PDF or Excel or an external web application.
ECT and TTA are integrated in SAP Solution Manager and ALM Cloud (the last planned for end of 2021).
Advice for TTA usage:
TTA is meant for SAP customers that already use the Test Suite in SAP Solution Manager and generally want to stick with that solution but would like to try the test automation in ECT. ECT on the other hand is the more complete bundle with respect to Test Data Management and Provisioning and all the other aspects already mentioned in ECT (grey parts in the picture). A customer can try TTA and switch to ECT later on without losses or redundant tasks.
Nevertheless the workflow will change form SAP Solution Manager with Focused Build + TTA to ECT as many integration parts from TTA to the Solution Manager are no longer needed. For a customer that aims on using the whole test solution in ECT it is better to switch to ECT directly not to lose time on the integration aspect including education of personal.
5. Change Impact Analysis (CIA)
The Change Impact Analysis (Tricentis: LiveCompare) is an external tool acquired by Tricentis in 2019 that will analyze Software Updates/Upgrades or Supportpackages and Hotfixes with respect of most-at-risk processes thus answering the question: “What to test”. In this form it delivers important information to the risk-based testcase design and improves overall efficiency and effectiveness in the testing process. It’s a modern application based on AI and will compare software code side-by-side to derive what parts have changed and have to be tested before upgrading a SUT in production. It integrates well with ECT/Tosca while providing a fit/gap analysis of test cases and test data to reveal where there are gaps in the test case repository to cover all processes at risk.
- CIA in many cases can reduce the testing scope and effort by 85% with smart impact analysis that identifies existing test coverage and gaps.
- Increase speed by 3X to market by identifying all the differences that impact business processes, system integrations, custom code, security and governance.
- Release with limited to ZERO defects in production by confidently knowing which custom changes will impact production.
With CIA you will be able to concentrate on the most-at-risk business processes and reducing redundancy in test cases significantly. Especially in repeating test cycles during development waves you want to focus on the change and the most important part to be fast and agile – CIA is your tool of choice:
The so-called “treemap” analysis of CIA will show you at first glance which processes and modules are impacted by any change and which of them are most-at-risk, so you can focus on these parts to perform a very efficient testing:
The blue squares depict the processes that are changed and are most-at-risk. The areas in dark grey are also affected by change but are not as important to your daily business, so they deserve less priority in testing. The areas in light grey are not affected at all and don’t need any testing.
As CIA is integrated into ECT you will get a so-called fit-gap analysis of your testcase repository to show which tests already contribute to the testing task in scope and where there are gaps in the testcase repository which have to be filled to cover all processes most-at-risk (blue areas):
The necessary testcases are automatically identified and put into the requirements section in ECT’s Tosca-Commander, the cockpit of ECT for test automation.
Furthermore you can analyze all aspects of the CIA-findings in deep detail, from application area, transaction level, usage frequency, level depth in call-structure and even down to codeline comparison:
6. Enterprise Performance Testing (EPT)
In April 2021 Tricentis acquired Neotys which is the leading company in Enterprise Performance Testing Solutions. Neotys product for load and performance testing is Neoload and will further on replace the so-called “flood”, which used to be Tricentis solution for load testing. On SAP pricelist the product will be offered as Enterprise Performance Testing (EPT) and will be well integrated in ECT.
In general performance testing is as important as functional testing as it will be able to detect software engineering problems already in early development state and will contribute together with ECT/functional testing to overall software stability and reliability. Performance problems even in possibly less-important parts can rapidly add-up to significant amounts thus making a solution unusable for the thought intent or at least less usable and less cost efficient. Functional testing and performance testing should interact together to get the best out of both testing disciplines and make the backbone of agile enterprise continuous development process.
6.1 Key benefits of Enterprise Performance Testing
With EPT you can perform automated API testing with bottleneck identification. Testcases generated in ECT/Tosca can directly be propagated to EPT for performance testing with a single Click. The technology used by EPT fits well for SAP Applications and can be used on the whole SAP technology stack on premise and in the cloud.
Integration between ECT/Tosca and EPT/Neoload is tight: whenever testcases in ECT are changed they also will update on EPT if used as performance test:
EPT comes with a modern Dashboard to show all performance parameters and results at a glance. You can drill down to detailed performance results as needed.
EPT / Neoload Architecture
You can deploy EPT central components in the cloud or on premise but there are client components as well that need to be installed on the frontend for the end-user access to performance testing. The load generators can be deployed on premise or in the cloud. Neotys offers its own cloud service (Neotys cloud) for the central components which can be used without extra cost (included in the license). There is a limitation in cloud usage according to licensed amount of load test users.
7. S/4HANA Migration with Tricentis Support
For most customers the transition to SAP S/4HANA is a challenging task demanding lots of expertise in business processes as well as technical aspects. Business processes managed by SAP run across the whole organization, and even out to other organizations, such as customers, suppliers, or partners. An interruption to any SAP process could potentially have disastrous consequences for everyone involved. Maintaining that business continuity presents a number of significant challenges, both from the business’s perspective and from a technical perspective.
Although the SAP S/4HANA solutions are carefully tested there are still several situations remaining for the customer to test himself:
- Custom Coding
- Third-party Add-ons
- Interfaces
- Solution Extensions
- Native HANA developments
- Integration to web services
It is important to make sure business will run flawlessly in SAP S/4HANA once the transition is done and goes life. Systems don’t stand alone but communicate widely in the corporation and with other partners and suppliers outside. Especially web based services become more and more important and it is obvious that quality assured processes and connections are essential. Tricentis Service Virtualization can help identifying bottlenecks in web connections and multi-customer access as well as analyzing any performance issues.
Furthermore functional and performance tests are important on any interface and connection from and to the SAP S/4HANA system to ensure steady business and reliability. You want to test and quality assure your transition processes as early as possible in each phase to keep costs down and detect issues as early as possible.
