Central monitoring of the entire IT landscape covering cloud-based applications and legacy on-premise solutions is more and more crucial today to oversee and react on business-critical incidents. Within this blog I will explain how SAP Cloud ALM and especially the Integration & Exception Monitoring application of this platform helps business and integration experts in their daily work. The blog provides a deeper insight into the integration and exception monitoring of the Cloud Integration capability of SAP Integration Suite. You will learn
- some background about SAP Cloud ALM and Integration & Exception Monitoring
- how to monitor end-to-end integration scenarios and drill-down from high-level integrationscenario to single message details for an analysis and navigate forward to the local monitoring tools for issue resolution
- how to monitor deployment exceptions and see the detailed error pattern
- how to track messages based on business or technical attributes
- how to use the embedded alert inbox to collaborate with colleagues during troubleshooting
- how to configure alerts for failed messages and deployment exceptions
- how to add integration scenarios to get them monitored
What is SAP Cloud ALM?
SAP Cloud ALM is the newest tool in the Application Lifecycle Management (ALM) portfolio and addressed to the needs of cloud-centric landscapes. It extends the classical platforms that are in place like SAP Solution Manager and SAP Focused Run. Both are focusing on customers with a big footprint in on-premise landscapes. It is included in your subscription with SAP Enterprise Support, cloud editions.
SAP Cloud ALM is a fully integrated ALM suite, cost-efficient offering built natively on the SAP Business Technology Platform. It has different use cases in scope, starting from business process monitoring down to IT infrastructure monitoring. Important use cases are
- Integration & Exception Monitoring focusing on integration flows and integration error situations, and
- Health Monitoring focusing on functional aspects of cloud services.
The Integration & Exception Monitoring application offers one central monitoring and alerting tool that spans the entire landscape and brings the technical message flow closer to the business context. It provides end-to-end monitoring, covering integration scenarios, and focuses on exceptional and error situations. It reports integration related exceptions to provide an overview where exceptions occur in your entire landscape and react on them. Additionally, you have the option to watch embedded alerts, start an operation flow, jump to your local monitoring for further investigations and do the appropriate actions.
As of today, Integration & Exception Monitoring covers the following functionality that all together support the problem resolution process efficiently:
- Message monitoring to visualize the healthiness check of your components in scope. You may drill down to individual messages, jump to the local monitoring tool, and solve an issue if required
- Tracking is more or less for on-demand monitoring to identify business-driven problems and search for a special message of interest
- Alerting is a kind of a proactive monitoring on failed messages or exception with ability to collaborate with responsible persons in business and IT
Monitor end-to-end integration scenarios
Integration owners have to know the status of certain business scenarios but also of incoming and outgoing messages of Cloud Integration tenants. Both use cases are feasible with the scope selection functionality that offers you two different kinds of entry points to start the Integration & Exception Monitoring application. You can adapt the overview page as per your needs as the GIF below shows it. You can either have a service-specific view as for your Cloud Integration tenants, or a scenario-specific view to monitor integration scenarios such as Hire to Retire or a Lead Transfer scenario. To accomplish this, you can organize cloud service and on-premise systems in scenarios to visualize a relationship between the respective applications. Further scenarios can be added to the set of existing ones.
What you already can see from the GIF above is that Integration & Exception Monitoring provides you an overview across different services, might it be a specific integration scenario and selected services. You don’t have to go to the monitoring capabilities of each of your service instances in your landscape, you get the big picture of the message path and on erroneous situations directly from here.
Correlation of messages
Instead of only collecting messages from various services, single messages are correlated to each other to an end-to-end message flow across cloud services and on-premise systems. To accomplish this the SAP Passport mechanism is leveraged and in the message flow a unique correlator is mixed in. Correlation of integration artifacts provides end-to-end visibility in message flows across all involved cloud services and on-premise systems following a message sequence from a sender to all intended receivers. This closes the gap between business and IT during the issue resolution process while searching where a message gets stuck in a transaction flow.
Topology view
In the following I am using an integration scenario to understand key functionaliy of the integration monitoring tool. A lead transfer is created in the marketing service, the transfer triggered to Cloud Integration and send further to the sales service for replication. For this scenario a pre-packaged integration flow from the SAP API Business Hub is leveraged.
In the topology view, the entry point of the monitoring section, you see the involved services. It offers you an overview either on multiple services or on one scenario depending on the selected scope and a selected time period. Each node represents a service and depending on the status of the incoming and outgoing messages transferred or the occurred exceptions the node border changes the color. If any message is in an erroneous status or an exception exists, the color switches to red. Other colors visualize that there are messages in warning or transfer status, or no messages available in the selected time frame, or the data collection for the service is not activated.
Monitoring categories
Through selecting a specific node, you get the status details in transferred messages and business impacting exceptions. These monitoring categories are service specific, and the data collection can be switched on or off. For Cloud Integration the following categories are available as of today:
- Status and performance of single messages
- Exceptions as deployment exceptions for integration artifacts
Analyze a cloud service in the service summary view
By clicking on the message category (blue link) or on a specific status (colored blocks) you drill down from the high-level topology perspective to a next level of details, the summary view of a cloud service and the selected category. For now we dig around messages not on exceptions. This view comes along with relevant interface information related to the selected service and additionally remaining in the context of the selected scope (integration scenario or set of services) and chosen time interval. Messages are grouped relating on the service and category status you have selected. This view differs from service to service. In the following combined picture you see above a examplary service summary view for SAP S/4HANA Cloud and for Cloud Integration below. They provide a different way how to structure incoming and outgoing messages. Both examples are not within the context of an end-to-end scenario. Is there is any scenario or status is selected the view automatically shrinks the results appropiately. In addition, you may narrow down the number of results by defining the time frame more precise or using one of the multiple filter options.
Drill down to interface view
From this service summary view you can drill down one step further in your analysis process by clicking on the number of messages on the left or the highlighted number of errorneous or successful exchanged messages on the right. This interface view comes along with interface information related to the selected service again in the context of selected scope and chosen time frame. For Cloud Integration you see all collected messages for a specific artifact and you can filter for e.g. a specific message ID or a message status.
Analyze message details and navigate to local monitoring for issue resolution
By clicking on a message ID you jump to the single message details view. Here you can see a big difference to the local monitoring in Cloud Integration. By adding a unique correlation ID plus the predecessor to a message an end-to-end message path can be assembled following a message sequence from a sender to all intended receivers. You see the same message, a lead created in the marketing service, transferred to Cloud Integration and send further to SAP Cloud for Customer.
In the current scenario there is no confirmation send back to marketing. In general it is possible to see also the confirmation message. For each service selected in the colored message path you see the related messages along with status details and application data. You have also the possibility to navigate to the local monitoring.
In case of Cloud Integration you are jumping to the MPL attachment for further trouble shooting and taking actions for issue resolution.
Monitor deployment exceptions
In the following I want to focus on integration related exceptions that are also visualized in the topology view, the entry view of the monitoring section. Along with the transferred messages you see in the lower part of the monitored categories pop-up the configured exceptions in your selected scope and time frame. For S/4HANA Cloud the ABAP gateway and data replication exceptions show up. In case of the Cloud Integration capability deployment exceptions for any integration artifacts can be monitored.
In the next level of details, you see the service summary on exceptions, listing all exceptions together with the corresponding deployment status, the timestamp, the deployment message, and further information coming from the service. In the below picture you see an example for SAP S/4HANA Cloud in a scenario together with SAP SuccessFactors.
You can further drill down to you see the error pattern of an individual exception, along with relevant context information. In the example below you see a successful deployment. In case of a deployment failure, you have to go to the Cloud Integration tenant to react.
Please note that only exceptions that have occurred since the monitoring data collection has started are displayed.
Track messages based on business context attributes
Tracking is a kind of on-demand monitoring and allows to search single messages based on specific business context attributes like the order number or employee id, but also on technical attributes. As a prerequisite context information must be exposed by the relevant service and send together with the message processing log (MPL) to SAP Cloud ALM. For Cloud Integration these are
- Technical properties such as message ID, correlation ID (relates integration flows that belong to one execution)
- Artifact details such as name and ID, also package name and ID
- Business and payload related content if added to the MPL during runtime:
- Sender ID, Receiver ID, custom status, application message ID and type, set by means of a content modifier step
- Custom header properties (name value pair) containing business or payload-related information to be set by means of scripts
- Sender ID, Receiver ID, custom status, application message ID and type, set by means of a content modifier step
The search in SAP Cloud ALM is executed cross-component. The screenshot below shows a search on an application message ID that comes back with all entries from the different services along with the message path. You may navigate forward to the single message detail view, that we know already from the monitoring part, analyze the erroneous message end-to-end in the message path and jump to the local monitoring to react on the issue.
Embedded alerting on failed messages or exceptions
Alerting is a kind of a proactive monitoring to inform responsible persons in business and IT and to trigger corrective measures. If a message goes into an error an alert can be raised and the integration owner informed about the critical issues. This is important for getting awareness together with the possibility to trigger the necessary steps to resolve the issue in collaboration with other responsible persons.
In SAP Cloud ALM alert inboxes are embedded in all dedicated monitoring applications as Integration & Exception Monitoring or Health Monitoring. Therefore, you can’t find a tile for alert management as this is the case in SAP Focused Run. The configuration is specific for every monitoring application and therefore not maintained centrally.
The alert inbox provides you a list of open alerts for those components that are in your scope. By selecting an alert, you see the alert details with the possibility to check them and take appropriate actions. You can add comments, assign a processor, send a notification, or start an operation flow context sensitively. Additionally, you can have a history view to get a better understanding on the communication and actions that happened so far during the issue resolving process. And you can navigate to the single message details view to see the message together its message path for further analysis.
Configuring alerts for failed messages and deployment exceptions
In Integration & Exception Monitoring you may configure alerts as per your needs. In general, it is possible to create alerts for failed messages and for exceptions. As a prerequisite the data collections for the monitoring categories must be activated:
Navigating further you can adjust the alert definition or create new ones. With the adjustment of alerts, you configure them very precisely as per your needs. As an example, I have created a definition that creates alerts for failed messages regarding integration artifacts that contain ‘GRC’.
Select an alert name, these are more or less condition types when an alert has to be created. You can select one of the below. For Cloud Integration alerts can be created if
- Erroneous CPI Message Detected: An alert is created for each failed message
- Erroneous CPI Messages Detected: if one or more failed messages were detected since the last data collection. Several messages are aggregated.
- Erroneous Integration Artifact: if an artifact has been deployed unsuccessfully
- Erroneous Integration Artifact Stateful Alert Notification: Creates an alert for an artifact deployment alert notification
You may specify a display name and a description. The display is used in the embedded alert inbox. In addition, filters can be added to restrict the messages. The filter fields are service specific, and the values can be chosen from an input help. For Cloud Integration you can filter
- in case of failed messages on properties from the MPL
- In case of deployment failures on the artifact and the deployment status
Additionally, you can adjust the frequency how often you would like to receive alerts.
New created alerts are activated automatically. As soon as any message fails that matches the specified filters you will get an alert in your alert inbox.
Create a new integration scenario and setup the alerts
For adding an end-to-end business scenario to your list of watched ones, go to the configuration for scenarios screen. Just choose all involved services related to the scenario. As a prerequisite they need to be imported to the Landscape Management Service (LMS). In a second step filters can be added to each service to squeeze the number of monitored messages to only the relevant ones. Filter categories and parameter values for each filter are available from an input help that simplifies the setup. Additionally, alerts can be added to each filter.
Below you see an example for the replicate lead scenario we had before already. The marketing, sales, and Cloud Integration service are added together with the appropriate interfaces.
