SAP Cloud ALM: Next Generation -Data Collection Infrastructure

Introduction

The Next Generation – Data Collection Infrastructure available with SAP Cloud ALM provides a standardized way to collect monitoring data using the OpenTelemetry instrumentation approach and protocol. With this approach, we reduce the instrumentation and operations efforts for the Cloud LoBs to the minimum. The observability data can be sent in a fire-and-forget manner, without the need for any persistence and with minimal integration efforts in SAP Cloud ALM. Currently, this approach is supported for Performance Monitoring, Integration Monitoring, Job Monitoring and Health Monitoring apps in SAP Cloud ALM.

SAP S/4HANA Cloud Private Edition Sales Certification Preparation Guide

Motivation

SAP Cloud ALM Run (CRUN) collects monitoring data for various use cases like Performance Monitoring, Integration Monitoring, Exception Monitoring etc., from Managed-Cloud services. This needs considerable instrumentation effort from the LoB applications and requires connectivity to be established by the customer.

At this juncture, Next Generation – Data Collection Infrastructure (NG-DCI) intervenes with an aim to:

• Reduce the Cloud LoB effort to integrate with CRUN
  
• Relieve customers from the task of establishing connectivity between the Managed-Cloud tenant and Cloud ALM tenant.
  
• Relieve Managed-Cloud (LoB) applications from having to maintain the configurations of tenant-specific connectivity between Cloud applications microservices and CALM instances
  
• Relieve multi-tenant applications and services from keeping up a state and using an unforeseeable number of resources (memory, CPU usage etc.)

Why use OpenTelemetry?

• OpenTelemetry (also referred to as OTel) is an open-source observability framework with a collection of tools, APIs, and SDKs.
  
• OTel is the new industry standard for adding observable instrumentation to cloud-native applications both inside and outside SAP.
  
• OTel enables us to instrument, generate, collect, and export telemetry data for analysis and to understand the application’s performance and behavior.
  
• OTel is vendor-agnostic instrumentation library (available per language) with support for both automatic and manual instrumentation.
  
• OTel makes observability effortless, inexpensive, and insightful.
  
• OTel is aligned with Cross Product Architecture vision for Logging & Monitoring and Technology Guidelines at SAP.

Concept and Architecture

NG-DCI has three core components, described along with their goals below:

1. Data Collection Instrumentation – Provide sufficient instrumentation support
   
   • Minimize instrumentation efforts for the customer and SAP
     
   • Use standard OpenTelemetry, whenever possible
     
   • Support all use cases in a standardized manner as much as possible. For example, connectivity handling and configuration exchange.
     
   • Enable end-to-end correlation by hybrid approach of SAP Passport and OpenTelemetry based correlation.
     
2. Data Collector Runtime – Provide unified data collection runtime approach
   
   • Unify runtime for push-based and pull-based data collection
     
   • Provide unified approach for customer’s and SAP internal use cases
     
   • Replace customer-managed connectivity with SAP-managed connectivity for SaaS scenarios.
     
   • Improve quality of data delivery by efficient queuing
     
3. Data Routing infrastructure in CALM – Provide unified approach for data routing
   
   • Enable dynamic routing of data to corresponding SAP Cloud ALM tenant across data centers.
     
   • Embrace openness regarding data exchange
     
   • Import data from third-party services or systems based on the APIs that are inbound to OpenTelemetry
     
   • Export data to third-party services or systems based on the APIs that are outbound from OpenTelemetry
     
   • Forward data to SAP Focused Run (FRUN)
     
   • Improve quality of data delivery by efficient queuing

Why NG-DCI?

Before NG-DCI	With NG-DCI
Tenant to tenant connectivity	Application to application connectivity
Customer managed connectivity and the need to manage multiple credentials.	Connectivity and authentication are managed by NG-DCI. No customer interaction required.
Multiple calls to SAP Cloud ALM (per use case)	Cross tenant calls managed by DCI (Data Collection Instrumentation).
Less efficient	Increased efficiency and scalability
High integration efforts	Reduced instrumentation and operations efforts
Multiple integration APIs exposed by SAP Cloud ALM use cases which result in various integration activities.	Standardized integration approach using OpenTelemetry