Locks and Concurrency Handling using ETags in SAP Netweaver Gateways
Introduction
The concurrency control in SAP Netweaver Gateways through OData is not well articulated and since it is fundamentals to businesses and workflows, in general, there is enough room to explore and implement such concepts. Concurrent editing (two users editing one resource at the same time) often leads to lost update problems. This article explores the know-how of handlings OData through Entity Tags (henceforth termed ‘Etags’ in this document).
Literally: con·cur·rence, /kənˈkərəns/
noun: concurrency
meaning: ‘the fact of two or more events or circumstances happening or existing at the same time’
technical: multiple users can update the same record at the same time causing confusion among the users and inconsistency in the expected output. This issue is coined as concurrency.
Concurrency Control
- Pessimistic concurrency control
In this scenario the application blocks until all the operations within the transaction have been successfully completed. This will normally involve a performance reduction, as the application(s) using the resource will all have to queue until the transaction can ensure the operations can successfully be applied. This scenario also has a much higher risk of deadlocks as the transaction may be waiting for a resource that is locked by another operation. It is pessimistic since it assumes that concurrent writes will occur and actively prevents it by aggressively acquiring the resources.
- Optimistic concurrency control
Here the operations in the transaction are all attempted and when one of them fails the entire set of
Operations are rolled back. This is normally the preferred solution due to its higher transaction throughput and less possibility of deadlocks. It is optimistic since it assumes the likelihood of a concurrent write is rare (though it does happen) and so all the operations in the transaction are attempted and rolled back if one fails.
- Semi-optimistic concurrency control
This is a blend of the pessimistic and optimistic concurrency controls so that it is possible to get higher throughput by managing the resource locks individually. These are normally a highly curated solution and for very specific use cases when scenarios are identified that are environmentally specific.
Concurrency Control in HTTP
Based on the above explanation it should be evident that HTTP can only use optimistic concurrency control.
Because each HTTP transaction is stateless.
For example, Lost update problem with concurrency control is illustrated below:
- User_A accesses a document using HTTP GET and starts editing it.
- User_B also accesses the same document again using HTTP GET and starts editing it.
- User_A saves their edits using HTTP PUT.
- User_B saves their edits but as user B didn’t see User_A’s edits,
- User_A then loses their edits as User_B overwrites them.
Detailed example
User_A and User_B both issues GET request to the server for the document X with both receiving the same Etag in the response field.
User_A edits and does not save changes (i.e.) does not send a PUT request to the server.
User_B edits and sends PUT request with If-Match field value as Etag value.
Since the Etag value remains same as the server, the status code 204 is received, then update of the content is performed and new entity tag is received through response. So old Etag is deleted and the new one is stored by User_B for future requests.
At the same time, User_A sends PUT request with If-Match field value as Etag value. Since the Etag value is changed (due to the action of User_B), status code 428 with preconditions failed is received as the response.
To get the updated version of document X, User_A send PUT request with If-None-Match field value as Etag value. So now the existing version of User_A’s document is overridden and new Etag which User_B has will be received in response with the resource. Now User_A can change the document and send PUT request with If-Match field value as Etag value. Now the changes will be reflected on the server.
Therefore by using entity tag along with If-Match and If-None-Match fields, concurrency is managed in HTTP transaction.
Procedure to Implement
This is what developers would be more interested in. ? We can implement property based Etags or Entity based Etags. In practical scenarios, we find Entity based Etags more often compared to property based Etags. This example will evolve with Entity based Etags.
Step – I: Create the Entity Structure
To start with, we need to create a structure which will hold the ‘Etag’ as a property. Etag essentially is a hash value which is generated depending on the value of the entity data. To accommodate the Etag, we enhanced the SCARR structure with Etag field.
Step – II: Create the Gateway Service Entity Type
Create your gateway Entity with the ZSCARR structure and generate your runtime artifacts to work with. My gateway service consists of SCARR and SPFLI both the structures although our scope of Etags will be restricted but not limited to SCARR entity type.
The image below explains the properties, navigations, associations built for the service taken for this example. Note ETag property of the ZSCARR Entity has been mapped on the ‘Etag’ field of the entity. This helps to appear the Etag in the metadata section of the get responses.
Your SCARR entity should look like below, once you are done with the additions.
Step – III: Create the ETag generation method
Now that you have generated the runtime artifacts, create one more method to calculate the hash value of the data record passed. I named the method – CALCULATE_HASH. The importing and exporting parameters are shown in the image.
Step – IV: Coding of the CALCULATE_HASH method
Let’s code the method. We will use the ‘CALCULATE_HASH_FOR_RAW’ function module to generate the hash value for a record. The FM output will be a hash value which will be treated as an Etag.
Step – V: Make changes to GET_ENTITY and GET_ENTITYSET methods
We need to remember, every time a resource is requested (GET) the hash value is calculated and passed back as response. Only if the entity data is changed, the hash value will be changed resulting mismatch in Etag. We will make necessary changes to accommodate the hash values in SCARRSET_GET_ENTITY and SCARRSET_GET_ENTITYSET of SCARR as we want the SCARR to be updated through the Etag validation.
Also, make changes in SCARRSET_GET_ENTITY.
Step – VI: Update a Record with Etag
Let’s test how the Etag is working in data record retrieve scenario. I am taking CARRID = ‘AS’ and trying to retrieve the data. The image below shows how the data record is in DB now.
Let’s run a SCARRSET_GET_ENTITY with ‘AS’ as the primary key. This should return us Etag in metadata section and Etag as the field of the SCARR entity.
Note – Highlighted entry in the header is ‘Etag’. You should also find “etag” in metadata section but it differs from the Header Etag. That is because of the “(apostrophe) used in the metadata. Apostrophes are handled with ( \” ) notation. To avoid confusion and manual alterations take the Etag value from response header.
Step – VII: Update using Session 1
This Etag represents the present data record against key = ‘AS’. Any alteration/modification will cause a change of Etag. In our update, we will use the present Etag to match the data record and update it. We use HTTP header “If-Match” to match an Etag value for a record and process further. If this Etag is mismatched then the update will fail. In Session 1, I will update with present Etag.
The image below shows my request. Note the header Etag.
In debugger we can check the request is navigating through GET. In GET, the Etag is generated once more to compare at the time of update.
Now control is navigated to “Update” method.
Once the update is successful the record will look like below –
Step – VIII: Update using Session 2
Run a separate update scenario at the same time with Session 2. Note, our initial record Etag will be used here even, since the 2nd session user is not aware of the fact that the record is updated meanwhile.
Control should first go to the SCARRSET_GET_ENTITY to retrieve the Etag for the data record.
Since the record is already updated, observe the change happened in ETAG value. This should stop the update to trigger and issue an error message.
Step – IX: Update scenario using “If-None-Match”
Like “If-Match” there exists another HTTP header “If-None-Match” which will check the Etag present first and proceed if there is a mismatch. Let’s update the same entity with this header.
Now the control should go to the UPDATE method and update the data record of ‘AS’ carrid.
Observe, the new update is successful now.
