Technical Articles
SAP Cloud Platform Backend service: Tutorial [17]: CDS: ETag
This blog is part of a series of tutorials explaining the usage of SAP Cloud Platform Backend service in detail.
Quicklinks:
Definition
Usage
Reference
If you’ve never heard about ETag: this is a good blog for you
If you’ve always wanted to know about these ETags: continue reading
If you’re familiar with ETags and only need to know the CDS syntax: do this hop
What is it, the ETag?
This name is so mystic…
But the meaning is quite down-to-earth
Abbreviation: entity tag
It is used for optimistic concurrency control
What is it, optimistic concurrency control?
It is not pessimistic concurrency control.
What is it, pessimistic concurrency control?
OK, in case of “pessimistic” we want to be 100% on the safe side.
Whenever a data entry in a backend (database) has to be modified, then first of all, a lock is applied.
Like that, only the current user (or application) is allowed to modify the data
Then the data is changed and safely written to the database.
Afterwards the lock is released.
This procedure is not very fast (and might cause deadlocks).
Especially, if the lock isn’t really required, because the resource is not likely to be accessed by multiple users at the same time.
Furthermore, an OData service is anyways “stateless”.
That’s why “pessimistic” doesn’t apply in our case, we have to look at “optimistic concurrency control”:
When a resource, an entity, is requested, it doesn’t get locked.
So it can be modified.
Nevertheless, it could be requested and modified by a second user at the same time.
If that occurs, changes can be overwritten or requests can fail, etc
To prevent such failure, we have the mighty ETag.
Being so great solution, it should be bold
To prevent such failure, we have the all-mighty ETAG.
The wonderful ETag provides concurrency control but doesn’t lock
With other words, the ETag indicates the version of an entity.
With even other words, the ETag is used for conditional requests
Why do we need it, the ETag ?
Because it prevents concurrent modifications, without overhead.
Example:
In a web application, a user opens the details of whatever
e.g. a product?
Yes, e.g. a product
The UI allows to edit, so the user changes some details, but doesn’t submit
At the same time a second users decides to delete that same product and submits the deletion.
When the first user finally decides to submit, the changed product doesn’t exist anymore, so a crash is unavoidable
So what now?
To prevent such failure, we have the all-mighty ETAG.
Nicely bold, but being so great solution, it should be colorful
To prevent such failure, we have the wonderful ETAG.
Who needs it, the ETag?
ETag is not mandatory for a resource requested via HTTP.
It is not required for services used in demo scenarios
But it should be used in productive APIs
How does it work, the ETag?
Being an electric tag, it works magically.
Kidding.
We have to add a tag to the entity.
That tag can be anything.
For example an ETag?
Whenever an entity is changed, the tag is changed as well.
Furthermore, the HTTP protocol has few status codes reserved for ETag handling.
Example flow:
An OData service implementation decides to support ETag
Whenever such an entity is sent to the client (e.g. single GET), the ETag-header is sent
If the client sends a modification request (e.g. PATCH or DELETE) for that entity, it has to send the ETag value in a header called If-Match
Taking it literally: “I want to modify this entity, but only if the ETag matches the given value.”
As such, the service implementation can check if the modification request is allowed.
If the client doesn’t send that header, the server responds with an error Precondition required
If the ETag is sent, but doesn’t match (entity has changed in the meantime), then the error is Precondition failed
If this is the case, then the client fetches the entity with GET and takes the new ETag to send a valid modification request
Another example flow:
Until now we’ve talked about concurrency control, I guess this scenario is now clear.
Quite clear, thanks
There’s a second advantage which comes with ETag:
OData services are stateless, but nevertheless it would be an advantage to avoid redundant calls
Like this:
A client application calls an entity. A big entity.
Even a huge entity
The client reads and remembers the ETag
When the client app does another request to that entity, it will send that ETag
If the (huge) entity is NOT modified, then the server indicates it by returning Not Modified
And the good thing: the (huge) entity is NOT sent again.
Means I send the following wish to the server:
“Please send me that huge entity only, IF the current ETag does NOT MATCH the ETag which I’m sending”
This is the actual use case of conditional GET request.
We’ll go through these examples in the Test-section below
How is it expressed in CDS, the ETag?
An entity has one property which is marked as ETag-relevant by adding the annotation
@odata.etag
to that property
As such, whenever the entity changes, then this property will be changed as well.
Usually, such property can be a timestamp,indicating when it was last changed. Such ETag value can be automatically generated
How do we express our CDS, the ETag?w
Would be nice to see it in action
Sure, we’re going to verify it all in an example.
Note:
To make life easier, let’s use a version-property, a number which we manually change when we change the entity.
Below there will be a second example with timestamp
To create an example, I’ve once again thought of a Pet shop, because some pets are known to change their mind often, so if they liked their food today, it might turn to be absolutely uneatable the next day. So for such pets we need a version property…?
Example CDS:
service PetService {
entity Pets {
key id: UUID;
name : String;
species : String;
food : String;
@odata.etag
version: Integer;
}
}
Have you seen it, the mighty ETag?
Yes
How does it look in OData, the ETag?
Well, it looks like an Excellent Tag, the ETag.
Kidding.
The contract of an OData service is written in stone, it is the metadata document and it has to show if and how ETag is used.
Because a client application has to be aware of it.
As such, the property which is used for ETag, has to be flagged.
In OData V2, it looks like this:
The ETag-property doesn’t have an attribute called ETag, even not etag, not even et, the attribute is called: ConcurrencyMode=”Fixed”
In OData V4, it doesn’t look at all, currently, but I’m sure it will look as described in the OData specification, once it is supported
Our example:
Have you seen it, the mighty ETag?
Yes
How do we use it at runtime, the ETag?
With other words: how do we test it, the Endless Tag?
Kidding.
Preparation
If not already done, create an API with the above sample CDS
Go to the cockpit and enable the OData V2 checkbox
Scroll down to API REFERENCE and select OData V2 in the drop down field
Create a sample entry
(See here for a description of the cockpit test tool )
Example payload:
{
“name”: “kitty”,
“species”: “cat”,
“food”: “fish”,
“version”: 1
}
Test 1: PATCH request with ETag
First perform a single GET request to the created entry
Note:
Remember, the Guid-key in v2 has to be composed like this:
guid’dc2c61cb-a855-4958-a4c9-188a4f501357′
Scroll down to see the response headers and find the ETag value, sent by the server:
As expected, the ETag contains the value of the version property which we’ve given at creation: 1
Now try to modify the entry:
Go to the PATCH section, ignore the payload, enter the key like above
Note:
while pasting the guid, make sure you don’t add a blank, as it would be interpreted as invalid character and would cause the request to fail
Execute the request and look at the expected error message.
Like described above, the HTTP response status is 428 Precondition Required
Good.
Good?
Yes, if we create an ETag-enabled service, then we cannot be surprised if it behaves like an ETag-enabled service
OK, we MUST send the ETag header in the PATCH request
So please go ahead and enter the header…
ohhhh, I’ve searched one hour and still don’t know how to do
Yes, OK, OK, that was mean….. the cockpit doesn’t allow to enter custom headers.
But don’t be frustrated, luckily you have this great series of tutorials, you can go to this blog to learn how to send requests from REST client to Backend service API
Read it, like it, and follow the explanation to call the service from external REST client
Oh, so much work
Yes, it is a bit tedious, but we’re happy that we can do it.
Actually, with Postman it is just few clicks to do
True.
OK, our service works as designed, so we need a REST client to send a PATCH request with custom headers
Now let’s go ahead
First execute a GET request in order to take a note of the ETag in the response-header
Example URL:
https://backend-service-api.<…>/odatav2/DEFAULT/PETSERVICE;v=1/Pets(guid’dc2c61cb-a855-4958-a4c9-188a4f501357′)
We knew that ETag already…
Anyways, take a note of it
Now compose the PATCH request
URL: the same
HTTP verb: PATCH
Request body:
{
“food” : “mouse”,
“version” : 2
}
Request headers (raw format, copy&paste ready):
Content-Type: application/json
If-Match: W/”1″
Execute the request and verify the Result:
We should get the correct 204 status code for PATCH request
And in the response-header section, we can see the new ETag (it is 2, like specified it in the request body)
See screenshot:
You can verify with another GET request:
The picky cat has now a different food: currently it prefers mouse over fish.
That’s life…
<Sigh>
Next steps:
Verify the different cases of scenarios with ETag.
Please don’t ask for a new screenshot every time
No?
No – I don’t want to take so many screenshots
Really?
Yes – only cats are allowed to be so picky….
Test 2: PATCH with invalid ETag
Send the same request like above, but this time with invalid ETag.
E.g.
Header:
If-Match: W/”99″
Result:
Status Code 412 Precondition Failed
Response Body:
Error message: The Data Services Request could not be completed due to a conflict with the current state of the resource
Response Headers:
no ETag
That error was expected, since we sent a wrong ETag
In a productive scenario this would mean that the client application would issue another GET request to obtain the newest ETag value. Then the user might need to re-thing his modifications to the data, because the data has changed in the meantime. Then the PATCH request would be re-sent with new ETag and potentially modified request body
Test 3: PATCH with *
Send the same request like above, but this time with ETag value as *, meaning everything is acceptable.
Header:
If-Match: *
Result: success
So if you’re too lazy to deal with correct values, e.g. in a test or demo scenario, then you can always just send a *
(If supported by the service)
Test 4: PATCH with no ETag
Same request, this time with no ETag at all
Result:
Status Code 428 Precondition Required
Response Body:
Error message: Precondition required
Response Headers:
no ETag
Test 5: GET with no ETag
Send a GET request to the same URL like above,
E.g.
https://backend-service-api.cfapps.eu10.hana.ondemand.com/odatav2/DEFAULT/PETSERVICE;v=1/Pets(guid’dc2c61b-18…8a47′)
Request headers:
No special header, no ETag header
Result:
Just normal result of single GET request
Response headers:
Contain the current ETag
Test 6: GET with If-Match
Send a GET request to the same URL like above
This time, specify an ETag header with valid ETag
E.g.
Request header:
If-Match:W/”2″
Result:
Since the condition is met, we receive a normal result of a normal GET request, Status 200 and same ETag in the response headers
Test 7: GET with invalid ETag
GET request to same URL like above, this time with invalid ETag
E.g.:
If-Match:W/”99″
Result:
As expected: 412
Test 8: GET with If-None-Match, invalidly-valid
A GET request with if-none-match is the actual use case of conditional GET request, as mentioned in the introduction
Send GET request to the same URL like above, this time with If-None-Match and invalid ETag
E.g.:
Header:
If-None-Match: W/”99″
Result:
Normal result like a normal GET request,
Status 200,
response payload in body,
valid ETag in response header
Test 9 : GET with If-None-Match, validly-invalid
Send GET request to the same URL like above, this time with If-None-Match and currently valid ETag
E.g.:
Header:
If-None-Match:W/”2″
Result:
NOT normal result like a normal GET request,
Status 304 Not Modified
Response body:
<empty>
Response header:
Valid ETag
This is the interesting scenario:
We can see that NO payload is sent.
Good, because it is anyways not modified
The client is informed about it: via the HTTP Status Code 304 Not Modified
Means, the client would not replace the current data in the UI-Table with the new – empty – payload.
No, no that would be wrong, obviously
But since the client does always interpret the response status code, everything’s good:
The UI remains unchanged
Exactly
Note:
Since the response status is not an error status code, the current ETag is sent along with the response
As you can see, this is the actual use case of conditional GET request:
Fetch data only if it has changed.
With other words:
If the data has NOT changed, we don’t need to fetch it.
Save time.
Save network resources
Save money
Save the planet
As a consequence:
More time for playing with cat
More bandwidth for cat videos
More money to buy fish (or mice)
More greenery to entertain cats
Test 10: DELETE
Nothing new, to be short:
- With no ETag:
All the same, the result is as expected, 428 - With invalid ETag:
All the same, the result is as expected, 412 - With valid ETag:
Deletion successful, status 204 as expected, no ETag in the response, obviously - With If-None-Match and valid Etag
412 - With If-None-Match and invalid Etag:
Successful deletion
Test 11: PATCH with If-None-Match
- If-None-Match and valid ETag: 412
- If-None-Match and invalid ETag: success
- If-None-Match with value null: success
- If-None-Match with value * : 412
Test 12: GET ALL
At the end, let’s have a look at a new case:
As result of a QUERY operation, we get a list, so all the ETags cannot be sent as header
As such, they are included in the payload
Try it:
URL:
https://backend-service-api.cfapps.eu10.hana.ondemand.com/odatav2/DEFAULT/PETSERVICE;v=1/Pets
Response:
<entry m:etag=”W/”2″”>
<id>https://<..>-backend-<…>/odatav2/DEF/SRV/Pets(guid’00…’)</id>
<title type=”text”>Pets</title>
…
Are we done with it, the ETag?
No, we have missed a few scenarios with the Exhausting Tag…
No, please not, I’m tired…
OK OK, I was kidding, don’t worry, we’ve gone through all relevant scenarios…
Thanks that sounds better…pouf
For me it has just started being great fun….
Sorry, I’m a bit picky sometimes….
Sometimes?
BTW, why do we write a W/ in the ETag value?
Not tired anymore?
No, not at all, why tired???
Sigh…
What is the syntax for it, for the ETag?
There are 2 possibilities:
- The ETag value with quotes:
“2” - The ETag value with quotes and preceeding W/:
W/”2”
Why are there 2 of them, of the ETags?
One is the so-called “weak” ETag. As the name indicates, it is less strict.
I’m a bit tired, so let me quote the OData spec:
“As OData allows multiple formats for representing the same structured information, services SHOULD use weak ETags that only depend on the representation-independent entity state. A strong ETag MUST change whenever the representation of an entity changes, so it has to depend on the Content-Type, the Content-Encoding, and potentially other characteristics of the response.”
I don’t get it
Sigh…
The background is that data can be represented as xml or json. As such, the value of an ETag-property can look different. But still the content is the same.
Therefore it makes sense to use weak ETags in OData
(See your next question for an example)
You can try a strong ETag as follows:
If-Match: “3”
How does a more meaningful ETag look like, the ETag?
A still very simple example, but this time using a meaningful ETag property:
Data type timestamp for a property like “last-modified”, “changed-at” or similar:
service PetService {
entity Pets {
key id: UUID;
name : String;
species : String;
@(odata.etag)
modifiedAt : DateTime;
}
}
Note:
Alternative notation: @(odata.etag)
It allows to add multiple annotations between the brackets
At runtime, do single GET request and try with different representations, by adding $format=json or $format=xml, which is the default in OData V2
You see that the value of the property “modifiedAt” looks different in json, compared to xml
However, have a look at the ETag header: it is the same in both cases
Try different requests
E.G.
Weak:
If-Match: W/”2019-05-15T15:51:04Z”
Strict
If-Match: “2019-05-15T15:51:04Z”
Change a second
If-Match: W/”2019-05-15T15:51:05Z”
Now try all 12 tests like before
Oh… I’m too tired
Hehe…
Summary
An ETag is like little version information, attached to an entity
More concrete, a property of an entity definition is annotated, to mark it as ETAg relevant
Whenever the value of this property changes, it means that the complete entity is different
When doing service requests, the value of ETag is sent in a header
With the help of ETag,
- a client can make sure not to fetch data which he already has
- a server can make sure that 2 clients don’t do concurrent modifications to an entity
Links
ETag in HTTP/1.1 specification
ETag in the OData specification:
Data modification with ETag
Response header with ETag
Request header If-Match
Request header If-None-Match
Optimistic Concurrency Control in Wikipedia
SAP Help Portal: ETag definition in CDS
Appendix 1: Quick Reference
CDS:
@odata.etag
Runtime:
The ETag value is taken from response header of a GET request
Required header names:
If-Match
If-None-Match
Workaround: Use * to avoid entering correct ETag
Appendix 2: The pet, the picky
Hi Carlos ,
I am working with RAP, i get the error- Precondition required for action in entity CDS~YI_BOOKING_XXX - use "If-Match" header
on clicking my action button what could be the reason please help
Hi Former Member
Your service seems to use ETag for your action. Reason would be that an action can modify a resource, like a PATCH request does (or DELETE, etc)
Using ETag, your service tries to avoid concurrency conflicts.
As such, you have to ensure that you're modifying the current version of the resource.
For that, you have to specify the ETag, when sending the request.
First, you have to obtain the ETag. The ETag is sent with every GET request (e.g. when doing a GET for the entity, on which your action operates).
You have to read the ETag from the response payload, then specify it in your POST request, when executing the action.
"Specify it in the POST request": this is done in the header.
The name of the header: "If-Match"
The value of the header: the ETag
That's it.
I thought I had explained it in this tutorial. Can you please let me know where I was unclear, so I can improve the explanations.
Cheers,
Carlos
Thanks Carlos.. I made the mistake with etag and after hours of debugging 🙂 I could fix it but was worth it.
I need another help from you if you may please- how will this model fit with the free style ui5 dev?
Any input on that and so behavior implementation is the new way for ODATA - CRUD and Action
Regards,
Prasenjit
Hello Former Member
I assume using etag is default in enterprise applications and Ui5 seems to have some out-of-the-bos support (see for example here and here)
Anyways, it is just a property with string-value and a header which need to be handled, so you can always do it manually
Kind Regards,
Carlos
Thanks Carlos.
HI,
I have added the eTag in the behavior class, like belows:
implementation unmanaged in class /SCWM/CL_BIL_API_WM_PI_LEAN unique; // implementation in class bp_a_whsephysinventoryitemlean unique;
define behavior for A_WHSEPHYSINVENTORYITEMLEAN implementation in class /SCWM/CL_BIL_API_WM_PI_LEAN unique
//lock master
etag PInvLogItemTimeStamp
{
action DeletePhysicalInventoryItem;
association _WhsePhysicalInventoryCntItem {}
update;
}
However in the metadata, for property PInvLogItemTimeStamp, the expected ConcurrencyMode is not shown, so through get request, I can't see the etag in __metadata, can you please help checking it?
Br,
Kimi
Hello Kimi, I'm sorry, I cannot help you here, as I'm not familiar with abap