Skip to Content
Technical Articles
Author's profile photo Toushif Ahmed

Solving Special Character Issue while sending data to Receiver System using HTTP Adapter

Introduction:

While sending data to target system using HTTP adapter, all the special characters in the data turns into  “?”.Languages like Hebrew, Turkish, Russian contains special characters and these data after sending to target system with HTTP receiver channel gets corrupted and does not reflect as expected. Reason behind this is the HTTP receiver adapter needs a byte array and uses the defined charset for the encoding (or ASCII if nothing is defined). Characters that do not exist in the used encoding are replaced by “?” signs.

 

Let’s follow some simple steps to overcome this issue:

 

We will define the required encoding in Content Modifier step. Add a new property in Exchange Property as below:

Action            Name                                  Type                          Value

Create           CamelCharsetName           Constant                    UTF-8

 

 

While providing payload in JSON structure, we provide in the below way:

Now, the issue is Hebrew most of the times contains a (“) Double Quote in between strings.

Example : **עובד כ”א זמני**–Here is one ” in between string. While sending payload in JSON structure this ” creates an issue. Because JSON payload always come in between a “<string>”.

Here, CPI accepts this as a double quote in between the JSON payload string that does not have an ending double quote. So, we need to escape this double quote. We will try to escape this using a Groovy Script.

Please find the Sample escape sequence in below code snippet:

 

The replaceAll() function escapes all the double quotes in strings and shapes the payload in a good structure.

 

By taking care of these simple steps we can avoid data corruption in target systems while using HTTP receiver channel.

 

Reference:

I found this KBA very helpful.

https://launchpad.support.sap.com/#/notes/2533303

 

 

Thanks 🙂

 

 

 

Assigned Tags

      13 Comments
      You must be Logged on to comment or reply to a post.
      Author's profile photo Iddo Rijsdijk
      Iddo Rijsdijk

      Hi Toushif Ahmed,

      Seems simple enough. I'm sending a JSON message containing the special character ë (Cliënt). I added the exchange property, and during the trace all looks OK right up to the point where the special character is sent to the receiving system. Then it changes into a ? (Cli?nt). I added the exchange profile to a content modifier shape right before the HTTP receiver, but it still messes up the character.

      Any ideas?

      Iddo

      Author's profile photo Toushif Ahmed
      Toushif Ahmed
      Blog Post Author

      Hey Iddo,

       

      Please add the exchange profile in the same content modifier where your request payload is in JSON format in the content modifier's body with Content-Type as "application/json" in the content modifier's header.

       

      Regards,

      Toushif.

      Author's profile photo Eduardo Marcelino
      Eduardo Marcelino

      Same problem here.

      Could you tell me what do you do to corret the encode ?

      Author's profile photo Toushif Ahmed
      Toushif Ahmed
      Blog Post Author

      Please add a Content Modifier before you insert data using HTTP adapter. Here in my case Payload in JSON is a content modifier.

      Now, in the content modifier’s Message Header section, add a new header as:

      Add the content encoding property in the Content Modifier’s exchange property as:

      and in the Message Body section, please add an expression like below:

      Hope the issue will be resolved.

      Kind Regards,

      Toushif.

      Author's profile photo Eduardo Marcelino
      Eduardo Marcelino

      No way, thats something wrong.

      Seems it loses the charset inside second HTTP step.

      error%20in%20second%20step%20of%20HTTP

      error in second step of HTTP

       

      Author's profile photo Toushif Ahmed
      Toushif Ahmed
      Blog Post Author

      Hi,

      There may be something wrong you are doing in the content modifier's configuration. Please check it again.

      And if there are two separate HTTP adapter for data insertion place two separate content modifiers with the same set of configuration.

      Another thing, please log the final payload to insert using a groovy script and place that groovy script just before the HTTP adapter.

       

      Regards,

      Toushif.

      Author's profile photo Eduardo Marcelino
      Eduardo Marcelino

      The message is ok in the first HTTP segment.

      The charset error occurs in the second segment of HTTP.

      Author's profile photo adrian di ruggiero
      adrian di ruggiero

      Hi!

      Did you find a way to solve this issue? I'm facing the same! It's very frustrating!

      Regards

      Adrian

      Author's profile photo Toushif Ahmed
      Toushif Ahmed
      Blog Post Author

      Hi Adrian,

      Is it solved ? or you are still having the issue.

      Please share a screenshot of your flow and let's see if I can help you anyway.

       

      Regards,

      Toushif.

      Author's profile photo adrian di ruggiero
      adrian di ruggiero

      Hi! Yes, I'm still having this issue. I'll prepare some screenshots to share and I'll post them here.

      Thanks!

      Author's profile photo Toushif Ahmed
      Toushif Ahmed
      Blog Post Author

      Please share the raw payload with me.

      Author's profile photo Eduardo Marcelino
      Eduardo Marcelino

      I'm sending from postman.

      <?xml version="1.0" encoding="utf-8"?>
      <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">
      <soap:Header></soap:Header>
          <soap:Body>
              <movies>
                  <movie>
                      <id>1</id>
                      <name>Ação</name>
                  </movie>
                  <movie>
                      <id>2</id>
                      <name>Animação</name>
                  </movie>
                  <movie>
                      <id>3</id>
                      <name>Aventura</name>
                  </movie>
                  <movie>
                      <id>4</id>
                      <name>Cinema de arte</name>
                  </movie>
                  <movie>
                      <id>5</id>
                      <name>Chanchada</name>
                  </movie>
                  <movie>
                      <id>6</id>
                      <name>Comédia</name>
                  </movie>
              </movies>
          </soap:Body>
      </soap:Envelope>
      Author's profile photo Toushif Ahmed
      Toushif Ahmed
      Blog Post Author

      Hi,

      Sorry for the delayed response. I am getting the payload properly encoded with the said configurations:

       

       

      P.S.: You can try without using a message mapping step after the SOAP input as the payload is in the message body, only the JSON conversion from it will work.

      Regards,

      Toushif.