Sorting the Roman alphabet is easy, you learn it at school. The alphabet starts with an ‘a’ and ends with a ‘z’. If you have an unsorted list of roman letters you can imagine how it will look after you’ve sorted it. At the top of the list there are all ‘a’s, at the bottom there are the ‘z’s and in between there are all the other letters in a well defined sort order. But besides the 26 lower and 26 upper case letters there are several other special characters of the 7-bit ASCII which also need to be sorted in a standardized way.

And here the story begins. Please take into account, that the described effects only appear on Non-Unicode SAP systems, because in such systems the sort sequence from the operating system is used.

ABAP or ALV lists normally contain letters or numbers, e.g. family names or street names or material numbers and so on. But to a certain extend there are also some rows in such lists which do not contain alpha numeric values (which refers a-z, A-Z and 0-9 in this document) only but special characters like pound or asterisk. When you try to sort these rows you will get different results depending which operating system the SAP application server is running on. The sort happens on the application server and the results are sent to the GUI. The GUI is not performing any sort operations on lists. We now have different results on different operating systems. As already said, sorting rows which contain strings that do not start with an alpha numeric values are not very common **but** it may be important how the sorting of the special characters is done.

Let me give you an example: If your professional end users (you may call them ‘power users’) know that they have to scroll down the list for three pages to get to a certain data set which they want to access it may be confusing if the sorting suddenly changes and the data set isn’t there. Please keep in mind, we do not discuss list columns which are sorted by alpha numeric but by special characters.

What we haven’t discussed so far, what system configuration you have to have in place to see this phenomenon. There are two major examples that can expose the wrong sorting to the end user. I’m not giving any operating system names here. Using OS1 and OS2 as two different ones is enough to describe the situation:

1) You have your database, your SAP Central Instance and several SAP Application Servers running on OS1. You figure out, that you have to add additional SAP Application Servers to your system because of a performance bottleneck. The new SAP Application Servers run on OS2. Due to mixed log on groups users may end on an SAP Application Server running on OS1 on day one but on day two they connect to an SAP Application Server running on OS2. The users have different sort orders every day! You can imagine that the end users call the IT department frequently. This may result in reduced productivity of the end users and the administrators as they could get frustrated by time.

2) You have a homogeneous system environment. Over the weekend you do a migration from OS1 to OS2. On Monday the users are faced with a new sorting. This also leads to phone calls from the end user to the IT department. The positive aspect compared to situation 1) is, that the changed sorting only happened once, but it is still dissatisfying.

I forgot to mention, the most operating systems have in common that these special characters still list before the roman characters. You normally have a list with some special characters and then you have your standard letters and numbers. When saying “most operating systems” there are some, which ignore the special characters while sorting. These operating systems simply ignore special characters (as if they don’t exist) and sort only the “remaining” alpha numeric characters. This does have strange effects on such lists because with your eyes you see the special characters but the lists look completely unsorted. You wouldn’t recognize them as sorted because your eyes cannot simply ignore the special characters.

One operating system which ignores the special characters is Linux. Novell does provide adapted locales starting with SLES10 SP2. The sap-locale package contains a modified de_DE.iso88591 (named de_DE.iso88591@SAP_HP) locale which sorts the special characters based on the HP-UX sorting. Other locales are currently not part of the package and must be requested through the standard Novell support channels. The same counts for Red Hat Enterprise Linux. With Red Hat, customers have to book this service from Red Hat consulting. The SAP note describing the procedure for both distributors is 1069443.

In general, the sort sequence changes when switching from any OS to another one. The following table gives you an impression how the sorting of the printable 7-bit ASCII characters is handled. Additionally, the sort sequence according to the ASCII character number is also included.

Every field contains three elements. The first element is the special character, the second element is the decimal ASCII number the third element is the Hexadecimal ASCII number.

command: | sort | sort | sort | sort -d | sort -d |

ASCII |
Windows |
Linux |
Solaris |
AIX |
HP-UX |

32 20 | ‘ 39 27 | 0 48 30 | 32 20 | 32 20 | 32 20 |

! 33 21 | – 45 2D | { 123 7B | ` 96 60 | { 123 7B | { 123 7B |

” 34 22 | 32 20 | | 124 7C | ^ 94 5E | | 124 7C | | 124 7C |

# 35 23 | ! 33 21 | } 125 7D | _ 95 5F | } 125 7D | } 125 7D |

$ 36 24 | ” 34 22 | ~ 126 7E | – 45 2D | ~ 126 7E | ~ 126 7E |

% 37 25 | # 35 23 | 1 49 31 | , 44 2C | ! 33 21 | ! 33 21 |

& 38 26 | $ 36 24 | 2 50 32 | ; 59 3B | ” 34 22 | ” 34 22 |

‘ 39 27 | % 37 25 | 32 20 | : 58 3A | # 35 23 | # 35 23 |

( 40 28 | & 38 26 | ! 33 21 | ! 33 21 | $ 36 24 | $ 36 24 |

) 41 29 | ( 40 28 | ” 34 22 | ? 63 3F | % 37 25 | % 37 25 |

* 42 2A | ) 41 29 | 3 51 33 | . 46 2E | & 38 26 | & 38 26 |

+ 43 2B | * 42 2A | # 35 23 | ‘ 39 27 | ‘ 39 27 | ‘ 39 27 |

, 44 2C | , 44 2C | $ 36 24 | ” 34 22 | ( 40 28 | ( 40 28 |

– 45 2D | . 46 2E | % 37 25 | ( 40 28 | ) 41 29 | ) 41 29 |

. 46 2E | / 47 2F | & 38 26 | ) 41 29 | * 42 2A | * 42 2A |

/ 47 2F | : 58 3A | ‘ 39 27 | [ 91 5B | + 43 2B | + 43 2B |

0 48 30 | ; 59 3B | ( 40 28 | ] 93 5D | , 44 2C | , 44 2C |

1 49 31 | ? 63 3F | ) 41 29 | { 123 7B | – 45 2D | – 45 2D |

2 50 32 | @ 64 40 | * 42 2A | } 125 7D | . 46 2E | . 46 2E |

3 51 33 | [ 91 5B | + 43 2B | @ 64 40 | / 47 2F | / 47 2F |

4 52 34 | \ 92 5C | , 44 2C | * 42 2A | : 58 3A | : 58 3A |

5 53 35 | ] 93 5D | 4 52 34 | / 47 2F | ; 59 3B | ; 59 3B |

6 54 36 | ^ 94 5E | – 45 2D | \ 92 5C | < 60 3C | < 60 3C |

7 55 37 | _ 95 5F | . 46 2E | & 38 26 | = 61 3D | = 61 3D |

8 56 38 | ` 96 60 | / 47 2F | # 35 23 | > 62 3E | > 62 3E |

9 57 39 | { 123 7B | 5 53 35 | % 37 25 | ? 63 3F | ? 63 3F |

: 58 3A | | 124 7C | : 58 3A | + 43 2B | @ 64 40 | @ 64 40 |

; 59 3B | } 125 7D | ; 59 3B | < 60 3C | [ 91 5B | [ 91 5B |

< 60 3C | ~ 126 7E | < 60 3C | = 61 3D | \ 92 5C | \ 92 5C |

= 61 3D | + 43 2B | = 61 3D | > 62 3E | ] 93 5D | ] 93 5D |

> 62 3E | < 60 3C | > 62 3E | | 124 7C | ^ 94 5E | ^ 94 5E |

? 63 3F | = 61 3D | ? 63 3F | ~ 126 7E | _ 95 5F | _ 95 5F |

@ 64 40 | > 62 3E | @ 64 40 | $ 36 24 | ` 96 60 | ` 96 60 |

A 65 41 | 0 48 30 | 6 54 36 | 0 48 30 | 0 48 30 | 0 48 30 |

B 66 42 | 1 49 31 | 7 55 37 | 1 49 31 | 1 49 31 | 1 49 31 |

C 67 43 | 2 50 32 | 8 56 38 | 2 50 32 | 2 50 32 | 2 50 32 |

D 68 44 | 3 51 33 | [ 91 5B | 3 51 33 | 3 51 33 | 3 51 33 |

E 69 45 | 4 52 34 | \ 92 5C | 4 52 34 | 4 52 34 | 4 52 34 |

F 70 46 | 5 53 35 | ] 93 5D | 5 53 35 | 5 53 35 | 5 53 35 |

G 71 47 | 6 54 36 | ^ 94 5E | 6 54 36 | 6 54 36 | 6 54 36 |

H 72 48 | 7 55 37 | _ 95 5F | 7 55 37 | 7 55 37 | 7 55 37 |

I 73 49 | 8 56 38 | 9 57 39 | 8 56 38 | 8 56 38 | 8 56 38 |

J 74 4A | 9 57 39 | ` 96 60 | 9 57 39 | 9 57 39 | 9 57 39 |

K 75 4B | A 65 41 | A 65 41 | A 65 41 | A 65 41 | A 65 41 |

L 76 4C | a 97 61 | a 97 61 | a 97 61 | B 66 42 | B 66 42 |

M 77 4D | B 66 42 | B 66 42 | B 66 42 | C 67 43 | C 67 43 |

N 78 4E | b 98 62 | b 98 62 | b 98 62 | D 68 44 | D 6844 |

O 79 4F | C 67 43 | C 67 43 | C 67 43 | E 69 45 | E 69 45 |

P 80 50 | c 99 63 | c 99 63 | c 99 63 | F 70 46 | F 70 46 |

Q 81 51 | d 100 64 | d 100 64 | d 100 64 | G 71 47 | G 71 47 |

R 82 52 | D 68 44 | D 68 44 | D 68 44 | H 72 48 | H 72 48 |

S 83 53 | e 101 65 | e 101 65 | e 101 65 | I 73 49 | I 73 49 |

T 84 54 | E 69 45 | E 69 45 | E 69 45 | J 74 4A | J 74 4A |

U 85 55 | f 102 66 | f 102 66 | f 102 66 | K 75 4B | K 75 4B |

V 86 56 | F 70 46 | F 70 46 | F 70 46 | L 76 4C | L 76 4C |

W 87 57 | g 103 67 | g 103 67 | g 103 67 | M 77 4D | M 77 4D |

X 88 58 | G 71 47 | G 71 47 | G 71 47 | N 78 4E | N 78 4E |

Y 89 59 | h 104 68 | h 104 68 | h 104 68 | O 79 4F | O 79 4F |

Z 90 5A | H 72 48 | H 72 48 | H 72 48 | P 80 50 | P 80 50 |

[ 91 5B | i 105 69 | i 105 69 | i 105 69 | Q 81 51 | Q 81 51 |

\ 92 5C | I 73 49 | I 73 49 | I 73 49 | R 82 52 | R 82 52 |

] 93 5D | j 106 6A | j 106 6A | j 106 6A | S 83 53 | S 83 53 |

^ 94 5E | J 74 4A | J 74 4A | J 74 4A | T 84 54 | T 84 54 |

_ 95 5F | k 107 6B | k 107 6B | k 107 6B | U 85 55 | U 85 55 |

` 96 60 | K 75 4B | K 75 4B | K 75 4B | V 86 56 | V 86 56 |

a 97 61 | l 108 6C | l 108 6C | l 108 6C | W 87 57 | W 87 57 |

b 98 62 | L 76 4C | L 76 4C | L 76 4C | X 88 58 | X 88 58 |

c 99 63 | m 109 6D | m 109 6D | m 109 6D | Y 89 59 | Y 89 59 |

d 100 64 | M 77 4D | M 77 4D | M 77 4D | Z 90 5A | Z 90 5A |

e 101 65 | n 110 6E | n 110 6E | n 110 6E | a 97 61 | a 97 61 |

f 102 66 | N 78 4E | N 78 4E | N 78 4E | b 98 62 | b 98 62 |

g 103 67 | o 111 6F | o 111 6F | o 111 6F | c 99 63 | c 99 63 |

h 104 68 | O 79 4F | O 79 4F | O 79 4F | d 100 64 | d 100 64 |

i 105 69 | p 112 70 | p 112 70 | p 112 70 | e 101 65 | e 101 65 |

j 106 6A | P 80 50 | P 80 50 | P 80 50 | f 102 66 | f 102 66 |

k 107 6B | q 113 71 | q 113 71 | q 113 71 | g 103 67 | g 103 67 |

l 108 6C | Q 81 51 | Q 81 51 | Q 81 51 | h 104 68 | h 104 68 |

m 109 6D | r 114 72 | r 114 72 | r 114 72 | i 105 69 | i 105 69 |

n 110 6E | R 82 52 | R 82 52 | R 82 52 | j 106 6A | j 106 6A |

o 111 6F | s 115 73 | s 115 73 | s 115 73 | k 107 6B | k 107 6B |

p 112 70 | S 83 53 | S 83 53 | S 83 53 | l 108 6C | l 108 6C |

q 113 71 | t 116 74 | t 116 74 | t 116 74 | m 109 6D | m 109 6D |

r 114 72 | T 84 54 | T 84 54 | T 84 54 | n 110 6E | n 110 6E |

s 115 73 | u 117 75 | u 117 75 | u 117 75 | o 111 6F | o 111 6F |

t 116 74 | U 85 55 | U 85 55 | U 85 55 | p 112 70 | p 112 70 |

u 117 75 | v 118 76 | v 118 76 | v 118 76 | q 113 71 | q 113 71 |

v 118 76 | V 86 56 | V 86 56 | V 86 56 | r 114 72 | r 114 72 |

w 119 77 | w 119 77 | w 119 77 | w 119 77 | s 115 73 | s 115 73 |

x 120 78 | W 87 57 | W 87 57 | W 87 57 | t 116 74 | t 116 74 |

y 121 79 | x 120 78 | x 120 78 | x 120 78 | u 117 75 | u 117 75 |

z 122 7A | X 88 58 | X 88 58 | X 88 58 | v 118 76 | v 118 76 |

{ 123 7B | y 121 79 | y 121 79 | y 121 79 | w 119 77 | w 119 77 |

| 124 7C | Y 89 59 | Y 89 59 | Y 89 59 | x 120 78 | x 120 78 |

} 125 7D | z 122 7A | z 122 7A | z 122 7A | y 121 79 | y 121 79 |

~ 126 7E | Z 90 5A | Z 90 5A | Z 90 5A | z 122 7A | z 122 7A |

The HP-UX and AIX sorting are similar, but the sort binary on these two platforms doesn’t do the sort like on Linux or Solaris. I had to pass the -d option, but the result is not the same. Anyway, for the Linux sorting you can clearly see, that the Linux column is sorted after the numbers behind the special characters. They are just ignored (as already mentioned).

A solution for the different sort sequences is very simply. Switch from your Non-Unicode SAP system to an Unicode SAP system! Starting with Unicode, the sort sequences are inside the SAP kernel and they are the same for all platforms. Your end users will be gradeful.

Joseph J. PichHannes, this is a great article. Do you know of any transactions in SAP that we could demonstrate the difference in sorting between HP-UX and Linux?