Guru: Practicing Safe Hex in RPG

March 2, 2020 Jon Paris

In this tip I’m going to address a question that arises regularly on RPG-oriented Internet lists, namely: “Is there an easy way to convert a character string to its hexadecimal equivalent?”

One answer, of course, would be to write your own routine using lookup tables, but there is a far easier way. We can take advantage of the system’s hex MI APIs. These were originally surfaced for use by C and C++ but, thanks to the joys of ILE, can be used by any ILE language. Not only that, RPG’s prototyping support makes them really easy to use. In fact, not only can you easily convert from a character representation to its hex equivalent, you can also go the other way, i. e., take a hex string and convert it to characters. Indeed, these APIs are not limited to character data but can work with any “chunk” of memory which means they can be used to encode/decode images, PDFs, etc. More on this later in the section “An Alternative Prototype?”

Suppose I need to create the hex equivalent of the character string ‘ABCDE’. The resulting hex string should be ‘C1C2C3C4C5′ since X’C1’ is the hex value for ‘A’ in EBCDIC, X’C2′ is the value for ‘B’ and so on. Similarly my name (Jon Paris) would convert to ‘D1969540D7819989A2’.

The two procedures that enable these conversions are cvtch, which converts from hex pairs to their corresponding character representation, and cvthc, which converts individual characters to their corresponding hex pair. If you want to know the full details of these procedures you can find them, and many others, documented in the manual ILE C/C++ for AS/400 MI Library Reference.

The names for these procedures can be a little misleading, and the manual description needs careful reading to understand which does what. For example, you might think that the name cvtch means “convert character to hex” based on its name. In fact it does the exact opposite, a fact made a little clearer by the full IBM title for the API, which is “Convert Eight Bit Character to Hex Nibbles.” OK . . . so that doesn’t make it that much more obvious! The full title should probably be “Convert Eight Bit Character Representation of a Hex Digit to the Corresponding Hex Nibble.” Of course even that only makes sense if you know that a nibble (or nybble as it is sometimes spelled) is four bits — half a byte. Half a byte is a nibble!

Let’s start by looking at the prototype for cvthc.

dcl-pr CharToHex ExtProc('cvthc');
   hexResult    Char(65534)  Options(*VarSize);
   charInput    Char(32767)  Options(*VarSize);
   charNibbles  Int(10)  Value;
End-Pr;

There are a few points to note here:

First, I named the prototype CharToHex rather than use the actual API name. This helps avoid misunderstandings due to the somewhat misleading name and makes it more obvious what the API does.

Second, I have defined both the input (charInput) and output (hexResult) parameters with Options(*VarSize) so that fields smaller than the declared maximum can be used. Note that the output parameter must always be twice the length of the input since two output characters will be generated for each input character. It is also worth noting that you can make the lengths pretty much anything you like.

The third parameter will contain the length of the input field in nibbles, i.e., the length of the input field multiplied by two.

Let’s look at the process in action.

    dcl-s  textString  Char(24);
       dcl-s  hexString   Char(48);
       dcl-s  length      int(10);
    Dsply  ('Enter Max 24 char string') ' ' textString;
       // Calculate the number of hex nibbles in the input
    length = %Size(textString) * 2;
    CharToHex ( hexString : textString : length );
       Dsply 'Hex result is:';
       Dsply hexstring;

<A> Defines the three variables used.

<B> The program requests an input string of up to 24 characters.

<C> This calculates the length in nibbles of the input string.

<D> We then call CharToHex() to perform the conversion and display the results. If you are thinking that I could have simply specified %Size(textString) * 2 as the length parameter rather than calculate it separately (doing away with the length field) you are quite correct.

CharToHex ( hexString : textString : %Size(textString) * 2 );

The prototype for the reverse procedure HexToChar (cvtch) is, as you might expect, basically the same. As before the result is placed in the first parameter, and the input is taken from the second parameter. You can see the call below beneath the prototype. The length of the input is passed in the third parameter. Note that this is the actual length in bytes and there’s no need to multiply by two.

dcl-pr HexToChar ExtProc('cvtch');
     charResult  Char(32767)  Options(*VarSize);
     hexInput    Char(65534)  Options(*VarSize);
     hexLength   Int(10)  Value;
End-Pr;
...
HexToChar ( textString : hexString : %Size(hexString));

An Alternative Prototype?

If you are experienced in working with prototypes for C-style APIs may have been surprised by the prototypes I showed earlier. From reading the documentation you would probably have expected to see something more like this:

dcl-pr cvtch ExtProc('cvtch');     
   charResult  pointer  Value;     
   hexInput    pointer  Value;     
   hexLength   Int(10)  Value;     
End-Pr;                            

Indeed that is a technically accurate representation of the APIs definition. So why did I choose to use a different prototype, and why do both of them work?

The reason I chose to use a different prototype for my main example is simply that in my opinion it makes the invocation more “RPG like.” What I mean by that is best explained by showing what the invocation of this version would have to look like:

HexToChar ( %Addr(textString) : %Addr(hexString) : %Size(hexString));

As you can see, it requires that we pass the address of each parameter rather than simply reference the field. So how come my first version works? Simply put, when you pass a variable by reference, which is what we are doing in the original example, a pointer to that variable is passed to the called routine. And that is exactly the same thing that we are doing when we pass %Addr() when the prototype defines the parameter as a pointer passed by value. That is, passing a variable by reference is the same as passing a pointer to that variable by value.

So do I ever use this second version? Yes. I use this when I am working with dynamic storage, since I already have a pointer to the storage. I also use it when processing fields in teraspace that exceed 16Mbs because such fields that are too large to define directly in RPG.

In the code package that accompanies this tip (which can be downloaded here) there is an example of allocating a large amount of storage (17Mb) and processing it with these procedures.

Wrap It Up

If you look at the MI reference I mentioned earlier, you will see that are a number of other low-level MI APIs available to the RPG programmer. If you have a need for any of these and encounter any issues, please let me know.

Jon Paris is one of the world’s foremost experts on programming on the IBM i platform. A frequent author, forum contributor, and speaker at User Groups and technical conferences around the world, he is also an IBM Champion and a partner at Partner400 and System i Developer. He hosts the RPG & DB2 Summit twice per year with partners Susan Gantner and Paul Tuohy.

RELATED STORIES

Fundamentals: Parameter Passing

Parameter Passing Fundamentals Of Programs Versus Procedures

Where Does Data Live? The New Basics