Guru: Web Concepts For The RPG Developer, Part 1

April 22, 2024 Chris Ringer

Way back in the 1990s, I recall accessing data with only RPG III F-Specs. But nowadays some of that critical data may live in the cloud. The good news is tools like HTTPAPI and RXS and SQL functions like SQL HTTP are available to access that remote data from the IBM i. But what you may not know is how to actually format components in those HTTP requests.

Here I will discuss some techniques to build those components in an HTTP request before sending it across the web.

HTTP Get Versus Post

The two most common methods for an HTTP request are the GET and the POST. The GET is used to retrieve a resource from a web server. You are doing a GET when you type a URL in the browser address bar. For a POST, think of a form with input fields and a submit button that may update data. Now let’s review the format for each method.

Example GET:

01 GET /usa/getWeather?zip=46256&type=hourly HTTP/1.1
02 Host: myCoolWeather.com
03 User-Agent: MyClientTool/1.0 (System Test/7.5)

Line 01: A GET requests a resource which could be a static web page, an image or a web API (program).

/usa/getWeather is the path and name of the resource on the web server. An optional query string of data in key=value pairs can follow the name of the web page in the URL and the web server may filter the results in the response based on the query string. The maximum length of a URL including the query string is about 2000 characters in Chrome so don’t exceed that. And HTTP/1.1 is the HTTP version that the client supports. In IBM i terms, it’s like saying v7.4 vs v7.5. HTTP/1.1 is an older version but has been updated periodically and is still very popular today.

Line 02: The Host is the web server that contains the /usa/getWeather page.

Line 03: The User-Agent is information about the client (agent) sending the request like the browser name and version and device. This can be any plain text really but there is some intelligence built into the User-Agent for browsers and smartphones.

Example POST:

01 POST /post HTTP/1.1
02 Host: httpbin.org
03 Authorization: Basic RGFubnlOb29uYW46R29sZmVy
04 Content-Type: text/plain
05 Content-Length: 4
06
07 ibm!

Line 01: POST means data is passed in the body of the request (see line 07). /post is the path and name of the web API (the program) on the web server. And again, HTTP/1.1 is the HTTP version that the client supports.

Lines 02 – 05: These lines are the headers of the request, the metadata, so the server knows what it’s receiving and how to format a response.

Line 02: Host is the web server that contains the /post web API.

Line 03: An Authorization header contains credentials or an access token, if the web API requires it.

In this example the credentials are encoded as base64 (more on this later).

Line 04: Here the data in the body is just plain text. In RPG terms, it’s like declaring a variable as varchar, integer, a data structure, and so on.

Line 05: The length of the data in the request body is 4 bytes. After the web server reads in 4 bytes from the body, it can generate a response.

Line 06: A blank line means we’re done with headers and the data is next. How exciting!

Line 07: The body of the POST is ibm! which is 4 bytes long. The maximum length of the body can be several megabytes and the limit is controlled by the web server configuration.

The specs of the web API  for the request will determine if a GET or a POST is needed (or a PUT or a DELETE).

The Base64 Scheme

Web APIs are normally designed to receive text data, not binary data like images and digital signatures. The base64 scheme allows a stream of bits to be encoded into the 64 plain text characters + / 0-9 a-z A-Z. And because + and / are reserved characters in a URL, sometimes dash – and underscore _ are used instead respectively. Base64 encoded data may be used in both a GET query string and the body of a POST.

When a string or file is encoded into base64, the stream of bits is sliced up into sextets (groups of 6 bits) each of which maps to the 64 characters in base64 (Figure 1). Notice that every 3 bytes expands into 4 bytes.

Figure 1

The final base64 string may be padded with 1 or 2 trailing equal signs so the length is evenly divisable by 4 (Figure 2). Some web APIs require this padding and some don’t.

Figure 2

Here is a code snippet to convert an EBCDIC string to base64. The critical thing to remember is that to end up with base64 ASCII, you must start with an ASCII string, not EBCDIC. And I make “TO” variables 35% larger than the “FROM” variables to avoid truncation.

01 dcl-s wkString        varChar(1000);
02 dcl-s wkStringAscii   varChar(1000) ccsid(*UTF8);
03 dcl-s wkStringAscii64 varChar(1350) ccsid(*UTF8);
04 dcl-s wkString64      varChar(1350);
05 wkString = 'ibm';
06 wkStringAscii = wkString;
07 exec sql set :wkStringAscii64 = BASE64_ENCODE(:wkStringAscii);
08 wkString64 = wkStringAscii64;  // aWJt

Lines 01 and 04: These lines are EBCDIC variables.

Lines 02 – 03: These lines are UTF-8 variables (CCSID 1208) but let’s call them ASCII to make this simple.

Line 05: Sets variable to ‘ibm’.

Line 06: Converts the EBCDIC string to ASCII.

Line 07: The SQL function BASE64_ENCODE converts our string to base64.

Line 08: Converts the ASCII string back to EBCDIC.

URL Encoding

Some characters in a URL such as & : . = + $ , / ? % #  are reserved characters with special meaning. To represent these characters in a query string as data they must be percent-encoded into hexadecimal digits. To relate this in RPG terms, a single quote literal must be doubled up when embedded in a string.

bookOwner = 'Bob''s book';

Here we use the SQL function URL_ENCODE to convert a string to be safely used in a URL.

01 dcl-s wkString     varChar(1000);
02 dcl-s wkStringURL  varChar(3000);
03 wkString = 'start :/?& middle =#+ end';
04 exec sql values(URL_ENCODE(:wkString)) into :wkStringURL;
05 exec sql values(replace(URL_ENCODE(:wkString),'+','%20')) 
06 into :wkStringURL;

Lines 01 – 02: Notice that the second variable is 3x the size of the first variable. The URL encoded string could grow 200 percent or more from the original value (figure 3).

Line 03: I added a few reserved characters to the string.

Line 04: The URL_ENCODE function encodes our string.

start+%3A%2F%3F%26+middle+%3D%23%2B+end

Lines 05 – 06: Notice the four plus signs in the line 04 URL string. In the query string of a URL, a space may be encoded as a plus sign or %20. The web API you are consuming may have a preference. When in doubt, use the SQL function REPLACE to convert all + to %20.

start%20%3A%2F%3F%26%20middle%20%3D%23%2B%20end

Figure 3

The technical specs of the web API you need to consume will ultimately determine which HTTP headers are required and how data needs to be formatted in the request. If you wish to try some of the code snippets in this article, you can copy the code into the RDi editor but of course it won’t like the line numbers. You can quickly remove them off by delimiting a block of code with the commands LLT2 and LLT (figure 4). This will shift those lines 2 characters to the left and trim off the line numbers.

Figure 4

Well that’s it for now. Stay tuned for Part 2. And happy coding!

Chris Ringer began coding RPG programs in 1989, and after a recent unexpected but valuable detour to C# is happy to be back in the IBM world. In his spare time he enjoys cycling and running – and taking the family dog Eddie for walks.

RELATED STORIES

Guru: The PHP Path To Victory, Part 1

PHP Crash Course For RPG Developers

Installing PHP on Your Laptop

Debugging Server Jobs In Green Screen