Guru: Dynamic Arrays Come To RPG – The Next Part Of The Story

August 17, 2020 Jon Paris

In my first tip on this topic I covered the automatic sizing option (*AUTO) for the new dynamic arrays. In this tip I am going to look at the second option (*VAR), which allows the programmer to directly control the capacity of the array, growing and shrinking it as required. In addition I will also briefly cover the third option (*CTDTA) which, as you may have guessed, relates to compile time arrays.

Using A Varying Length Array

Let’s start with a brief example of using a variable sized array in conjunction with SQL. This approach answers the classic question of “How big should I make the array to hold the SQL result set?” With this new support we can deal with this problem by dynamically setting the size of the array based on the number of rows that will be retrieved.

In the earlier tip we used %ELEM to tell us the number of elements currently in the array or how many elements could fit into the currently allocated size for an array. We needed this because with *AUTO arrays the programmer is only indirectly controlling the maximum capacity. In this tip we’ll be using %ELEM to directly set the size of the array.

Below is the code, but first some explanation.

(A) We begin by defining the data structure (DS) array as type *VAR with an absolute maximum size of 32,000 elements. (In reality the actual maximum for embedded SQL is currently 32,763 – but 32K was easier to type!). For simplicity the DS is defined based on the table that I am using in this example.

(B) In order to obtain an accurate row count once the cursor has been opened but before the rows are fetched, I must declare it as an “insensitive” and “scrollable” cursor.

(C) Once the cursor has been opened, I can now use GET DIAGNOSTICS to obtain the row count (DB2_NUMBER_ROWS) and load it into the host variable count.

(D) Here’s where the magic happens. I now use the count to set the number of active elements in the array. Prior to this point that value would have been zero.

(A)    dcl-ds  dsArray  ExtName('ANIMALS')  Dim(*Var : 32000)  Qualified;
       End-ds;

       dcl-s  i  int(5);
       dcl-s  count  Int(5);
       dcl-s  type  Like(dsArray.type);

       Dsply 'Search for type? ' ' ' type;


       DoU type = 'exit';

(B)       exec SQL Declare ACursor insensitive scroll Cursor for
                                   Select * From ANIMALS where TYPE = :type;
          exec SQL Open ACursor;

(C)       exec sql get diagnostics :count = DB2_NUMBER_ROWS;

(D)       %Elem(dsArray) = count;

          If count > 0;  // If there are rows load the array
(E)          exec SQL Fetch ACursor for :count Rows Into :dsArray;
          Endif;

(F)       exec SQL Close ACursor;

          // Process the result set in the array or issue "Not found" message
          If count > 0;  // Show results
(G)          for i = 1 to count;  // Loop through all results
                Dsply ( 'Index ' + %Char(i) + ' name is ' + dsArray(i).NAME );
             EndFor;
          else;
             dsply ( 'No data for type ' + type );
          endif;

          Dsply 'Enter type (exit)? ' ' ' type;

       EndDo;             *InLr = *on;

(E) Now that the array is large enough to hold the complete result set I can fetch the rows into it.

(F) Once the data has been retrieved I can close the cursor ready for another query.

(G) Last but not least I use the row count I obtained earlier to loop though and display the results or output an appropriate “not found” message.

So what is the advantage of using a variable array rather than the conventional approach with a fixed-size array?

• First, and probably most importantly, a reduction in memory usage. With the traditional approach the full amount of memory is always used – even if 99.9% of the time the number of rows retrieved comes nowhere near the capacity of the array. If you attempt to reduce memory usage by keeping the array size low then you increase the risk that one day your program will blow with a size error.
• Another advantage is that any subsequent processing of the array can be against the full array. There is no need to use %SUBARR to restrict the range when using operations such as %LOOKUP, SORTA, etc. This results in cleaner, easier to understand logic.

Dynamically Controlling The Number Of Active Elements

In the example above I used %ELEM to set the number of active elements in the array at the start of each iteration of the process. But should I need to, I can adjust the size of the array at any time. For example, I might have a process that scans incoming data for (say) a product code. Whenever I encounter a new product (i.e., one that is not already in the array) I add it for subsequent processing.

Perhaps most days I only deal with about 100 or so different products, but once in a while (Christmas?) I get 1,000+ in a run. In these circumstances I could initially set the array size to accommodate the normal number (100) and then subsequently increase the limit should the number of entries climb above this number.

The code below is a simplified demonstration of one way of handling such a situation. Note that I have used a MONITOR block rather than having to add logic to constantly test if I am within the current capacity of the array.

(H) The first thing of note is the definition of the constant indexOutOfRange. As you will see in a minute this is used in the MONITOR block to detect when we attempt to use an index value beyond the array’s current capacity.

(I) Here I simply display the current array capacity, which should be zero.

(J) This simply attempts to place a value in the array. Since the initial capacity is zero we would expect this to trigger an exception (and it will).

(K) Here I catch only out of range errors. Any other error will follow the normal default RPG error path since I did not code a generic ON-ERROR clause.

(L) On detecting an out of range error, I simply increase the array capacity by using %ELEM to set the new limit to the current value plus 10. That is, I have allowed not just for the new element but for a further nine more.

(M) Now that the array capacity has been increased I can safely add the new element.

And that is all there is to it. In case you are wondering, yes, I could reduce as well as increase the array capacity and I will be looking at an example of doing that in the next part of this series.

n the next part of this series. CODE2

       dcl-s  dynArray  int(5)  Dim( *Var : 2000 );

       dcl-s  i  int(5);

(H)    dcl-c  indexOutOfRange  121;


(I)    Dsply ( 'Starting with ' + %Char(%Elem( dynArray )) + ' elements');

       for i = 1 to 21;

          Monitor;
(J)       dynArray(i) = i;  // Try to add new element

(K)       On-Error indexOutOfRange;
             // Current array capacity reached so increase by 10
(L)          %elem( dynArray ) = %elem( dynArray ) + 10;
             Dsply ( 'Item ' + %Char( i ) + ' capacity now '
                   +  %Char( %Elem( dynArray  )) );
             // Now we can add the new element
(M)           dynArray( i ) = i;
          EndMon;
       EndFor;       

Last But Not Least

For those addicted to compile time arrays, there is a third type of dynamic array available. This one is requested by coding the DIM like so: DIM(*CTDTA). This will automatically size the array based on the number of compile time data elements found in the source. Since I never use compile time arrays it is difficult for me to get excited about this option, but I can see how useful it would be for those of you do. No longer do you have to remember to go back to the array definition every time you add new elements to the compile time data. The compiler will calculate the new size for you! Of course this won’t help much unless you are also using %ELEM rather than a hard coded limit for your array loop control.

Next Time

In the final part of this series I will discuss some of the limitations of this new array support and how to overcome some of them. I will also be happy to answer any questions that may come to your mind after reading these first two installments.

Jon Paris is one of the world’s foremost experts on programming on the IBM i platform. A frequent author, forum contributor, and speaker at User Groups and technical conferences around the world, he is also an IBM Champion and a partner at Partner400 and System i Developer. If it wasn’t for Covid-19 he would be hosting the RPG & DB2 Summit twice per year with partners Susan Gantner and Paul Tuohy.

RELATED STORY

Guru: Dynamic Arrays Come To RPG