Guru: How Thorough Was Your Last Test? RDi Code Coverage Can Tell You
July 22, 2019 Susan Gantner
When you’ve made changes to one or more programs, you test all the changes – right? And, of course, you also test all the rest of the code just to make sure you didn’t break anything else. Did you do that with your last set of changes? Did you test ALL the code? Enabling you to answer that last question is what RDi’s Code Coverage facility is all about.
This is an introduction to Code Coverage — the basics of both why and how to use it. Before I go into how to run it, it may pique your interest to see the results of a Code Coverage test and what they can do for you. Figure 1 below shows a Code Coverage summary report of a test run on a single program, showing what percentage of the lines of code in the program and in each subprocedure in the program were actually run during the test. Some significant changes to the reporting were made in version 9.6.0.5. If you have used Code Coverage in earlier releases your report will look a bit different, but the information on it is basically the same.
The report shows that my test touched only 73 percent of the program’s code. Drilling down to see the procedures inside the program, it is clear that one procedure (FILLPRCODE) was never tested at all during the analysis. It also shows that a number of lines of code in the main procedure and in another subprocedure were not covered. To find more detail about what lines of code were and were not covered, I can double click on the name of the program to open up the Code Coverage source view, which is shown in Figure 2.
As you can see, this brings up the source marked up to the left of the sequence number with red or green indicators to show exactly which lines of code were (and were not) touched during the test run. Note the two lines of code with the red mark to the left. This shows that these lines were never executed, which explains the two features of this program that were not tested.
How To Run Code Coverage
If you’re interested in trying it for yourself, let’s look at how to run a Code Coverage analysis test. I have created a short video to cover these basics as well as describing the steps here.
Code Coverage uses the debug engine. That means that if you are comfortable using the RDi debugger, you will likely find the process simple. If not, a few pointers may be helpful. It also means that, just like the debugger, the programs being analyzed must have source or listing view debug data included (or both).
Also like the debugger, there are two primary approaches to starting a Code Coverage session: you can use Service Entry Points (SEPs) or you can manually create configurations. In this first tip, I’ll concentrate on using SEPs for Code Coverage. A later tip will go into the details of creating and using manual configurations.
Before going further, you should take a look at the “IBM i Debug” preference page in your RDi workspace. If you normally need to say “Update production files *Yes” when you debug programs, then you probably want to set that as your preference on this page. It is not turned on by default. This preference works for both Code Coverage sessions as well as for debugging.
Running Code Coverage Using SEPs
Let’s start with the SEP process. Here’s the big picture of what we’ll do to analyze the testing of a program using Code Coverage with SEPs. Details for each step follow in the text.
– Set a Service Entry point on the initial program to test
– Ensure the SEP is for Code Coverage (vs Debug)
– If needed, add more programs or service programs by modifying the SEP launch configuration.
– Run the program(s) to be tested.
- When the initial program ends, the Code Coverage Results view appears and the editor window will be populated to show the summary report. Double click on a program to see the source level detail.
Step1: In the Remote Systems view, right click on a program (or service program) you want to analyze. If you want to analyze more than 1 object, you can add more later, but the program you select here should be the first of those programs that will be called during the test. From the context menu, choose Debug or Code Coverage (Service Entry) > Set Service Entry Point. RDi will then notify you that the SEP is started by opening the “IBM i Service Entry Points” view with an entry in the table for your program. Note that the entry also includes your user ID. When you run your test, you must be running with that user ID. You can change the user ID from your own if necessary by modifying the entry in this table.
Step 2: If you have used SEPs for debug, here’s an important difference to pay attention to. An SEP can be in either Debug mode or Code Coverage mode. By default, it will initially assume Debug mode. You may need to change the mode. This can be done from the context menu or by clicking the Code Coverage icon just to the right of the debug icon in the toolbar. Both options are highlighted in Figure 3.
Step 3: If you want include other programs and/or service programs in your analysis, you need to add them to your session. You do that by modifying the generic SEP configuration used by RDi. Choose “Open Launch Configuration” from the context menu of the SEP entry. Figure 3 above shows the options from that context menu.
Figure 4 below shows the dialog for adding programs to the SEP launch configuration together with the pop-up window that appears after pressing the Add… button.
Now that you have the SEP set up in Code Coverage mode, the host system is now on the lookout for your user profile (the one specified in the SEP) to start running the program (also specified in the SEP) anywhere it may occur on the system – just as it works with the debugger. I wrote a tip on using SEPs with the RDi debugger here if you want to read more about SEPs in general.
Step 4: Your next step is to run your test. Since you’re using an SEP, that test may be in an interactive job, a batch job, a web or database server job – it doesn’t matter. For my example, I’ll simply call my program from an interactive session.
Step 5: When the program finishes running, RDi will open (if necessary) the Code Coverage Results view that will contain an entry for your test run. Figure 5 below shows the Code Coverage Results view for this run along with results from some earlier testing sessions. It will also automatically open a Code Coverage report in the editor window like the one shown in Figure 1 above.
There are some limitations when using SEPs. The most significant one is that the initial program must be an ILE language program (e.g., it cannot be CLP or RPG400.) For those situations, I could create a small CLLE or RPGLE program that does nothing more than call the CLP or RPG400 program. Alternatively I could manually create a configuration that will call the program for me. I’ll cover the process for creating and using a manual configuration for Code Coverage in my next tip.
More To Come
I’ve just covered the basics of Code Coverage here. In addition to covering manual configurations in my next tip, I’ll talk about more tidbits about the tool, including how to see a report that is a merger of multiple code coverage runs and how to run Code Coverage via a CL command on the host rather than using the process from RDi described here.
Susan Gantner, an IBM Champion and co-author of the popular Redbook, Who Knew You Could Do That with RPG IV, is one of the top speakers/writers/trainers on IBM i development topics. She is a partner at Partner400 and System i Developer, and she hosts the RPG & DB2 Summit twice per year with partners Jon Paris and Paul Tuohy.
