Is the IBM i Skills Shortage Accelerating Platform Migrations?

There are a lot of headwinds working against the IBM i platform, including a high up-front cost, a legacy perception, and executives who automatically default to X64 technology with technology decisions. These are all external challenges that the IBM i community can’t do much about. But there’s one headwind that’s internal to the IBM i community: A growing lack of personnel with the requisite IBM i skills.

One company that’s transitioning off the IBM i platform due, in part, to a lack of skills is PRGX. For decades, the Atlanta, Georgia, company used IBM‘s midrange server to automate many aspects of its accounts payable recovery audit service, which involves tracking down deals and discounts promised to retailers and grocery stores by manufacturers and distributors, but never delivered. PRGX recovers more than $1 billion annually on behalf of its clients, which include the biggest names in retail.

PRGX Director of Data Services Jonathon Whitton says speed was the main reason why the company decided to migrate the bulk of the ETL workload associated with its audit recovery services from the IBM i server to run on a distributed cluster of X64 servers running Hadoop. “We saw at least a 10x gain in speed,” Whitton told IT Jungle in a recent interview. “Things that were running in 140 hours on the AS/400 are completing in six to eight hours” on Hadoop.

That speed has a direct impact on the capability of PRGX’s data services division to feed actionable insight to the audit teams, who perform the actual recoveries. The more invoices, purchase orders, receipts, and emails that PRGX can load into the system and analyze, the greater the odds of spotting the missing deals, and the more money PRGX and its clients recover.

PRGX rewrote many of the RPG-based ETL jobs that formerly ran on the IBM i server into HiveQL-based batch jobs that execute in its Hadoop SQL environment, called Apache Hive. PRGX hasn’t completely migrated off the IBM i platform yet–one-third of the workload for one large client remain on the box. But it is close to making a full transition away from the proprietary scale-up server to the open source scale-out cluster.

It’s a major migration for the $138-million company, which is also moving a large percentage of processing work that formerly ran on more than 100 SQL Server systems to the Cloudera Distribution of Hadoop (CDH) system. The company is relying on ETL software vendor Talend to help it leverage new Hadoop technologies, such as Apache Spark, to crank the performance needle even more and enable analysts to explore the data and iterate more quickly and easily than before.

Whitton acknowledges PRGX’s digital transformation could have occurred on the IBM i system instead of switching to Hadoop. Other approaches could have been taken, especially considering the older iron PRGX was on, the immense amount of processing power in the latest Power8 servers, and continued SQL advances that IBM pumps into DB2, which is a significantly more mature database than Hive, even though it lags Hive in price/performance.

“It is possible we could have bought another AS/400 and done it in there,” Whitton said. “But I have a feeling the cost would have been different.”

The skills question also factored prominently into PRGX’s decision to migrate off the IBM i platform, Whitton said. “Granted there could have been some re-engineering that could have gone on with the old code on the AS/400,” he said. “With the AS/400, we have limited skills in our environment. Most of our business analysts are comfortable with SQL, not RPG or COBOL. For us, it was definitely a better fit.”

The lack of IBM i skills has emerged as a major theme the past few years. For example, it was somewhat surprising to see that the shortage of IBM i skills–such as RPG proficiency and general knowledge of the IBM i OS–surfaced as the number one barrier to digital transformation in a SoftLanding Systems survey conducted earlier this year.

The shortage of technologists with IBM i skills was cited by 50 percent of the respondents who took the SoftLanding survey. That was the top vote-getter in the category, followed by cost consideration (46 percent) and risks of making changes to core systems (42 percent).

“For people to say ‘IBM i skills’ was a bit of a stumbling block. That was surprising,” SoftLanding operations manager Jim Fisher told IT Jungle for our July story. “I would have expected it would be PHP or Java or whatever language for Web-based development.”

The shrinking of the IBM i skills pool is also impacting the movement of IBM i workloads to the cloud. Roger Mellman, a solutions architect with LightEdge Solutions, an Iowa-based managed service provider that hosts IBM i environments, says a retiring IBM i administrator is good business for the MSP.

“There are nuances to how you manage an IBM i environment,” he says, “but what we see from engagement to engagement is the lack of implementing basic AS/400 tasks because the administrator is also the CFO, HR or the person who has been at the company the longest. We typically engage with both SMBs and enterprise customers alike because their IBM i administrator is retiring or their business is requiring changes to the platform to accommodate new compliance requirements.”

IBM has done a lot to keep the IBM i server relevant in the modern world. It runs SQL. It runs all the latest scripting languages like Node.JS and Python. There is still a relatively healthy and diverse collection of ERP systems actively supported on it (see related story “How IBM i Fared in Top ERP List”).

There’s a core (albeit shrinking) group of vendors hawking add-on tools for the IBM i, and the platform’s security and reliability are what you would expect from a business-class server. By comparison, security is only now being built into Hadoop, which was initially developed at Yahoo to index the World Wide Web.

If the IBM i server is going to remain relevant, IBM i supporters will need to win their share of PRGXs. There will always be companies that look to the greener fields of emerging technologies, and make the leap off the ‘400. Migrations will happen, and some for very good reasons, as PRGX’s appears to be. But if the IBM i server is going to remain competitive, the emerging lack of IBM i skills in the workforce needs to be addressed.

There are things that the IBM i community can do to address the IBM i skills issue. First and foremost, IBM i shops must make a commitment to educate a new generation of programmers, operators, administrators, and architects on the box. This is especially true as the current generation of IBM i talent–largely baby boomers and Gen Xers–start to retire. Companies have invested billions into their IBM i systems, and they would be remiss if they didn’t maintain this investment.

Investing in people is arguably the most effective path for the IBM i platform to maintain its relevance in a fast-changing IT landscape.

RELATED STORIES

IBM i Skills Shortage A Barrier To Digital Transformation

Advice For IBM i Shops: Balance The Old With The New

IBM i Skills Shortage: Now You See It, Now You Don’t

There Is No Lack Of RPG Programmers, IBM i Community Contends

RPG Programmer Shortage Blamed For CSC’s Earnings Miss