The State Of The IBM i Base 2022: Third Party Software Conundrum
April 11, 2022 Timothy Prickett Morgan
Aside from death, most problems are not intractable. But people surely can be, and sometimes are. But luckily not often, and the thing about people is that, generally speaking, they can be reasonable when they are reasoned with. It is with all of this in mind that we come to the next in the State of IBM i Base stories for 2022, where we want to talk about the software trap that the remaining OS/400, i5/OS, and some IBM i shops have gotten themselves into and how we might help them get out of it to the mutual benefit of all.
As best we can figure, based on the data from the annual HelpSystems survey of the IBM i base, about two-third of the companies that take the survey have consistently said that they have homegrown applications running on their systems – something that was not asked in the original surveys from years gone by and a thing that I pointed out to HelpSystems and had the question changed because I simply did not believe that most of the base had third-party applications. Over the decades, the readers of The Four Hundred have consistently been do-it-yourself application shops and I just simply did not believe that somewhere along the way that changed so dramatically.
In theory, that means that two-thirds of the base is not facing what I will call the third-party software conundrum, where they are stuck on old releases without maintenance and no easy or cost-effective way to get those applications current. In practice, many IBM i shops are facing massive technical debt in their code, a lack of people with skills and insufficient funding to update their older RPG III, RPG IV, and ILE RPG applications to free form RPG, or worse yet, have lost the source code for their applications. And as a consequence they are just as stuck as anyone using an old suite of applications from a vendor that ended up inside of Infor or Oracle or one of the few remaining mid-sized ERP vendors catering to the IBM i crowd.
Part of the problem with regard to third-party application software, I think, is the fact that there is a long history of open source application code in the IBM midrange, and another part of the problem is the long practice of selling software with a perpetual use license that also has an annual software maintenance fee.
The fact that many of the thousands of application suites available for System/3X and AS/400 systems were available as source code meant that companies buying the software could indulge in customizing software at a level that we have generally not seen in the application space heretofore. There are plenty of IBM midrange shops that used a mix of custom code and heavily customized third-party code to create the systems that run their businesses, and at some point, the code has changed so much that there is no point in paying third-party maintenance on it. Companies could not upgrade to new application versions and suites form the vendor if they wanted because all of those customizations would have to be done again. So it is not just a matter of people not wanting to pay maintenance on application software – it would not get them anything if they had.
There are, of course, IBM i shops that have done a modest amount of customization on third-party code and when the budget gets tight, they stop paying for maintenance on it because they are not changing it, even if they do have the source code. And these days, with modern ERP, CRM, and SCM suites, they probably are not getting the source code for the new software unless it is grandfathered into their vendor contracts.
But even absent that, the way these licenses are sold were always a budgetary headache, and the problem is that people costs rise with gross domestic product and do not have Moore’s Law economic scaling, where things get cheaper per unit of capacity with each passing year, as happens with most elements of the system. This is why software maintenance really exists, and it is why it is set at 15 percent to 25 percent of the list price of the application software. That means every four to seven years, the maintenance fees are like buying the software all over again from an economic standpoint. And if the code is not changing – and because customers don’t want it to –and all the vendor is really doing is supplying security patches, then you can understand why IBM i customers might resent these fees.
Yes, it is unfair that some customers stayed on maintenance and paid and others did not, and that some IBM i shops expect a break on after license maintenance fees if they return to the fold and upgrade to software that is certified for modern IBM i operating systems and modern Power Systems hardware. But as I explained to a reader on LinkedIn last week in response to the software release problem with IBM i, where so many customers are on 7.1 or earlier releases, we can all dig our heels in and go straight to hell together, or we can figure out some way that everyone gives a little and we all benefit.
The application ISVs can dig their heels in and say they are entitled to all the back maintenance before getting customers current, and they will probably not get very far. If customers are going to spend huge amounts of money and have to massively customize a newer version of the code anyway, they will very likely just move to a different platform for political reasons more than technical ones. The economics will suck no matter what.
The customers who just sit there on older releases of operating systems and applications are sitting on a ticking timebomb, but sometimes this is in fact the least risky behavior as well as the least costly – right up to the catastrophe where all of this comes home to roost.
IBM has a hand in this, too, and has shown the way with amnesties on after license charges for IBM i Software Maintenance in 2015 and again in 2020, which you can see in the Related Stories section below.
All I know is that IBM i shops, application ISVs, and Big Blue have to all work together to solve this problem for those companies who rely on third party applications, and the fees that IBM i shops have to pay should be proportional to the amount of work it takes to get either old suites certified on new IBM hardware and operating systems or to get customizations ported to new suites that are already ported to them. (I think we all know which one is easier and cheaper.)
There is one more thing that we know. Real mitigation for Log4j security vulnerabilities has to be done, and that means IBM has to write new and secure logging software that snaps in place of Log4j and allows IBM i 7.1 releases and forward, including the Heritage Navigator as well as the new IBM i Navigator to work. Telling people with older releases to turn off Log4j and only turn it on to use Heritage Navigator at their own risk is not doing right by the customer, and IBM damned well knows it. When nearly half of the base is on IBM i 7.1, as I believe it is, and another fifth is on IBM i 6.1, and many of them are stuck, Big Blue simply cannot behave this way.
RELATED STORIES
7.1 Flew Over The Cuckoo’s Nest
The State Of The IBM Base 2022, Part Three: The Rusting Iron
The State Of The IBM i Base 2022, Part Two: Upgrade Plans
The State Of The IBM i Base 2022, Part One: The Operating System
IBM Grants Amnesty On Software Maintenance After License Charges
Where Is The Power Systems-IBM i Stimulus Package?
IBM Grants After License Amnesty For Software Maintenance
IBM i 7.1 Extended Out To 2024 And Up To The IBM Cloud
Big Blue Revives IBM i 7.1 With Power9 Support
IBM Further Extends Service Extension For IBM i 7.1
Service Extension Outlined For IBM i 7.1 And PowerHA 7.1
I just read your State of the IBM i Base: Third Party Software Conundrum. What you describe is eerily similar to our situation, but with a twist. I’ve been here 25 years as of last week. We installed a 3rd party ERP shortly after my arrival. Over the years I have made slight modifications, maintained our annual maintenance, and basically built our business around it.
However, in our case the ISV has (for the last 10 years) not held up their end of agreement. Our last version upgrade was 2009, and basically no “modernization” has been done to the application. They are, in my opinion, very close to just closing up shop. They recently let go their last support programmer of 27 years.
While we have the source code, I feel that we have no choice but to make a change. At 53, I need to be sure my small company is on the right path when I decide to walk away.
So we have basically paid 2.5x for no software improvement whatsoever – Yes, we got good support (for the handful of times we needed it). But now he’s gone.
Myself and ownership would rather stay on the IBM i platform if at all possible… We would like to find a modern “replacement” ERP that is as solid as what we have now. I have no idea where to look.
–Greg
Greg, thanks so much for reaching out. This is a twist I had not fully considered. But I suspect that a lot of software maintenance is a lot of horse hockey. Level 3 tech support should not cost the same as real application development, and maybe “support” or “maintenance” should be sold separately, as your case so aptly illustrates.
–TPM
Re: The State Of The IBM i Base 2022: Third Party Software Conundrum
100% Right On!
You have expressed very well the biggest problem with IBM i today and for the past 20 years.
Fortunately my version of ERP software is just old enough that it does not have the system change validation program. Newer versions of this software will stop working if you upgrade the hardware or OS. Forcing you to pay for continuous maintenance or repurchasing the software every 4 years.
We have rewritten the entire package and added tons of new functionality at this point.
And what do you get for installing the new software? No new features – they have not updated the software for 30 years.
They just update the system checker program so they can charge you again next time.
Given this treatment why would anyone purchase a new software application from this vendor. They are demanding money for nothing because they can.
I am aware of just a couple of software vendors that have been adding value to their systems in the past 5 or 10 years. The rest are owned by investment companies that have bought up all the applications and are milking the customer base without adding value.
Which leads to the other problem with IBM i applications as I see it.
We need active software vendors with business applications that are being continuously updated. The reason AS/400 was so popular when it came out was in the name: Application System. There were a ton of applications that ran on the system.
That’s why you buy a computer.
— Doug
Thanks for telling us your situation. I really appreciate it. This is a nuanced diamond to cleave to make a gem, isn’t it?
— TPM