Is IBM i Ready for the Agentic AI Revolution?

March 24, 2025 Alex Woodie

The AI world moves fast. Just when you get your head around large language models (LLMs) and the chatbots and coding co-pilots they enable, the AI world has moved onto something new. In 2025, that something new is the reasoning model, which is enabling autonomous AI agents and the world of agentic AI.

Is IBM i ready for the revolution that’s about to unfold?

By now you have undoubtedly read about LLMs such as OpenAI’s ChatGPT, Google’s Gemini, and Meta’s Llama. These LLMs are popular ways to power text-based generative AI applications, such as chatbots, coding co-pilots, and question-and-answer (Q&A) systems. The better search engine results you (hopefully) are experiencing are likely the result of GenAI technology, such as LLMs, vector databases, and k-nearest neighbor algorithms, working behind the scenes to provide a better match for your (flawed and/or misspelled) search terms than what brute-force keyword-matching can provide.

IBM is embracing the world of GenAI with its watsonx product and its Granite line of open source foundation models, which provide a range of AI capabilities for things like text understanding, Q&A systems, code generation, image-to-text conversion, and retrieval augmented generation (RAG). In February, IBM announced Granite 2B Vision, a multimodal LLM (or MLLM) that can provide computer vision capabilities, such as understanding documents that contain visual elements, such as images, charts, and graphs. Just as LLMs have upended the old ways of doing natural language processing (NLP), MLLMs based on modern deep learning technologies are giving us a big performance boost over the fussy optical character recognition (OCR) tech of old.

The IBM i world is already moving to adopt some of these new LLMs. IBM’s IBM i development team in Rochester, Minnesota currently is working to create a co-pilot for RPG. Dubbed RPG Code Assist, the product is being developed to accomplish three tasks: explaining RPG code, generating RPG code, and generating use cases to test for RPG. IBM is currently hoping to have a beta of RPG Code Assist ready for by the end of the second quarter of 2025, with general availability hopefully in the second half of the year, according to a January update by IBM i chief technology officer Steve Will.

The IBM i vendor ecosystem is also moving forward with GenAI. For instance, ARCAD Software is supporting AI-powered code analysis in its Discover product, which is often used as part of application modernization projects. As ARCAD CTO Michel Mouchon recently explained in an IT Jungle article:

“ARCAD Discover has been equipped with a conversational AI front end – a chatbot in the modern parlance – so developers can ask questions about the applications in plain English, French, Spanish, and other languages and get answers back in the same language. Because of this AI chatbot, users of Discover do not have to have the in-depth knowledge of RPG or SQL programs that developers have; they can be business managers, project managers, application and system analysts, or quality assurance engineers…”

What ARCAD Discover is not doing – and which no vendor can do yet – is directly analyzing RPG code (or generating it, for that matter). As Mouchon points out, you could throw all of your RPG code into an LLM and see what comes out. But that isn’t likely to work very well, for the simple reason that few examples of RPG exist in the wild for LLMs to be trained on (as opposed to COBOL, which is plentiful on the Internet). That is why IBM’s effort to build RPG Code Assist is so important, and why you might want to consider donating some of your RPG code to IBM to be used for training.

Fresche Solutions is also moving forward with GenAI. In October, it launched its AI-Celerate program, which it describes as “a 12-week strategic advisory framework” for developing “personalized AI assessments, strategies, and roadmaps” catered to individual organizations. That could include helping customers figure out how to use AI to help generate plans from complex business requirements, to design and prototype code and user interfaces, and to automate testing.

Fresche is placing a big emphasis on the importance of IBM i data with its AI-Celerate program. “AI thrives on data and IBM i environments provide exactly that – rich, historical data that help drive meaningful AI initiatives,” Fresche chief executive officer Joe Zarrehparvar said in a statement accompanying the launch. “We’ve already seen considerable engagement from our customers on how to strategically use their IBM i’s vast data to power their AI engine.”

Profound Logic is also moving forward with GenAI, which is so central to the company now that it spells out its AI goal on the front page of its website. “Our mission is to help you transform these legacy systems into modern, AI-enhanced applications while protecting all that valuable business logic you’ve built over the years – because we believe you shouldn’t have to choose between innovation and preserving your investments.”

The company developed ProfoundAI, which serves as an abstraction layer that IBM i shops can put between their existing applications and databases, including Db2 for i, and the most popular commercial and open source LLMs, including Open AI’s GPT, Google’s Gemini, Meta Platforms’ Llama, and Anthropic’s Claude. ProfoundAI functions like a junior-level data analyst. For instance, you could ask the software what revenues in certain regions have been like for a certain timeframe, or even create a graph in an Excel spreadsheet based on the data from a query. The software also functions as a copilot, generating code for new applications (but not RPG, obviously) and even executing basic tasks. In that respect, Profound is already embracing the emerging world of agentic AI.

Agentic AI is the next evolution of GenAI. Whereas chatbots and copilots can generate words and code in response to input, AI agents take the next step and autonomously execute tasks, such as running that SQL query that a copilot generated, kicking off a software test as part of a DevOps cycle, or even booking a flight and a hotel for the CEO’s next business trip.

The age of agentic AI is already here. According to Deloitte, 25 percent of companies that use AI will launch agentic AI pilots of proofs of concept this year, growing to 50 percent by 2027. “While chatbots and agents share the same foundation – large language models (LLM) – additional technologies and techniques enable agents to act independently, break a job down into discrete steps, and complete their work with minimal human supervision or intervention,” Deloitte executives write.

Several technological breakthroughs are enabling agentic AI to take off so rapidly. The first one is the creation of reasoning models, such as OpenAI’s o1 and DeepSeek R-1. Just like a human may think for a bit before responding to a difficult question, a reasoning model uses what’s dubbed “chain of thought” reasoning to solve a problem.

OpenAI described the reasoning process in o1 in a recent blog post. “Through reinforcement learning, o1 learns to hone its chain of thought and refine the strategies it uses. It learns to recognize and correct its mistakes. It learns to break down tricky steps into simpler ones. It learns to try a different approach when the current one isn’t working. This process dramatically improves the model’s ability to reason.”

While this approach can dramatically reduce the error rate associated with first-gen LLMs and put a much-needed check on their tendency to hallucinate answers, reasoning models come with costs. For starters, they require significantly more computing power to train and to run, and they don’t provide instant answers. What’s so important about the introduction of DeepSeek R-1 is that it showed that reasoning models could be trained at a fraction of the cost that was previously required. In DeepSeek’s case, the training cost $5.5 million and was done on older GPUs from Nvidia, which triggered a massive selloff of that company’s shares.

But you might want to hold onto those Nvidia shares a bit longer, as things are just starting to get interesting in AI. Thanks to a process called model distillation, which is essentially training a new model on the output of existing models, the costs of training high quality AI models will plummet. Even if a model like GPT-4 isn’t open source, one can essentially get the benefits of that model and encapsulate its “knowledge” in a new model with a smaller footprint. When you fine-tune an open source reasoning model on your company’s own private data, you have a powerful tool for automating tasks in your company.

The other big thing that’s happening with agentic AI is improvement in code generation. While first-generation copilots wrote code that was often buggy, that has changed dramatically. Benchmarks like SWE-bench are cataloging the marked improvement in AI code generation, which in some instances are exceeding human ability. Companies are beginning to trust the SQL, Java, and Python code generated by copilots and put it into production.

“We are not far from a world – I think we’ll be there in three to six months–where AI is writing 90 percent of the code,” Anthropic co-founder and chief executive officer Dario Amodei said earlier this month. “And then in twelve months, we may be in a world where AI is writing essentially all of the code.”

Instead of hand-coding software to retrieve data from computers, in the future we’ll describe to a computer what we want to happen, and agentic AI will make it happen, Nvidia co-founder and chief executive officer Jensen Huang said in his keynote address at the GPU Technology Conference (GTC) in San Jose. “Whereas in the past we wrote the software and we ran it on computers, in the future, the computers are going to generate the tokens for the software,” Huang said. “So the computer has become a generator of tokens, not a retrieval of files. [It’s gone] from retrieval-based computing to generative-based computing.”

If agentic AI pans out the way Amodei, Huang, and others believe, it will mark a colossal shift in how software is developed and run. The days of needing hire a large team of software engineers to develop a compelling application appear to be numbered. It is just a matter of time before AI models learn RPG, which isn’t the only language for IBM i but is by far the most dominant. When AI learns RPG, these agentic AI advances will suddenly open up to IBM i shops. The big question is: Are they ready?

Are you ready?

This Issue Sponsored By

Table of Contents

Content archive

Recent Posts

Subscribe

Pages

Search