What The Open Source Roadmap Holds For IBM i In 2021
March 15, 2021 Alex Woodie
2020 was whirlwind of a year when it comes to open source on IBM i, with many major new open source products coming to the platform. The roadmap for open source software in 2021 may not be quite as active as last year, but there will be some compelling new technology to play with, coming from IBM and the community, IBM’s open source guru Jesse Gorzinski tells us.
“Last year was a pretty big year for open source,” says Gorzinski, whose official IBM title is senior business architect for open source software on IBM i. “In 2021, we’re going to have some big, big advancements.”
Historically, there has been a lot of focus on bringing open source languages to the platform, but Gorzinski would like to change the conversation.
PHP was the first open source language brought to IBM i way back in 2006. But it was actually preceded by several years by the HTTP Server (you know, the one powered by Apache). Many of today’s common infrastructure components are open source, including Git and Jenkins, not to mention the bevy of open source databases that came to IBM i in 2020, including MariaDB, MongoDB, Redis, and PostgreSQL.
Expect more open source infrastructure components to come to IBM i this year, Gorzinski says
“I think we’re finally turning that corner and that’s really a trend that I think we’re going to see moving forward in 2021 and beyond is the stuff beyond just programming languages,” Gorzinski says. “We’re seeing more and more interest in being able to tie in with certain monitoring tools like Grafana, Nagios. So that’s definitely a trend that we’re seeing.”
Gorzinski predicts that Node-RED will become more popular as members of the IBM i community see what can be done with it. The Node-RED dashboards that folks like Matt Seeberger are building are starting to turn heads, which will help to build momentum for open source in general.
There will also be more interest in big data frameworks. IBM has done a lot of work to promote these frameworks in the x86 Linux world. Apache Hadoop was the headliner for this new wave of development, and IBM had its own distribution of Hadoop for a bit before committing to support the one developed by Hortonworks, which merged with Cloudera. The Java-based Hadoop product itself has fallen out of favor, but many of the sub-projects that it helped spawn have continued to grow.
Chief among these post-Hadoop frameworks is Apache Spark. The Scala-based framework has surpassed Hadoop in popularity as a general-purpose computation engine, and it’s widely used to write large-scale data transformation jobs as part of an ETL pipeline, in addition to processing SQL queries, stream processing, and machine learning.
Alas, there is not much demand for Spark, Gorzinski says. But there is some interesting stuff going on with a related project, called Apache Kafka, which IBM brought to the PASE side of the platform in 2020.
Kafka today has blossomed into the de-facto standard message bus used to move data from one system to another, while simultaneously performing some actions on the data, such as preparing it for loading into a data warehouse or even performing computation or analysis directly upon the stream (stream processing or streaming data analytics).
While Hadoop and Spark are non-staters for pragmatic IBM i shops, Kafka is tickling their fancy, according Gorzinski.
“I would say there’s not a lot of interest with necessarily running Spark on i, but there’s definitely a lot of interest in using Kafka as this message bus to integrate with Spark and all kinds of other technologies,” he tells IT Jungle. “That’s one of the larger growth areas that we’re seeing right now is that the use of Kafka . . . . We have a pretty growing demand right now for this enterprise messaging stuff with IBM i, and a lot of that is driven by Kafka.”
There is internet among IBM i shops to push transactional data through the Kafka bus to downstream systems, which could include Spark-based AI systems, Gorzinski says.
“Once you have this data that’s flowing through these Kafka streams, then you have your AI infusion, you’re monitoring solutions, your Apache Spark, your REST gateways,” he says. “All of that stuff that sometimes is just built on Kafka with IBM i just kind of powering the data behind this message bus, if you will.”
In addition to Kafka, IBM i is getting support for other data services, such as Apache Camel, which he described as a “Swiss Army knife of integration tools” in a conversation with IT Jungle in 2020.
“When we did the new data queue services for the database, the reason that we actually published those was for direct database integration with Apache Camel,” he says. “Apache Camel, of course, then integrate you with Kafka Streams or Active MQ or Artemis or pick your technology really.
“The ability to have database triggers to hook on your transactional processing and stream your transactional data in real time to something like Kafka — that has value for people wanting real time transactional analysis through Spark for AI processing or monitoring,” he continues.
In addition to Kafka, Gorzinski sees other open source elements getting attention in 2021, including the work that has been done recently with the Node.JS connectivity libraries and the Python connectivity libraries.
“The Seiden Group folks have done some really great stuff with the toolkit over there,” he says. “And so moving forward into 2021 that’s going to be a continued trend. What we’re looking for right now is we want to make sure that our customers can be successful as they move into these hybrid cloud style deployments.”
IBM is also investing in updating its compilers. “We’ve issued the statement of direction of the new LLVM-based compiler framework for PASE for IBM i, so keep your eyes open for that,” he advises. “We’re going to have updated versions of our what I call our flagship languages from IBM, which is Node.JS and Python. And so definitely a lot of those big things are going to continue to happen and a lot of it beyond that is it’s going to be community driven as well.”
IBM has taken the lead in a lot of open source projects, but Gorzinski is seeing the IBM i community stepping up to contribute more to open source projects. “2020 was a very odd year,” he says. “We had a lot of technology we delivered. But 2020 had, more than ever before, community members stepping up, contributing examples out there.”
For example, Gorzinski credits Jack Woehr with stepping up and providing some “heavy lifting” work, specifically with getting a Rexx SQL interpreter running in PASE. “We had a lot of people outside of the IBM team stepping up,” Gorzinski says. “That’s the trend that I’m going to expect to see growing in 2021. We’re going to see more and more of these community contributions. ”
There will also be some headway with Java, which falls under the open source rubric in Rochester, Minnesota. There will likely be a delivery of the JV1 development kit sometime in 2021, Gorzinski says.
RELATED STORIES
Why Open Source Is Critical for Digital Transformation
Apache Kafka And Zookeeper Now Supported On IBM i
