One Repository To Rule The Source – And Object – Code

February 4, 2019 Floyd Del Muro

(Sponsored Content) The concept of a single repository for source is not necessarily a new one. When I interviewed with ARCAD back in 2011, I did so at the at the Rational conference called Innovate in Orlando. The research and development team and our chief technology officer were already in dialogue with IBM to resell ARCAD technology alongside its Rational development suite, adding power to Rational Team Concert that development organizations could effectively have a similar repository for IBM i and open source applications.

At the time, RTC supported the open source world very well, just like Git does now. But back then, Rational was a competitor of ours, and the other commercial offering was Team Foundation Server from Microsoft, what is now called Azure DevOps.

When I spoke about DevOps back at the COMMON conference in Disneyland back in 2015, I was able to thank the two people that were there personally. It wasn’t that bad, but even though DevOps had resonated very well with the open source world, the ideas behind it were new to the IBM i platform. But to make a long story short, the commercial offerings from IBM and Microsoft have a very, very large footprint and they also have a very large price. IBM’s RTC was doing well in the open source arena and in the System z mainframe shops, but it wasn’t really resonating with i. At the same time, there was obviously a massive movement of open source development and code sharing going on. There were already a couple of products out there that were free open source tools – CVS and Subversion were the two big ones, but neither one of those really operated well nor were they being supported to the level that Linus Torvalds, the creator and maintainer of the Linux operating system, likes. So in his wisdom, Linus created Git, and it became the 800 pound gorilla for the versioning of open source projects. Git became so prevalent that even the big boys, Microsoft and IBM, now support the Git repository within their commercial offerings and Microsoft went so far as to spend $7.5 billion to acquire GitHub, the commercial company behind the Git repository, and IBM has countered to a certain extent by shelling out $34 billion to buy Red Hat.

These deals and strategies by IBM and Microsoft really validate where open source development is going and how it’s important. ARCAD had already done the R&D, and put in the effort, to be able to support a single repository model. It wasn’t a big leap for us to then support the other candidates such as CVS or SVN and ultimately Git as well in our tools. We were already in that single repository, where as a traditional change management software provider on the IBM i platform, we were actually relinquishing the source control component of change management and giving it to one of these open source tools. We were smart enough to start doing this back in 2010.

Now, IBM likes to say that Git is enterprise ready. ARCAD thinks that Git is enterprise ready for the open source community because it supports their way of coding and it comes from their world. But we do not believe that Git is enterprise ready from an IBM i perspective, and that’s where ARCAD fills in. The IBM i development world has some very platform-specific needs that ARCAD supports. So even though we have relinquished the ability to manage the source code to Git, we still do cross platform cross referencing, we still do all of the build dependencies, we still manage all the authorities and objects without source code on the IBM i platform – all of these gaps that Git by itself would not understand.

But we also had to support a workflow – a Git flow if you will – to have the kind of normal terms and dialogues that team development had in the IBM i world. Our implementation of Git allows IBM i to move away from what we call a pessimistic source model, where you check code out and you lock it. Git opened itself up to a more optimistic source model, where there is no locking. We thought this was the wave of the future, and now IBM i shops can give unfettered collaborative capability as they code. It allows for code review, project management, integrations with other additional tooling that enterprises may have, social coding, and issue tracking.

The concurrent development, the social coding, the conflict resolution, and the ability to merge code – those were things that were important. And because we had already had the relationship with IBM six years ago, it was an easy transition for us to partner with GitHub two years ago and make that same offering with Git as your repository source. Now, Git doesn’t have a project management component. But there are other open source tools in the marketplace, such as Atlassian Jira, that also has an enterprise for-pay model. Together, Jira and GitHub are really kind of the same model as IBM RTC, but split over two different products. They easily integrate, very open technology. Yet the hard sell is for us to convince the community that it does provide all of the necessary vetting and security. Everybody thinks that they’re going to put their code in the cloud, but it doesn’t necessarily have to be that way. You could do it in the cloud or on-premises.

In terms of running GitHub, we always recommend that you have a development environment that is distinct from the production environment, whether they are different physical machines or separate logical partitions on the same system. The idea is that you have separate libraries for development and production. GitHub installs on its own server, and that could be on premises or in the cloud, and even in IBM i shops this is typically run on a system running Linux or Windows Server

Here’s a funny observation from watching IBM i shops adopt GitHub. In general, when you have cutting edge technology, especially open source, some of the smaller companies tend to be a little bit more flexible and agile. They don’t have as many constraints to try new things, and they are the first dabblers in onboarding some of this new open source software. But what we’re really seeing with GitHub is that larger enterprises are leading the way here, because they have many different platforms and it makes sense for them to standardize on a single process regardless of the number and type of platforms they support.

Most of the dialogue we see today is around the idea of a single repository, of having this flexibility of concurrent development and using the branching and merging capabilities of Git to be more responsive to the business. But most companies are pretty pragmatic. They understand that they need to do it in phases. We work in an IBM i environment where SEU hasn’t been changed – I think it’s been deprecated for close to 12 years now and some developers are still working in SEU and they haven’t even adopted Rational Developer for i yet. So there is a lot of different moving parts that will have to change – you need an Eclipse IDE and you need to adjust to the optimistic source model of Git.

For those in the open source world, there is virtually no impact moving to Git because they already work this way. So anybody that’s in PHP or Java is probably using some sort of source versioning, they are probably using Git already in some shape or form. Even the .NET developers with Team Foundation Server or now Azure DevOps today are using that same versioning model.

When you think about what ARCAD is promoting from a DevOps perspective, it is about knocking down all of those silos and setting up a more collaborative environment – across languages, across applications, and across platforms. And Git is a perfect versioning tool to allow that. But as I mentioned, IBM i has some very specific needs. When I talk at conferences, I remind people that we are the marsupials of the mammalian world – we are mammals, but we have a beaver tail and a duck bill. So we have some platform-specific requirements that the open source doesn’t deal with, such as build dependencies. Moreover, Git only understands objects with source, and in our world, we have objects that don’t have source and they have to be managed and handled properly to do a secure and valid deployment. So, those are the gaps that ARCAD fills, which allows you to use an open source tool such as Git through a social development company such as GitHub and get all of the requirements at the enterprise level and still manage all of the platform specific requirements of the IBM i system.

ARCAD is also a very large player on the application deployment side, with our Drops product, which allows us to deploy anything and deploy it anywhere –Windows, Linux, IBM i, AIX, whatever. When you start to standardize on a single versioning tool where you have all of your source regardless of platform, it also helps to have a tool that allows you to orchestrate the movement of the artifacts from Dev to Test to Production ultimately. And at the same time, we integrate GitHub with continuous integration tools, such Jenkins, which orchestrates the application builds. So now you’ve got a place for source, a place to orchestrate builds, and a place to orchestrate deployment.

You can now understand how this approach we have developed at ARCAD would resonate with top-level executives and application development teams because it will allow them to have a process that’s consistent, easier for training new people, and obviously with lower costs. And at the end of it, companies can be more responsive to the business because they can get application changes back in the hands of the end users and we can get their feedback.

The key for Git is to get the right kind of integration and support for IBM i. I know there are people out there who are teaching Git, and these are very smart people, but this is what I would call raw Git. It’s like trying to teach somebody how to work a computer but they can only use DOS commands. So what IBM is providing with Git is raw, and you’re going to have to work a command line, you’re going to have to issue commands with switches. And on a small scale, this is probably manageable with experts that are willing to dabble and learn. On a larger scale, you’re going to want the kind of living conversation that exists in GitHub where you could track through all the changes, all of the talk and dialogue, and discuss the details. It’s this — again –living meeting where you can conduct reviews and easily define or see differences in code, comment in context there, and get that feedback loop and it’s sustained there. You can, for instance, look five years back and see the dialogue as to why a certain code path was chosen based on the dialogue across the team.

Eventually, all IBM i shops will adopt open source technologies, and those that don’t are probably not going to last. And there’s the flipside. We are seeing new people taking jobs at IBM i shops that were not born and bred of this community. None of this technology is foreign to them or frightens them. They want to get legacy RPG or COBOL code into a GitHub repository and manage the source in the same way that they manage open source programs. There are some people that tell me this is all too much technology, that they will never use it. I don’t know how long these people will stay viable at their companies. If you’re doing anything customer-facing, I can’t see how you can survive without having some sort of hybrid development and the tools and culture to handle all of that rapid code change.