If You Aren’t Automating Testing, You Aren’t Doing DevSecOps

October 17, 2022 Andrew Clark

If you are going to automate application development and operations like a hyperscaler, it is not as simple as moving code to a Git repository and then you are done. Security is critical to DevOps – so much that people are now calling it DevSecOps to highlight that fact. The “Sec” in DevSecOps impacts several phases in the development lifecycle – from code quality and security checking to developer-side unit tests and, further downstream, functional regression tests. Provided they are automated, each technique brings major savings in development by “shifting defects left.” In this article, we’ll explore how.

Today, if you aren’t automating your testing, then you are not really doing DevOps in an automated way like the Googles and Facebooks and Amazons of this world. Yet – even in the IBM i market – all the tools exist for you to automate all aspects of the application pipeline.

If you are going to automate your DevOps cycle, you need 100 percent automated unit testing. There’s no ifs, ands, or buts about it. If you are using third-party software – whether it is created by a commercial software house or through an open source software project – you have to be aware of the security considerations with regard to any code that does not originate with the IBM i system and your own application development organization. There are a ton of holes in some third party and open source products, which can cause big problems in the development process with regard to security. The security holes in the Apache Log4j logging utility are but one of hundreds of examples that surface every year in open source projects that are either directly used in the IBM i stack or are popularly employed by IBM i shops or their software partners.

The mindset that IBM i shops – and indeed all IT organizations – need to adopt is simple, and it is what we call test-centric development. This is, as that term suggests, where you think of testing as a priority in the development cycle, not as an afterthought. What this means is very easy, but it can be very hard to organizations to take seriously at first: For every single business function implemented in the code, there has to be a test case to prove that it is working properly and securely. There are very few IT departments that do this, but the hyperscalers certainly act this way.

It’s not like IT departments do not do testing. They put together manual scripts that will run against new code when it gets to the quality assurance part of the development cycle. The problem is, when you do the testing at this point, you can catch the defects, but you are doing it much later in the DevOps cycle when it is much, much more costly to do so. If a defect costs 1X to fix immediately after the code is created and has been automatically tested, it costs 640X as much to fix when the code is in production and the end users find it.

This situation, which is entirely due to the lack of automated unit testing, is completely avoidable. And this lack of automated testing is the major bottleneck in the DevOps cycle and one that has obvious return on investment when it is fixed.

The major bottleneck is that most people don’t have their business functions in self-contained procedures, even though they should because that’s the best way to maintain the code, and create readable and usable code. Most programmers don’t write code this way, so the business logic gets mixed in with other code. But the reality is, if you can isolate the business code and the business function into a single procedure, then it’s simple to write a test case. You know what the inputs are, you know what the outputs are, and you can run the tests when it is cheapest to fix any flaws. If you have a rule that knows which unit tests fit to which pieces of code, then then you can automate the process, and that means you don’t have to run your entire suite of test cases. You can just run the ones that are actually affected by any changes that are made.

So having code that’s constructed in a way that makes it easy to test, and then actually having the test cases for those business functions, is step number one and the most critical step that most people do not take. The hyperscalers do it this way, which is why Amazon, for instance, does 1,000 software releases per hour. They have purely automated release cycles, and purely automated testing of code right after it is coded. You don’t have to be a hyperscaler like Google or Amazon or Facebook to do this the right way.

The problem is that we have this history of bad, monolithic code – millions of lines of it at hundreds of thousands of customers. This code was written before there was freeform RPG, and it is monolithic because that was the only way to create it at the time. You have to work at it, and refactor it to make it modular. But for all new code, there is no excuse. If you are writing new code that is not modular, and therefore testable, then shame on you. If there is any manual process in your DevOps pipeline, then you are not really doing DevOps.

The IBM i base has a long way to go with DevOps. According to survey data from 2021, two-third of organizations polled said they were either high performers (doing releases every day), or elite performers (doing releases ever six hours). In the IBM i world, I would guess that fewer than 5 percent of customers are high or elite DevOps performers, and ARCAD Software has a lot of these customers that are high and elite performers.

According to our ROI calculator, which is based on our experience, companies implementing a true DevSecOps pipeline save about $30,000 per developer per year by doing their testing correctly – way early in the cycle. We are talking about significant money here. But equally importantly, you want to automate testing because developers don’t want to do it manually. They want to write new code that drives the business, and these manual testing processes do not make them happy, and also result in much higher debugging and testing costs.

To evaluate how to automate unit testing and code quality as well as security checking in your environment, download our free software trial today:

[Free Trial] ARCAD iUnit – Unit Test automation for IBM i

[Free Trial] ARCAD CodeChecker – Code Quality & Security checking for IBM i

This content was sponsored by ARCAD Software.

This Issue Sponsored By

Table of Contents

Content archive

Recent Posts

Subscribe

Pages

Search