Performance Management: A Personal History, And Turning A New Page

April 10, 2023 Doug Mewmaw

The term “performance management” is broad, comprehensive, and uniquely personal depending upon the system, situation, and experience of the user defining it. I stumbled into the world of performance management early into my career, without even knowing it. As I reflect back on my career, it is also natural to reflect upon the history of performance management.

I thought this reflection might be interesting to not only the old pros out there but also the new guys just coming into the industry. I have been fortunate to be on an interesting journey, and one that is about to change not only my career, but will re-define the practice of performance management in the IBM i market.

My Early Years Of Performance Management – The 1980s

The mainframe was the first system I worked on professionally. I was a young systems programmer and watched my computer elders with awe. Since I was new in my career, I ate up all information I could get my hands on. I quickly learned the language of performance management.

I learned about Million Instructions per Second, or MIPS, a number used for capacity planning to this day, I learned about Customer Information Control System (CICS), also still used today. I also learned CICS applications were mission critical as key company business operations were performed daily and monthly and are measured in this way.

In the summer of 1984, I learned performance tuning and capacity planning on the mainframe. I don’t remember seeing performance management reporting. This facet of performance management is still to come. I do remember my peers constantly asking the end users how their computer experience felt. This became a key indicator to me for how performance is measured in a non-data centered way. If the end user is frustrated by slow processes, or disfunction, the experience feels terrible, mandating improvements. But where to begin looking? This is what performance management used to look like. There was no easy roadmap for finding the problems in a system.

In 1988, I was asked to lead the systems team as the AS/400 platform was just introduced and that culminated in the IBM i platform on Power Systems that we know today. I attended a systems class where the instructor informed the class that systems programmers were no longer needed with this innovative platform. Two hours into class, all terminals went blank. I watched the instructor struggle for an hour as he troubleshot the culprit. As a systems programmer myself, I couldn’t stop smiling as he talked to the systems support team for help. The need for a performance management tool was evident to me.

That year, I learned CL, work management, and all the procedures for successfully running an AS/400 computer system, which were all still in their infancy.

Lesson Learned: There is always a need for a systems support team. Capacity planning and performance management are complex. The end-user experience matters.

The Middle Years Of Performance Management – The 1990s

During this period, I added IBM’s collection services functionality to my toolkit. This concept, which IBM performance experts discussed for years, was created so that system programmers could see how the AS/400 was always performing. I made queries and created monthly performance graphs. It helped the team understand the level at which the expensive new hardware was operating. This was the first step in an official performance management methodology.

Turn the clock ahead to May 1992. At this point in my career, I left the systems team and moved over to the application development side of the house. During this period, I learned the difference between good and bad code. More importantly, I learned how application rollouts affected the system, both positively and negatively. In terms of understanding performance management, this career move was one of the best I ever made.

My next career move was as the systems manager for all platforms. It was the job I coveted for years. It was during this period that software companies started developing performance management tools. These companies utilized IBM’s performance files and created colorful graphs to show core metrics performance. The industry was finally shown a big picture of performance. During this period, I developed a performance report that correlated system performance (CPU, disk, and memory) with changes in the system – answering the question: What is the impact of adding more memory? This effective performance management question began to shape my vision for what I felt was the logical next phase for state-of-the-art performance management. The concept, “Understanding the impact of Change,” was an effective troubleshooting technique and one I’ve been teaching for years. Even though this was a very tedious process to manage, the feedback I received from peers was that this was a very positive addition to the performance management toolkit.

Lesson Learned: Don’t be afraid to get out of your comfort zone. Learning all facets of performance is critical for becoming an experienced system programmer. Measuring all performance components is essential to understanding the entire system. The systems and the applications should be measured daily, weekly, and monthly.

The Middle Years Of Performance Management – Early 2000s

In the early 2000s, companies went from a process improvement culture to a risk avoidance culture. This culture change made performance management more difficult as employees were hands off when it came to tuning a poorly running system.

During this period, companies, including MSPs, were purchasing performance management tools and providing core metric graphs throughout the industry. During many engagements, I witnessed presentations where the presenter had no idea of what they were talking about. One business partner asked me to change a best practice guideline on a graph so that his customer would be happy with the analysis.

Lesson Learned: Perseverance is a good trait when a difficult culture exists. That is, sometimes the performance message takes a little extra time to get through. Merely having a performance management software tool does not make one an expert in performance management.

More Middle Years Of Performance Management – 2005 To 2018

During this period, I had the pleasure of working for a performance management company. This company was dedicated to helping customers with performance management and capacity planning needs. During this period, I learned how to manage other platforms such as AIX and Linux and helped non-IBM i companies understand how to create structured performance reporting. When I left the company, I had written over 500,000 lines of code, proving that the leap of faith to the application side of the house was well worth it.

Finally, I worked with companies (including MSPs) to ensure all parties understood the performance impact when mission-critical systems were being moved from on-prem to the cloud.

Lesson Learned: There is value in being a customer first. Collaboration with other people in the industry is essential. From a software management tool standpoint, the best ideas often come from people in the trenches. Finally, when a system is moved to the cloud, analysis is still needed to understand system performance. Every cloud vendor operates their server in a different way.

My Recent Years Of Performance Management

After working for corporations since 1983, and realizing the void in experienced performance analysts, I decided to start my own performance management company. Building upon my years of performance management experience I was ready to help others get the most productivity out of their systems. I understood what it took to show intelligent and succinct reports, how to drill into the culprits causing the systems to glitch, and most importantly, how to convey this information to decision makers. Then Covid happened.

While the pandemic contributed to my decision to go to a plan B, I will never regret the choice I made to follow my passion for performance management.

I now find myself in collaboration with two incredibly talented colleagues, and together we started a company called Greymine. We focus on system and applications performance and services. We have created internal programs to expedite our performance management analysis processes. We have never stopped thinking about the industry nor the customer perspective. In the coming weeks, we will be making a major announcement that will add another chapter to the history of performance management.

Lesson Learned: Change is hard. Sometimes a “mistake” in a career is a stepping stone to something bigger. I recognize there is a void in the IT world for a dedicated performance management company. That is about to change. Stay tuned for a big announcement.

Doug Mewmaw is vice president of performance services at Greymine.

This content is sponsored by Greymine.

RELATED STORIES

Relieving The Anxiety About IBM i Performance

Melding System Monitoring And Capacity Planning

Where Did My Faulting Guidelines Go?

Is Perception the Reality?

Memory Management: It’s Your Fault, Now Fix It

Using i5/OS Performance Adjuster to Better Manage Memory

SSD Performance: Be Careful Before You Buy

Why Are Systems Programmers Always To Blame?

AS/400s Are From Rochester, RS/6000s Are From Austin

System Performance Management Is Like Having Insurance

The Law of Attraction

Thermometer Money: Changing a Business Partner Paradigm