Power9 Big Iron “Fleetwood/Mack” Rumors

October 9, 2017 Timothy Prickett Morgan

Sometimes, the universe just hands you a great title, like that one above. As the Power9 platform is rolling out across the field to various launch pads, with entry, midrange, and high-end systems in the works, word always inevitably leaks out about what Big Blue has planned with its future systems.

In the case of the high-end machines, we are hearing rumors that the current top-end Power E880 and its half-pint Power E870, which went by the code-name “Brazos” within IBM, likely in deference to the river in New Mexico, will be replaced by a machine code-named “Fleetwood,” no doubt relating to the vintage Cadillac sedan, and its companion SMP and system controller, code-named “Mack,” after the truck company, I presume. If history is any guide, then the combination of one to four “Fleetwood” server nodes and the “Mack” controller will comprise what I am hearing will be called the Power E970 and Power E980. These systems will represent the upper echelons of single system scalability and will very likely be among the most powerful, in terms of aggregate compute, and capacious, in terms of physical memory, NUMA systems on the market.

While there are not a lot of IBM i shops that need the capacity of a very large Power Systems machine, there are some and the continuation of a long – and impressive – line of big iron Power-based servers is necessary for some and comforting to many. One of the issues with the System/38 and many early generations of AS/400 systems is that the processing and memory capacity of the top-end machines was never quite enough for the biggest jobs, and server consolidation and a huge number of mergers and acquisitions across all industry sectors put even more pressure on IBM to engineer faster processors and more scalable systems in the 1990s. And, to its great credit, Big Blue did this brilliantly and pushed many competitors out if its markets while at the same time giving its AIX and OS/400-IBM i customers increasingly better bang for the buck.

It is not at all clear what the price/performance curve will look like once the Fleetwood Power9 systems are announced, mainly because we do not have a good sense of pricing for Power7+ and Power8 machines that have eight or more sockets. IBM has been cagey about revealing pricing information for these systems, and we don’t expect that to change. What we do expect is that pricing will come down a little, but capacity will increase by a lot more, and the overall price/performance will be attractive to customers who invested in Power5-based i5 595s, which were launched in October 2004, and Power6-based i5 595s, which launched a few years later, as well as the Power7-based Power 795s, the last generation of integrated, scale-up systems that used a high-speed backplane rather than the multi-node approaches that IBM first debuted with the Power5-based eServer i5 570 way back in 2004. The Power 795s came out in August 2010, quintupling performance over their Power 595 predecessors, and they are getting a little long in the tooth where they are still installed.

The blessing and curse of the Power server line, regardless of the name that IBM has slapped on them, is that they are reliable and their technical life often outstrips their economic life. The good news is that IBM has a fairly large base of customers who have such iron and who are likely to be looking for an upgrade in the Power9 cycle, and maybe even a few Power8 shops might be encouraged to upgrade, particularly since IBM has said it will allow upgrades from Power8 to Power9 for big iron boxes. (The smaller machines are not upgradable, and have not been for years.)

The thing to remember is this: No matter if the machine is running IBM i, AIX, or Linux, or a mix of them, if a company is investing heavily in big Power Systems iron it is helping all customers who invest in Power Systems, since this is where IBM derives a large portion of its revenues for Power Systems and an even higher proportion of the product line’s profits. So, even if this doesn’t matter to you directly, it matters to you indirectly. And besides, these big iron machines are just plain interesting.

While IBM has not said so, I am beginning to think that the Power8, Power8+ and Power9 processors are socket compatible, and if not, the sockets are not all that different, with perhaps more I/O pins on the later Power9 chips to allow for the substantial increase in I/O bandwidth. Thanks to this, I believe that the Power9 server line will look a lot like the Power8 server line, and the rumors we are catching wind of certainly indicate that this might be the case with the 16-socket future Power9 system, which is very likely to be called the Power E980 as we pointed out above.

From what we are hearing, IBM is expanding the memory capacity and I/O capacity of the systems, thanks to fatter DDR4 memory and the move to faster PCI-Express 4.0 peripheral interconnects. But we think the new machines will bear more than a passing resemblance to the existing Brazos Power E870 and Power E880 systems, which came out in October 2014 and were subsequently tweaked with some I/O and memory enhancements in September 2016 with the Power E870C and Power E880C models. This, by the way, is when I think IBM had originally intended to do a Power8+ refresh but decided not to.

Here are the feeds and speeds for the most important generations of high-end NUMA machines based on Power processors over more than a decade, which shows the technical progression of the machines and the substantial performance that was brought to bear:

The font is a little small, so click on the image to get a full-sized view. Back in the day, IBM had up to 32 sockets in its largest machines, but as the core count on the Power chips rose, IBM decided to cut the number of sockets in half and concentrate on hanging more memory and peripherals off each socket. This allowed the system to run more efficiently and for the performance to be scaled up significantly.

IBM always offers a few different processor SKUs with its biggest Power iron, and we expect this to be no different with the Power9 systems. We are guessing, just for fun, that IBM will use some of the process shrink from the 22 nanometers used with the Power8 chips to the 14 nanometers that the Power9 is etched in to not only cram more transistors on the chip, but also to crank the clocks a bit. The architectural changes between the chips and the radically increased bandwidth and interconnectivity on the NUMA links with the Power9 systems are all going to contribute to a big performance increase. Add it all up, and we think the performance, as gauged by IBM’s Commercial Performance Workload (CPW) test could rise by around 60 percent between the Power8 and Power9 generations of the big iron boxes. Depending on the number of cores and their clock speeds, the big Power E980 should deliver somewhere between 2.5 million and 3.3 million CPWs. We are assuming that IBM only plans to deliver twelve-core “Cumulus” Power9 scale-up chips in these big NUMA systems – with what it calls SMT8 threading, or eight threads per core, as the IBM i and AIX operating systems and the PowerVM hypervisor expect – and it will not use the 24-core “Nimbus” scale-out processors that are aimed at two-socket servers. As such, we are only expecting about a 30 percent performance improvement thanks to architecture tweaks and then another 30 percent or so from clock speed hikes and improved NUMA efficiency thanks to the new interconnect.

Here is what that updated NUMA interconnect used the Fleetwood Power E970 and E980 looks like:

With this improved NUMA interconnect, the SMP X buses, which are not properly named because this is a NUMA architecture but whatever, connect four processors on a system board together, and the SMP A buses are used to hook multiple server nodes to each other up to the four node maximum with the NUMA architecture has used for a very long time in these enterprise-class systems. The SMP X buses are going to run at 16 Gb/sec. twice as fast as the SMP X links used in Power8 iron, and the SMP A links are going to run at the same 25 Gb/sec speeds as the other interconnect on the Power9 chip, including the OpenCAPI and NVLink 2.0 ports used to link the processor complex to accelerators. In fact, as the Fleetwood system scales up, customers have to burn OpenCAPI and NVLink ports to do it. We think the increase in east-west bandwidth on the Fleetwood motherboard and in the north-south bandwidth across multiple Fleetwood nodes is going to do a lot to increase NUMA performance. Hence, my aggressive performance expectations.

IBM has a habit of beating my expectations in this regard, I know. So the CPWs could go even higher.

While the Power9 architecture can, in theory, support up to 8 TB of memory per socket, or 128 TB across a machine with 16 sockets, I don’t think IBM will push it that far on production systems. IBM always wants to keep a doubling of capacity in its back pocket for an RPQ special upgrade, and that means the main memory out of the chute on the Power E980 system will probably be 64 TB. That is twice that of the Power E880 using the Power8 chips. Intel can only do 1.5 TB per socket with its “Skylake” Xeon SP processors, and the largest systems available today using any Xeon processor, which are the UV 300 systems from Hewlett Packard Enterprise (formerly SGI), which scale up to 64 sockets and 64 TB of main memory. IBM does not have to push beyond this to be competitive, and given how expensive memory is, it is highly unlikely anyone will buy the fattest memory and fill up any server. It is good that these machines are coming out in 2018, when memory prices might return to something resembling normal.

I am flat out guessing about how the I/O will be configured in the Fleetwood system, but it stands to reason that with 60 percent more compute and twice the memory, the I/O should be increased bot in the number of slots and in the bandwidth per slot. The PCI-Express 4.0 slots will be particularly important for flash storage that employs the lean NVM-Express protocol, which substantially boosts I/O rates and lowers latency on flash. I expect for IBM to have NVM-Express slots in the Fleetwood machines, and indeed in all of the Power9 systems. This is a requirement these days. PCI-Express 4.0 uses 16 Gb/sec signaling, double that of the PCI-Express 3.0 slots that are in many prior generations of Power Systems machines and matching the SMP X buses that link processor sockets on a single Power9 motherboard. (That is not a coincidence, I think.)

So, that is what I am hearing about Fleetwood and how I think it will play out. The May 2018 launch date is just a guess, based on nothing more than a feeling in my bones.

This Issue Sponsored By

Table of Contents

Content archive

Recent Posts

Subscribe

Pages

Search