Drilling Down Into The New Power9 Entry Servers

February 19, 2018 Timothy Prickett Morgan

In last Wednesday’s issue of The Four Hundred, we gave you a high-level overview of the six new Power9 entry servers, code-named “ZZ” by IBM, as well as an initial pass on the changes that came with the latest Technology Refreshes for IBM i 7.2 and 7.3. If you haven’t read these, please do, because they give you information on IBM’s strategy with regard to the Power9 iron and the IBM i platform.

In this issue, we are going to drill down into the six new Power9 systems, taking particular care with the single-socket Power S914 and dual-socket Power S924 machines that can meet the data processing and storage needs of the vast majority of IBM i shops. It is these machines that will be the platform for business for IBM i shops for the next three, four, or maybe even five years, depending on when they upgrade and depending on how far out the Power9+ and Power10 chips might be and how compelling they are – or are not – to customers.

We have been waiting to get into the feeds and speeds and pricing of the new machines for years, and now the time has come. IBM will formally unveil the machines at the upcoming OpenPower Summit in Las Vegas on March 19 and throughout the Think 2018 partner and customer conference being held in the same venue from March 19 through 22, but a lot of data about the machines is available in announcement letters and other presentations if you do some digging.

So, with that, let’s get on with it.

The best place to start is with the compute aspect of the new ZZ machines. We have gone over the distinctions between the “Nimbus” and “Cumulus” Power9 chips and the differences between the skinny SMT4 (four thread) and fat SMT8 (eight thread cores for either style of chip. (See Power9 Gets Ready To Roll In Systems In 2017 for all the details on the Power9 architecture. In general, the Nimbus chips have only two X Bus interfaces running at 16 Gb/sec that implement the NUMA interconnect between two sockets in a glueless fashion. (Meaning, all of the electronics for that NUMA connection are on the chip and do not require an external chipset.) For the Cumulus versions of the Power9, some of the I/O ports that are used to provide advanced OpenCAPI or NVLink ports to link to peripherals in a coherent fashion with the Power9 processors are used to provide more NUMA interconnects, allowing for four, eight, twelve, or sixteen sockets (really groups of four-socket machines, as we explained in talking about the future “Fleetwood” Power E970 and Power E980 systems back in October last year.) The expectation is that only the SMT8 implementations, with 12 cores per socket, will be done with Cumulus chips, but IBM could offer SMT4 variations with 24 cores per socket if there is customer demand. The bigger boxes will have IBM’s “Centaur” buffer chips on them, which will allow twice as much memory to be added per socket (4 TB) compared to the unbuffered memory used on the Nimbus Power9 chips (which top out at 2 TB per socket).

So, here is how the compute stacks up on the six new ZZ machines:

You know the drill on IBM’s system and feature naming conventions. The S means it is a standard, scale out machine that can run IBM i, AIX, or Linux atop the PowerVM hypervisor; the L means it is Linux only, and the H, which is new, means it is a Linux system that is tailored to run SAP HANA in-memory databases and their applications. The H series machines can also host IBM i and AIX on logical partitions, but IBM is requiring that no more than 25 percent of the aggregate performance of the systems can be dedicated to either (or both) of these operating systems. The 9 is for Power9 processors, of course, and the following number (a 1 or a 2 in this case) designates the number of sockets in the machine. The final number, in this case a 2 or a 4, tells you the size, in industry standard rack units (which are 1.75 inches high and 19 inches wide), of the chassis that the machine is based upon.

All of the Power9 chips used in these ZZ systems are single chips, not dual chip implementations that share a single package and present themselves as a single processing complex. Apparently someone (Oracle, you know who you are) had discriminatory pricing against DCMs, so making the Nimbus and Cumulus chips all one die fixes that issue. As you can see, IBM is offering three different processor options for these ZZ systems for each model of the machines. Some of them are the same across different series, but generally, they are not. The Power S914 has totally different chips from the other machines, and only the Power S924 and the Power H924 machines use the same literal processor features. The table shows their names, by feature numbers. As we explained last week, IBM is no longer charging for processor activations; they come for free and once you buy the processor card, they are activated. It is interesting to note that IBM is still keeping the processor core activations in the process, presumably should it decide to change its mind, it can go back to charging for them separately.

The Power9 chip has dynamic frequency optimization, and IBM has given some guidance on what the low-end and high-end of the clock speeds are for the processors. The low-end of the frequency range varies a lot, but the high end pulls into a tight range of between 3.8 GHz and 4 GHz. IBM has given a bunch of thermal ranges for the Nimbus Power9 chips used in the ZZ systems, and we have done our best to guess what the low and high heat dissipations are. (IBM could just provide this and be done with it.)

The Power S914 and Power S924 machines are based on the same core design, with the big difference being that the Power S914 has only one processor and its various peripheral controllers and memory slots and the Power S924 has two processor cards and therefore expanded I/O. The Power9 socket without buffer chips has two memory sticks for each of the eight DDR4 memory controllers on the die, for a total of sixteen slots per socket. With the four-core variant of the Power S914, which is aimed at IBM i customers in the P05 software tier, the machine tops out at 64 GB and has no disk or flash expansion enclosures. The Power S922 and its two variants have two sockets, but because they are implemented in a 2U form factor, there is less storage and fewer peripheral slots. Here is how the storage stacks up:

We will get into how this is implemented in a second, but this is just a handy dandy reference summing up the key salients. One other thing before we look at the system boards and mechanical designs for the machines. IBM is offering four different memory cards, which come in capacities of 16 GB, 32 GB, 64 GB, and 128 GB. The 16 GB sticks run at 2.67 GHz and the others run at 2.4 GHz – and here is the important part – until you have ten or more memory sticks per slot on the system. Then, because the memory is not buffered, it has to be geared down to 2.13 GHz no matter what the capacity is. We will be talking a look at how this memory is different from prior Power7 through Power8 memory, in terms of capabilities and pricing in a future story.

Now, let’s take a look at the system boards and the mechanical designs of the three different types of ZZ systems. First, here is what the Power S914 looks like, as rendered by IBM’s hardware engineers:

You can see the Power9 processor on the front left, with the heat sink that looks like an office building skyscraper in midtown Manhattan. There are four banks of memory, plus two internal PCI-Express 3.0 slots that can take NVM-Express m.2 flash devices or a pair of RAID disk controllers. As you can see from the mechanicals, there is a tower and a rack version of the machine, which can have 18 2.5-inch drives or 12 2.5-inch drives plus an RDX backup media device.

Below is a block diagram of the system board, showing all of the key components and peripheral slots:

The Power9 chip has four PCI-Express 4.0 peripheral controllers, and two of them are used to hook into PCI-Express 3.0 switches that in turn fan out to PCI-Express slots with varying lanes and bandwidths. This machine has two PCI-Express 4.0 slots, both of which are mechanically x16 slots (meaning they have 16 lanes of bidirectional bandwidth bundled up for them) but one of them has only eight lanes of actual I/O on it. Both of the PCI-Express 4.0 slots have CAPI 2.0 coherence on them as an option, and there is at the top the single 25 Gb/sec OpenCAPI port. One of the PCI-Express 3.0 x8 slots is burned up for the base Ethernet, as you can see, which is a four-port adapter running at a ridiculously slow 1 Gb/sec speed. The two internal storage controllers or NVM-Express ports hang off the PCI-Express 3.0 switches.

Moving over to the Power S924, it is really just a Power S914 with a second processor card in it. (Albeit, with different Power9 options, with more cores and higher clock speeds.) The second processor brings another sixteen memory slots and more I/O, as you can see in the Power S924 block diagram:

You can see the pair of 16 Gb/sec X Bus NUMA ports linking the chips into a shared memory NUMA cluster; these X Bus ports run at twice the speed of the 8 Gb/sec X Bus links used in the Power8 and Power8+ systems. The second processor doubles the memory capacity as well as the number of PCI-Express 4.0 ports (four out of five of which are CAPI 2.0 enabled) and OpenCAPI ports. The extra processor does not mean there is not room for in-chassis disk or flash storage. It has the same options as the Power S914, as you can see here in the mechanicals for the Power S924:

And just for fun, here is the one picture of the Power S924 that IBM has put out:

To make a Power S922, IBM has to take most of a Power S924 and cram it into a 2U chassis instead of the 4U chassis used in the Power S914 and Power S924. That means some storage and peripheral expansion has to be sacrificed, and in this case, you lose ten drive bays or four drive bays and the RDX device and you lose two PCI-Express ports. This seems like a fair trade where what you want is the density of compute, as you might for certain Linux or AIX applications, or when building a cloud.

You can see the tighter hardware squeeze here:

The Power9 heat sinks are half height, and so are the PCI-Express slots. There are six external drive bays and two internal ones, and there is still space for the pair of NVM-Express M.2 cards.

Here is what the Power S922 system board block diagram looks like, because I know you are nerds like me:

That leaves pricing. I have gathered up the basic pricing just as a starting point from the announcement letters for each of the six machines. The base Power S914/S924 chassis is the same without anything in it, but for some reason when it is a Power S914, the base machine costs $3,150 and for the Power S924 it costs $9,150. The base Power S922 chassis, which has less stuff and less room, costs the same as the Power S914 base chassis at $3,150.

To this, you add the processor, memory, storage, and I/O features. I have shown the cost of each processor card, and they range from a low of $1,815 for the Power9 card with four 2.3 GHz cores to a high of $28,322 for a card with a Power9 that has a dozen 3.4 GHz cores.

For the configured systems shown, it is the cost of the maximum number of processor cards in the system (remember all of the cards are activated) plus the base chassis; to this is added 32 GB of memory per core using the 32 GB memory sticks (this might be a little high for IBM i shops) plus four 600 GB SAS disk drives. No other peripherals are added to this. What you get when you do this is a line of machines that range in price from under $13,000 to nearly $100,000, ranging from the P05 to P20 software tiers. I have no idea how much work these different configurations will do yet, and I have not ginned up what real configured systems cost, but this is a baseline from which to start our musings.

If you have any thoughts on this, pipe up.

Next, I will drill down into the Power S914 and Power S924 and how they compare to prior generations of Power Systems iron running IBM i. Stay tuned.

2 thoughts on “Drilling Down Into The New Power9 Entry Servers”

Bart says:

February 20, 2018 at 1:40 pm

Tim,
Not sure what you meant by “no cpu activation per core”. CPUs and MEM in Scale out POWER8 servers were never activated per core or a Gigabyte. Whatever was installed , become available for use ( Im not talking about soft licenses). Only enterprise models allow it

Timothy Prickett Morgan says:

February 21, 2018 at 8:38 pm

There was a separate price for the card and then to activate the cores as far as I know, and that is also what IBM said, too. Weird. I always priced with all cores activated, and just assumed you had to option to not activate them all if you wanted. I realize that you don’t have to put IBM i on each core, of course. In any event, the core activations now have a $0 price, and the processor card prices are the same or higher, and in some cases, lower than you might have expected based on the old way of doing it.

This Issue Sponsored By

Table of Contents

Content archive

Recent Posts

Subscribe

Pages

Search