I/O, Memory Boosted On Entry, Enterprise Power Systems

October 17, 2011 Timothy Prickett Morgan

I can sum up IBM‘s entry Power Systems announcements on October 12, which are not being formally announced until sometime this week, in a way that will flash you back to either your own or your parents’ childhood. This is the refrain of the theme song from The Power Systems Show: “They’re servers, nearly identical servers: They look alike, they cost alike, at times they even compute alike–you can lose your mind when computers are two of a kind.”

Aren’t you glad I didn’t sing that? Be grateful that I think a newsletter should be printed and not a multimedia extravaganza.

OK, we’ll try a visual of hieroglyphics that conveys the announcement:

Perfectly clear, right?

I’ll start doing my job now and stop messing around. As expected, IBM has rolled out six new Power Systems servers as part of its October announcements. There are four new entry Power Systems servers that will eventually replace the existing Power 710, 720, 730, and 740 machines that were announced last August. The four new machines sport new motherboards that support PCI-Express 2.0 peripherals slots and have an extra slot in them as well in addition to a dedicated slot for an additional Ethernet controller in addition to the dual-port Gigabit Ethernet controller on the entry Power Systems main boards.

And equally important for big database jobs and increasing virtualization density on these entry machines, the new systems offer the ability to use 16 GB DDR3 main memory modules, thereby allowing for the system memory on the new machines to be twice as large as on the earlier machines, which can only support 8 GB sticks. (You can use 2 GB, 4 GB, or 8 GB sticks in either the old or new entry Power machines. You don’t have to use 16 GB sticks.)

These new entry machines now have five PCI-Express 2.0 slots, which have a signaling rate of 2.5 GHz and a per-land bandwidth of 500 MB/sec. That is twice the rate and bandwidth of the PCI-Express 1.0 slots used in the original Power 710, 720, 730, and 740 machines announced last August (and indeed, used across the entire Power7 line that was announced throughout 2010). So these x8 peripheral slots now support 4 GB/sec of aggregate bandwidth per slot (or 32Gb/sec with some overhead for encoding). This latter number is important because an InfiniBand NIC running at 40Gb/sec or 56Gb/sec can easily choke a PCI-Express 1.0 x8 slot, and so can a GPU co-processor. Because of a move to a much more efficient encoding scheme, PCI-Express 3.0, which will debut first in Intel’s “Sandy Bridge-EP” Xeon E5 processors for two-socket servers, will approximately double peripheral slot bandwidth even though the aggregate bandwidth for this spec is only increasing by 60 percent.

The extra main memory is welcome, of course, particularly for jobs that are more memory-constrained than CPU-constrained. But you will pay for it. A pair of 16 GB sticks running at 1.07 GHz will run you $6,390, or $199.69 per GB. IBM is only charging $33.25 per GB for 2 GB, 4 GB, and 8 GB memory sticks for these four new machines. By the way, Oracle is charging $112.50 per GB for 16 GB memory sticks at list price for its new Sparc T4 systems, a lot less than IBM is doing for the fat memory, but Oracle is making it up on the skinny memory by charging $81 per GB for 4 GB sticks and $102 per GB for 8 GB sticks. And for further comparison, the Power 520 system from the Power6 generation had 2 GB, 4 GB, and 8 GB memory sticks that cost $260 per GB.

Let’s start from the bottom and work our way up. The new Power 710 machine is the 8231-E1C, and it is a single-socket box that comes in a 2U chassis. This machine has eight memory slots, for a maximum of 128 GB of main memory, and has eight low-profile PCI-Express 2.0 x8 peripheral slots. The machine has six drive bays, which can support 2.5-inch SAS or SSD drives. The machine also has an integrated RAID 10 disk controller on the board (RAID 0 and 1 levels are also supported, but not RAID 5). The machine has three different Power7 processor options: 4-core 3.0 GHz, 6-core 3.7 GHz, and 8-core 3.55 GHz. Yes, these are the same feeds and speeds as the prior Power 710 machines. The 4-core version is in the IBM i P05 software tier and has no 12X I/O loops for remote I/O expansion. The 6-core and 8-core versions are in the P10 software tier and have no 12X I/O loops as well.

The Power 730 comes in the same 2U chassis as the Power 710 and has the model designation of 8231-E2C, just so you can keep track of the new machines. But it comes with a system board with two processor cards installed, with the following Power7 processor options: 4-core 3 GHz, 4-core 3.7 GHz, 6-core 3.7 GHz, and 8-core 3.55 GHz. The machine has two 12X I/O loops (which are created with an adapter from the GX++ slot in the machine) for remote I/O enclosures and supports up to 256 GB of main memory using the 16 GB sticks.

The Power 720 is the workhorse of the IBM i lineup, and the new 8202-E4C machine will continue in that role. The Power 720 comes in a 4U chassis with a single processor card that has 4-core, 6-core, or 8-core Power7 chips running at 3 GHz. (That’s the same as last year’s models.) Main memory peaks at 128 GB for the 4-core system and at 256 GB for the 6-core and 8-core system. The machine has eight drive bays and has an optional four low-profile PCI-Express 2.0 slots that can be added to the five on the system board through a riser card. It also has one GX++ bus slot to implement the 12X remote I/O drawers.

The Power 740 is essentially the same 4U chassis, but you can put two processor cards in the machine and double up the main memory to a maximum of 512 GB in a two-socket configuration. The new Power 740 has the base five PCI-Express 2.0 x8 slots as well as the four additional low-profile PCI-Express 2.0 slots that are optional on the Power 720. The new Power 740, machine number 8205-EC6, has the following processor options, which must be installed in identical pairs in two-socket configurations: 4-core 3.3 GHz, 4-core 3.7 GHz, 6-core 3.7 GHz, and 8-core 3.55 GHz.

Here’s how IBM positions the machines against each other:

The Power 750 midrange machine was goosed with slightly faster Power7 processors back in April of this year, and these machines did not get the doubled up memory or PCI-Express 2.0 slots this time around. The Power 795 just started shipping a year ago and has not been touched since, including last week. Sibley said that the Power 750 and 795 would be enhanced with faster I/O and other stuff in the future, and added that it would not be early next year, either.

The Power 770 and 780 machines, which sit between the midrange Power 750 and the high-end Power 795, were enhanced last week. The three new machines in this class announced last week are based on the same multiple-chassis SMP clustering design that has been the hallmark of the enterprise-class Power Systems since the Power5 generation a zillion years ago.

The Power 770 has from one to four server nodes, which are clustered together into a shared memory system that sports two sockets per node. Last year, the Power 770 had 6-core Power7 chips running at 3.5 GHz and 8-core variants running at 3.1 GHz. This year’s model has 6-core chips revved up to 3.72 GHz and 8-core chips spinning at 3.3 GHz. You have to use the same processor card features within and across nodes in the Power 770–no mixin’ and matchin’ is allowed. The new Power 770 node supports up to 1 TB of memory using 16 GB sticks in each node and the system board has six PCI-Express 2.0 x8 peripheral slots per node, so fully loaded, this baby has 4 TB of memory and 24 base peripheral slots. Each node has two GX++ slots for I/O loop expansion, topping out at 184 slots at the PCI-Express 2.0 speeds. (The I/O drawers can support PCI-X and PCI-Express 1.0 slots, depending on which one you buy.)

The new Power 780 has slightly tweaked processors as well as the double-stuffed memory and PCI-Express 2.0 slots. In MaxCore mode, with all cores activated, the chips now run at 3.92 GHz instead of last year’s 3.86 GHz; in TurboCore mode, with half of the cores shut off, the clocks run at the same 4.14 GHz as last year’s model.

And then finally there is the new 96-core version of the Power 780. This machine scales to four chassis, just like the other Power 770 and 780 machines, but instead of having two sockets per node it has four sockets per node. This variant of the Power 780 can only use 6-core Power7 chips running at 3.44GHz. This machine delivers somewhere between 106,000 and 363,000 CPWs of relative IBM i performance. The top-end 64-core Power 780 from last year was rated at 343,050 CPWs. That’s only 5.8 percent more aggregate performance for 50 percent more cores–and 50 percent higher software licensing costs since IBM i is priced on a per-core basis.

For most IBM i customers, it would seem that this new 96-core Power 780 is a bad idea. In fact, what these customers no doubt would love to see is a 5 GHz Power7+ processor that runs their work faster and takes a lot fewer cores to do it. This 96-core Power 780 machine can host 960 logical partitions and might make sense as a consolidation box for Linux workloads with a smattering of AIX or IBM i.

All of the four new entry Power7 machines and the new Power 770 and two new Power 780s will be available on October 21.

Steve Sibley, director of product management for the Power Systems line at Big Blue, said that the new Power Systems machines were priced the same as the prior models they are most like (excepting the new 16 GB memory feature, of course) and that was intentionally done so business partners could focus on selling a particular machine, not the new or old vintage of that machine.

While this is commendable, giving customers the new stuff at the same price, the other way to look at it is that the older machines should have a lower price because they are less capable in terms of memory and I/O capacity. In other words, if you are shopping now for a Power 710, 720, 730, or 740, if you ain’t getting the new 8231-E1C, 8231-E2C, 8202-E4C, or 8205-E6C entry machines or the new 9117-MMC, 9179-MHB, or 9179-MHC enterprise machines, then you should be getting some kind of compensation for taking the less-expandable machine. I am not sure what this is worth; I leave that up to you to figure out. I’ll be digging through the pricing to see if there are any gotchas.

The seven new Power Systems machines are all support IBM i 6.1 and 7.1, AIX 5.3, 6.1, and 7.1, Red Hat Enterprise Linux 5.7 and 6.1, and SUSE Linux Enterprise Server 10 and 11.