Surprise! Power7+ Chips Launched In Flex System p260 Servers

November 26, 2012 Timothy Prickett Morgan

Big Blue was pretty clear that it was not going to roll out any more servers using its Power7+ processors when it launched the Power 770+ and Power 780+ enterprise-class boxes back on October 3, but somewhere along the way, someone either changed their minds inside of IBM or didn’t get the memo that the Flex System p260 server nodes would be upgraded with the Power7+ chips. Because that, in fact, is what happened on November 13, just before we took a break for the Thanksgiving holiday.

And those new p260 machines give IBM i and AIX shops one more thing to think about as they are figuring out how to dispose of whatever “budget flush” they may have or what their plans might be for 2013.

In announcement letter 112-169, you will find three new Power7+ processors have been slipped lovingly into the two sockets of the p260 server node, which was announced back in April using Power7 chips. (And, I think, was supposed to be announced with Power7+ processors sometime in February or so, no matter what IBM has said publicly.) The Power7 and Power7+ chips are socket-compatible, and as I have explained in a series of stories throughout the summer and fall (see the Related Stories below), Power Systems shops have a lot to think about as they consider their options. The Power7+ chips have more L3 cache per core, on-chip accelerators for memory compression, encryption, hashing, and other functions, and slightly higher clock speeds.

The cache and clocks are particularly important for IBM i shops, who license software by the core, who have big batch jobs where cache and clock speed matter more than other factors in terms of systems performance, and who should take a hard look at the Flex System p260 server node even if they have no intention of moving to PureFlex modular systems. The reason is simple: what is going into the p260 node will likely end up in a Power 720+ and Power 750+ server next year.

The p260 node was launched with a four-core Power7 chip cranking at 3.3 GHz, an eight-core chip running at 3.2 GHz, or another eight-core chip running at 3.55 GHz. The new Power7+ chips run at higher clock speeds, SKU for SKU, and have more cache and those accelerators as well as other microarchitecture tweaks to improve performance, so it is hard to say how much more performance per node using the Power7+ chips will yield. This is particularly difficult given that IBM has not yet released Commercial Performance Workload (CPW) ratings for IBM i and Relative Performance (rPerf) ratings for AIX yet. But it doesn’t take a genius to realize that moving to a four-core Power7+ chip running at 4.09 GHz, an eight-core Power7+ chip running at 3.61 GHz, or an eight-core Power7+ spinning at 4.12 GHz will get more work done. If clock speeds alone were the determinant of performance, the boost would be 23.9 percent, 12.8 percent, and 16.1 percent, respectively, for those three different Power7+ processor options. The bump should be higher given that the Power7 chips have a wonking 80 MB of L3 cache, compared to 32 MB of the Power7 chip.

To balance that extra performance in the p260 node, IBM is also rolling out a new 32 GB DDR3 low-profile memory stick, which can be used across the two-socket p260 and four-socket p460 server nodes in the Flex System. That doubles up main memory on the p260 node to a maximum of 512 GB. But to use that low-profile memory, you can’t use 2.5-inch disk drives inside the server node, which are lashed to the underside of the top cover of the server node and which nestle into the space above the memory sticks. (The Power7 and Power7+ chips are so much larger and, at 190 watts, hotter than Intel’s Xeon chips that IBM can’t put front-mounted disk drives in the nodes.) If you want to use disk drives for local operating system storage on the node, then you need to use very low profile (VLP) DDR3 memory sticks, and at the moment IBM is only shipping VLP memory in 4 GB or 8 GB capacities. With 16 slots per node, that means you top out at 128 GB per node, a quarter of what you can push it to when you use smaller 1.8-inch SSDs for local node storage.

But there’s a catch. A 300 GB 10K RPM SAS disk costs $309, a 600 GB unit costs $619, and a 900 GB disk costs $1,225 at list price. One of those 1.8-inch SSDs, which has 177 GB of capacity and admittedly has a whole lot more I/O capacity, costs a whopping $4,400. This is all IBM’s way of saying store your IBM i, AIX, and Linux operating system images on the Storwize V7000 array that is part of the Flex System setup. Whether or not that makes economic sense is another issue.

Back to the new Power7+ processor nodes. The base 7895-23X server node costs a miniscule $49. The diskless top cover costs $47, and if you want to add the interposer for disks, slap in another $768, and if you want those SSDs, then get out another $1,288.

Now, pick your processor. A feature EPRD motherboard with two 4.09GHz chips costs $8,603, and to activate all of the cores on the board costs another $800 for a total of $9,403. If you want more aggregate performance (but slower cores, which makes no sense for IBM i shops), the feature EPRB board with two eight-core Power7+ chips running at 3.61 GHz costs $15,409, plus another $1,600 to activate the cores for a total of $17,009. The feature EPRA board has two eight-core chips running at 4.12 GHz and costs $17,524, plus another $1,600 to activate the cores for a total of $19,124.

You are going to need some memory, of course, and mezzanine cards to link the nodes to the Flex System midplane and then on out to switches in the Flex System chassis.

A pair of VLP memory sticks with 8 GB of capacity costs $639, and a pair with 16 GB of capacity costs $850. Go with the fatter sticks here. If you are going diskless or with the flash drives, then a pair of 32 GB LP sticks with 64 GB of total capacity costs $6,400, but if you step back to a pair of 16 GB sticks, it only costs $1,700. IBM is charging a hefty premium for those 16 GB memory sticks. The 16 GB sticks cost $53 per GB, compared to $100 per GB for the 32 GB sticks.

A dual-power Ethernet card that supports InfiniBand’s Remote Direct Memory Access (RDMA) over the converged Ethernet protocol running at 10 Gigabit speeds, known as RoCE and yielding significant latency improvements over regular Ethernet, costs $2,100. A quad-port Gigabit Ethernet adapter costs $695, a quad-port 10GE adapter costs $849, and an eight-port 10GE adapter that snaps into the server node costs $3,000. IBM is also peddling InfiniBand adapters running at QDR (40Gb/sec) speeds, and a two-porter costs $1,885. A Fibre Channel mezzanine card that can link the node to the midplane at 8Gb/sec speeds and on out to Storwize arrays (which also can link over Ethernet if you want that) costs $935.

The new p260 node requires IBM i 6.1 or 7.1, AIX 6.1 or 7.1, and the most recent Linux released from Red Hat and SUSE Linux. IBM said in a statement of direction that it would support AIX 5.3 at Technology Level 12 at some point in the future. The new node will be available on December 3, and will be included in PureFlex preconfigured “infrastructure clouds” starting December 10.

In a related move, in announcement letter 112-176, IBM is expanding the node types that are available in those PureFlex setups. Up until now, you had to get stacks with the x240 nodes, which are based on Intel’s Xeon E5-2600 processors. Starting December 10, the p260 and p460 plain vanilla nodes and the Power Linux p24L Linux-only nodes can be configured into PureFlex stacks. IBM is also allowing customers to go with the x220 nodes, based on the Xeon E5-2400 processors, and the x440 nodes, based on the Xeon E5-4600s, in the PureFlex machines.

One more thing. The p460 did not get Power7+ processors. It’s probably a yield thing. It seems likely that very similar, if not the same, SKUs will be available in the coming months on the p460 node.