IBM Is Prepping Power7+ and Pondering Power8

May 2, 2011 Timothy Prickett Morgan

It has been more than a year since IBM got its first Power7-based machines out the door, and about six months since the chips were fully ramped across the Power Systems lineup. The server processor racket waits for no one, and a slow poke will quickly get left behind in the volume and midrange space. And so Big Blue has to continue to advance the Power chips if it wants to get all of those systems, software, and services revenues these chips drive.

IBM has not been exactly forthcoming about the future Power7+ and Power8 processors, but the company is at least admitting publicly that there will be a Power7+ processor and that it is working on Power8 chips. And hopefully it will be a bit more interesting than the Power6+ chip, which came out almost exactly two years ago and offered a modest speed bump over the Power6 chips.

The cadence of Power chip announcements has not been particularly regular, and the plan was for it to be as regular as Intel likes to pretend it has always been–and it most certainly not has been–with its Xeon server chips with their “tick-tock” schedule. The tick is a major processor redesign and the tock is a process shrink that enables clock speeds to be boosted and some features (like cache memory) to be expanded. In the wake of the dual-core Power4 chips, IBM had planned for a similar tick-tocking, and on a 12-month schedule.

IBM’s plan at one point was for Power6 in 2006, Power6+ in 2007, Power7 in 2008, and Power7+ in 2009. But when the former Sun Microsystems (now part of Oracle) started slipping with its UltraSparc chips, partner Fujitsu did the same with its own Sparc64 chips, and Intel started flubbing the Itanium line, IBM took its foot off the Moore’s Law gas and put the Power chip on a new rhythm of a new chip every three years or so with a “plus” bump in between.

Here’s a historical view of the Power chip family, moving from the Power1 implemented in IBM’s 1 micrometer chip baking processes up to the Power7 implemented in 45 nanometer processes:

This is the latest-greatest public roadmap for the Power chips, which as you can see is a little short on details for Power8 and which doesn’t even mention the plus versions of the chips:

Here’s another one that IBM’s marketeers and engineers in the supercomputer racket have been passing around:

I like this roadmap because it at least admits there are Power7+ chips coming in blade servers sometime between now and 2012 and pegs the Power8 introduction for somewhere around the middle of 2013.

The following AIX roadmap is interesting in that it shows IBM’s Unix variant being tweaked to exploit the Power7+ chips sometime in the second half of 2011.

Whether the Power7+ chips will ship in volume in the second half of 2011 remains unclear, of course. But IBM could roll out Power7+ chips in very specific machines to target selected customers–notably, supercomputing customers who want faster clocks. This presumes, of course, that IBM is going to move to 32 nanometer processes with the Power7+ chips and be able to goose the clocks on these chips. IBM could instead add cores or other features to the Power7+ designs to target other customers, say, for instance, Microsoft, Sony, and Nintendo with their game consoles. I think this is a less likely option, but IBM sells many more chips to these three companies than it consumes itself in its Power Systems lineup, so it is a possibility. That said, I think the workloads for an enterprise server and a game console are so wickedly different that you could not really make one chip that does either job well.

All of these roadmaps don’t have enough detail to be useful, and this is intentionally so. Steve Sibley, director of product management for the Power Systems line at Big Blue, answered a direct question when I asked if there was going to be a Power7+ chip in the future. (Yes, I was a bit surprised.)

“Yes, there will be such a thing,” Sibley said. “We certainly have a Power7+ in the next year to 18 months. We’re still working on this.”

When pressed as to whether or not the Power7+ chip would be socket compatible with the Power7 chips, Sibley started dancing a little, saying that IBM was cooking up I/O enhancements for the Power Systems line over the same 12- to 18-month period and that to move to the Power7+ chips, customers would have to move up to system boards that supported whatever these I/O enhancements are. It stands to reason that if IBM forces a board swap along with a processor bump that it will offer system upgrades to customers, but that really depends on how much of the system changes. Accountants have very strict rules about what is a system replacement and what is an upgrade.

That’s not a lot more to go on when it comes to Power7, but it is something.

To see where IBM might be taking the Power7+ and Power8 chips, it makes sense to look at how the chips and memory components have evolved over time. Here’s how the latest several generations of Power chips have stacked up:

Here’s how the memory systems have stacked up:

I think that IBM wanted to do a 45 nanometer shrink with the Power6+ chips and double up the cores and either it had trouble doing that or decided against it. And I think that IBM definitely wants to do a 32 nanometer shrink with the Power7+ chips. (The Power4+ and Power5+ chips both had process shrinks and are more representative of the goals IBM and Intel both share with their tick-tocking.) As I have said before, I anticipate that IBM will stay with the same basic Power7 chip design with the Power7+ shrink. That means chips with four, six, and eight cores activated and with the same L1 and L2 caches. The process shrink should allow Big Blue to crank the CPU clocks somewhere between 25 and 30 percent.

At the same time, perhaps IBM will beef up the on-chip memory controller to support faster and denser DDR3 main memory. The Power7-based Power Systems machines top out at 8 GB DDR3 memory sticks running at 1.07 GHz, but the controller, in theory, supports 1.33 GHz and 1.67 GHz speeds and fatter 16 GB.

I also think that, given the very substantial performance improvement that the segmented embedded DRAM L3 cache memory on the Power7 chips have compared to the off-chip L3 caches with prior Power chips, Big Blue will boost the eDRAM capacity on the chip.

It seems unlikely that IBM would boost the core count with the Power7+ chips, but with the shrink, the company could do what it did in the Power5+ generation–and what Advanced Micro Devices is doing with its Opteron chips: take two shrinking processors, gear them down, and cram them into one processor socket. This would allow IBM to put a lot of cores and threads into the same Power System machines.

The Power7+ I/O enhancements could be tweaks to the memory controllers as well as to the integrated GX bus on the Power processors, which implements a double data rate (DDR) InfiniBand link from the chips out to remote I/O drawers. DDR InfiniBand runs at 20 Gb/sec and is a bit long in the tooth compared to 40 Gb/sec QDR InfiniBand, which has been out for years, and 56 Gb/sec FDR InfiniBand, which will be coming to market later this year.

If I were IBM, I would push up to at least QDR, doubling up the I/O bandwidth coming in off peripherals into the chips. QDR will be necessary for sure if IBM wants to support PCI-Express 3.0 peripheral slots, which will have a total bandwidth of 32 GB/sec bi-directionally for an x16 slot. InfiniBand is moving from 8b/10b encoding (where you send 10 bits for every 8 bits of data) to 64b/66b encoding (that’s 66 bits for every 64 bits of data). The PCI-Express 3.0 bus is also ditching 8b/10b encoding, and is shifting to an even more efficient 128b/130b encoding scheme. The first PCI-Express controller chips are expected by the end of this year, so if I had to guess, I would say IBM is moving the GX bus and related High Speed Loop to QDR or FDR InfiniBand and the peripheral bus to PCI-Express 3.0.

There are rumors that IBM is working on some deep-sleep, low power state modes for the Power7+ processors. I also think IBM should bring the MaxCore/TurboCore functionality across the entire Power7+ range, allowing customers to run all the cores at a rated speed (MaxCore) or run them somewhat faster if they turn half the cores off (TurboCore). It would be useful if the TurboCore mode could be initiated on the fly, not requiring a reboot of the system. Right now, this feature is only available on the Power 780 and 795 high-end machines.

Some of these things that I speculate about Power7+ could, of course, end up in Power8 chips. About the only thing that it is safe to guess on with Power8 is that it will use a 32 nanometer or 28 nanometer process and will come out sometime in 2013.