zEnterprise EC12 Mainframe: Still The Big Iron

September 4, 2012 Timothy Prickett Morgan

Power Systems had better thank their lucky stars that IBM is still investing in mainframes and can still rake in billions of dollars–my guess is around $15 billion over a two-year cycle–selling mainframe hardware and systems software. Because if IBM did not throw off most of that as profits, there would not be an IBM fab and there would not be a Power Systems biz at all. IBM would have long since ditched the server hardware business to focus on software and services.

To a very large extent, the System z product line not only keeps the rest of Big Blue supported, but also generates some of the ideas that eventually get ported down to Power Systems and System x iron and their relevant systems software. Some ideas are too specific to be moved down to Power-based systems, but there are nonetheless plenty of ideas in the new zEnterprise EC12 mainframes and their “zNext” z12 processors to steal and bring to Power machinery some day.

At the Hot Chips conference in Cupertino, California last week, Kevin Shum, senior technical staff member for System z processor development in IBM’s Systems and Technology Group, divulged many of the feeds and speeds of the z12 processors, as has been planned for many months. What was not planned was a launch of the new System zEnterprise EC12 mainframes that use them on the second day of the conference. But yields on the z12 chips were good and the systems were ready to go about a month and a half earlier than expected, and to give new IBM chief executive officer Ginni Rometty a good close to her first year running Big Blue, IBM is getting the new mainframes out early and, it is hoped, starting out with a big bang in sales in the fourth quarter.

The new z12 engines are six-core processors, up from the four cores used in the z11 engines inside the zEnterprise 114 and 196 machines announced in 2010 more or less concurrently with the eight-core Power7 processors. The chips are implemented in a 32 nanometer high-k metal gate process that also weaves in IBM’s long-establish copper and silicon-on insulator processes. The z12 processor has 48MB of embedded DRAM (eDRAM) that is used as L3 cache for the z12 cores. This is twice the amount of eDRAM L3 cache that the z11 processors had and is made possible through the shrink from 45 nanometer processes. IBM has also tweaked the out-of-order execution pipelines with the z12 processors, which debuted two years ago in the z11 chips, to help goose performance. IBM is using some of the shrink to goose the clock speeds on the cores up to 5.5GHz, a slight rise from the 5.2GHz of the z11 chips, but IBM says that the z12 engine was designed to crank the clocks up as high as 6GHz at some point. The chip has 2.75 billion transistors on it, making it one of the fattest processors around, and includes two memory controllers, a GX I/O bus controller, and SMP connections to an outboard SMP hub and L4 cache unit. That SMP hub lashes multiple processors together and also sports 384MB of L4 cache, which is used to keep the L3 caches fed and happy. That’s twice the L4 cache memory as was available in the z11’s SMP hub chip.

Yields are never perfect for any chip, and therefore IBM is using a mix of z12 processors with various working L3 cache amounts and four, five, or six cores to create five distinct zEnterprise EC12 machines with anywhere from one to four processor books (what people outside of IBM call system boards) with five processor sockets per board and anywhere from 27 to 120 active cores that can be deployed as central processors, I/O processors, cluster processors, Linux processors, or zIIP or zAAP coprocessors to speed up DB2 or Java workloads, respectively. All of the machines also have some cores that are kept in reserve for IBM and as standard spares that customers can activate if something goes wonky. (You can see my full description of the zEnterprise EC12 models in this story over here at The Register.)

A single z12 core should deliver about 1,600 MIPS, which is about 25 percent more oomph than a z11 engine running at top speed had, and the whole system should offer around 75,000 aggregate MIPS of processing capacity, which is about 50 percent more than a fully loaded z11 machine with 80 cores spinning could do. Those are very respectable scalability increases, and no one else is doing better. (The Power7+ design, as we report elsewhere, has processors that run about 25 percent faster, but no increase in core count using the faster chips. If you want more scalability, you have to move to dual-core modules that put two slower Power7+ chips into a single socket, but the clock speeds are lower so each individual thread runs slower.)

The zEnterprise EC12 machines sport 3 TB of RAIM main memory, with that being short for Redundant Array of Independent Memory and providing the same kind of data striping and parity that RAID 5 disk controllers do for disk drives. This is the same maximum capacity and RAIM striping that the z11-based machines offered, and it is reasonable to expect that at some point, perhaps with the Power8 machines, IBM will offer RAIM on Power Systems iron.

The new mainframe is also the first of IBM’s servers to support what is called transactional memory, something that most system makers have been working on for the past decade and none have brought to market yet. With transactional memory, you do what IBM calls “opportunistic locking,” which means letting transactions run through the system without locking down resources and then checking if there was any contention after the fact. A lot of the time, there is no contention, and on multithreaded DB2 database and Java virtual machine workloads, IBM is seeing as much as a 45 percent boost adding up the effects of higher clock speeds, bigger caches, and transactional memory on these heavily threaded jobs. This is a very substantial performance benefit, and that is also why I think we will see transactional memory in the Power8 chips. (There was no hint at Hot Chips that Power7+ chips would support transactional memory.)

The System zEC12 machines also include PCI-Express 3.0 peripherals, which the Power7+ machine had better have if IBM hopes to be competitive with its RISC, Itanium, and Xeon peers. IBM has not said what peripheral controllers to expect with Power7+ chips. The original Power7 chips had PCI-Express 1.0 peripheral slots, and the Power7′ (pronounced “power seven prime”) machines announced last October sporting Power7 chips and double main memory capacities for entry and midrange machines, had PCI-Express 2.0 peripheral slots.

The zEC12 mainframes ship on September 19, and they run the latest two or three releases of IBM’s z/OS, z/VM, z/VSE, and z/TPF operating systems as well as the latest Linuxes from Red Hat and SUSE Linux.