IBM Adds A Bunch Of I/O Devices To Power Systems

October 15, 2012 Timothy Prickett Morgan

While the Power7+ chip, the new Power 770+ and Power 780+ systems, and IBM i 7.1 Technology Refresh 5 were the dominant parts of the October 3 announcements, IBM made a bunch of changes to its lineup of I/O devices–both storage and networking–that have an effect on the entire Power Systems range and that keep the product line moving forward along with the relentless tide of ever-increasing capacity that sweeps us along in the IT racket.

The I/O enhancements from IBM for the Power Systems machines were all rolled up in announcement letter 112-180. Perhaps the most exciting of these is an enhanced EXP30 Ultra drawer for flash-based solid-state drives (SSDs) to significantly beef up the I/O operations per second for Power boxes.

Speeding up reads with flash and hierarchical storage management.

The EXP30 Ultra drawer, with feature number EDR1, is a 1U rack-mounted unit that can hold up to 30 SSDs, which has packaging that is twice as dense as the existing EXP24S drawer, which is feature 5887 in the IBM catalog and which packed 24 drives into a 2U rack-mounted chassis. If you have already invested in the EXP24S drawers and SSD drives, you can hang up to two of these off of an EXP30 Ultra drawer, with another 48 drives added to the overall 5U combination of enclosures. As far as I know, IBM is not letting customers daisy chain multiple EXP30 Ultra drawers together, but that would probably be a more interesting option and one that Big Blue may allow at some point in the future when all of the Power7+ servers are launched next year.

The EXP30 Ultra drawer (EDR1) uses enterprise multi-level cell (eMLC) flash memory and each 1.8-inch drive in the unit offers 387 GB of capacity for a total of 11.6 TB for the 1U device when fully loaded. It has two SAS controllers to manage the drives with 3.1GB of write cache and enough oomph in the controller to not be a bottleneck, and the controllers are redundant for high availability as well as throughput.

This SSD drawer is similar to the EXP30 Ultra drawer (feature code 5888) that IBM announced back in April for attachment to Power 710, 720, 730, and 740 systems, although this one is designed to hang off the improved GX++ bus on the Power 770+ and Power 780+ machines and will no doubt also be available on future Power7+ machines, too. And in this case, when attached to the new Power7+ machines and using the new-and-improved GX++ bus, the feature EDR1 box offers somewhat higher performance on reads and writes than the feature 5888 box. It can drive 480,000 I/O operations per second (IOPS) in read-only mode and 325,000 IOPS in write-only mode, with a mix of 60 percent reads and 40 percent writes driving on the order of 410,000 IOPS, which is roughly a 20 percent performance boost over the feature 5888 drawer. It is not clear what the I/O throughput is if you hang one or two of the older EXP24S drawers off either of the EXP30 Ultra drawers. You can put flash drives or disk drives inside of those EXP24S drawers, and if you use the new 900GB disks (more on that below) you can have 11.6TB of flash front-ending 43TB of disk, all managed by IBM’s EasyTier software, moving hot and cold data around between disk and flash as needed.

IBM has a new dual-port GX++ adapter (feature 1914) that is required for the Power 770+ or Power 780+ servers to link to the EXP30 Ultra SSD drawer. Each x8 port on the GX++ card feeds out to one of the SAS controllers in the drawer, which can do a number of different levels of RAID protection across the SSDs if you want to do that. You can also cross-couple multiple GX++ ports in two different servers to multiple EXP30 drawers for redundancy. The SAS controllers have big fat capacitors that hold charge, and in the event of a power failure has just enough juice to take the contents of the write cache on the SAS controllers and dump it onto the flash memory for later retrieval. No more batteries.

Now, here’s the interesting bit. The SSDs are formatted with 528 byte sectors, which is what is required by OS/400 and IBM i, rather than the 512 byte sectors that are used for AIX and Linux on Power Systems storage. But like the EXP30 drawer announced in April, this new one for the Power 770+ and Power 780+ machines does not support IBM i 7.1. However, after much grumbling from Power Systems-IBM i shops, Big Blue did say in a statement of direction that IBM i 7.1 or higher would eventually get native (by which I think IBM means not through the Virtual I/O Server) support. And, to put it bluntly, native support for this flashy device should be available on any Power System box based on Power7 or Power7 chips running IBM i 7.1.

The EXP30 Ultra drawer (feature 5888) that attaches to low-end Power machines costs $24,500 and the 387GB flash drives costs $6,200 a pop, but the feature EDR1 version of the EXP30 Ultra drawer that attaches to Power 770+ and 780+ machines costs $32,092 and the same flash drives cost $7,990 apiece. So on the low-end boxes, you are talking $207,500 for around 400,000 IOPS, or 52 cents per IOPS, while on the bigger Power7+ machines announced two weeks ago you will pay $271,792 for the same flash capacity and that will work out to 57 cents per IOPS because the price is rising faster than the IOPS.

Out of the gate, AIX 6.1 and 7.1 can drive the EXP30 Ultra drawer, as can Red Hat Enterprise Linux 5.7 or 6.3, SUSE Linux Enterprise Server 11 SP2, and IBM VIOS 2.2.1.5 (planned for December 19) and 2.2.2.0 (which is available now).

New disks, internal and removable

It wouldn’t be a fall server announcement without some sort of new disk drives for customers to slap into systems or enclosures, and this fall is no different with a new set of 900 GB disk features that spin at 10K RPM. IBM is offering 900 GB drives in the two slightly different 2.5-inch carrier formats (which it calls SFF-1 and SFF-2) formatted in 512 byte sectors and two 856 GB units with the same carrier options that are formatted with 528 byte sectors. Feature 1737 comes in the SFF-1 carrier and can be plugged into Power Systems machines directly if they have 2.5-inch slots, as well as feature 5802 and 5803 expansion drawers that hang off of 12X I/O loops; it costs $1,500. Feature 1738 comes with the SFF-2 carrier and can plug into Power Systems machines as well as the feature 5888 EXP24S expansion drawer; it also costs $1,500. The AIX/Linux equivalents of these two units are features 1751 and 1752, respectively, and they cost the same $1,500. You can reformat the drives to either format as needed at your site, by the way–the first time I’ve seen IBM admit this was possible. The drives require IBM i 7.1 or AIX 5.3, 6.1, or 7.1 among the IBM operating systems as well as SLES 10 or 11 or RHEL 5.7 or 6.X.

The RDX removable drive, which plugs into Power Systems machines through a USB port and which was updated with a 320 GB unit back in April, and now, after long last, with IBM i 7.1 TR5, this RDX drive as well as a much fatter and faster one can be used by the IBM i platform.

The new feature EU23 (Power 710 and 730), EU03 (Power 720, 740, 750, and 755), and EU04 (Power 710 through 780) RDX docking bays fit in a half-high storage bay and plug into a USB-3 port. A new SATA RDX docking station (feature EU07) is also supported on selected Power6 and Power7 machines running IBM i 6.1 or 7.1 in the high bay of the machine. The EU23 and EU03 units cost $225, while the EU04 and EU07 units cost $275.

Another tweak that is coming with IBM i 7.1 TR5 is that you can now use USB-attached DAT160 tape drives with IBM i, which you could do with AIX and Linux systems previous. IBM i’s use of DAT160 drives was restricted to SAS/SATA attachment up until now. Now you can attach the feature EU16 DAT160 tape drive to a Power 720, 740, or 750.

Another change is that IBM is allowing customers to hang adapters right off the GX++ slots to get around the bandwidth limitations of PCI-Express 1.0 peripheral slots in the older Power Systems boxes themselves and their expansion drawers. IBM is doing this first on the high-end Power 795 machine, which has four of the GX++ slots on each of its eight processor books, for a total of 32 slots. IBM is offering two different cards that can plug into the GX++ slots on the Power 795 machines.

The first is feature EN22, which is a two-port 10 Gigabit Ethernet adapter card that supports the Fibre Channel over Ethernet (FCoE) protocol and this can allow the Power 795 to talk directly over Ethernet to storage arrays without going through the 12X I/O loops, which split down the 20Gb/sec of bandwidth that comes out of the GX++ port. This card costs a whopping $12,000, so you really have to need it badly. The feature EN23 card that plugs into the GX++ slot is a two-port adapter as well, but this time it speaks 16Gb/sec Fibre Channel, again very useful for lashing the Power 795 server directly to storage. This one costs $13,000.