Single-Level Store Redux In New Power-Flash Hybrids

October 6, 2014 Timothy Prickett Morgan

With the System x division now in the hands of Lenovo Group, IBM is being more aggressive about promoting its Power Systems lineup and, in particular, a new machine that brings Power processors together Tesla GPU coprocessors from Nvidia and field programmable gate arrays (FPGAs) from Altera to create hybrid computing platforms.

The immediate benefit of these technologies are not apparent to IBM i shops, who run mostly RPG and Java code, but such hybrid machines will very likely be more common as GPU-accelerated Java comes to market in the next couple of years–perhaps next year, perhaps the year after depending on how the effort goes. Any application that does number crunching, sorting, or any other kind of sophisticated manipulation of data can be accelerated by the parallel processing inherent in a GPU, which has thousands of cores in the latest implementations from Nvidia. (2,880 CUDA cores, to be precise.) In general, such GPUs can accelerate certain kinds of applications by as much as a factor of 10X. I discuss the current machine IBM announced last week, the Power S824L, a Linux-only box with two Nvidia Tesla K40 GPU coprocessors, over at EnterpriseTech, and you can read all about it over there. You can also get the low-down on two other Linux-only setups, one that hosts NoSQL data stores and one that runs a souped up version of Hadoop that uses a mix of IBM file systems and middleware to replace big chunks of the Hadoop stack and speed it up.

From the IBM i perspective, something in the two Data Engine Solution Editions as the latter machines are called, caught my eye. It was the FPGA cards and how IBM was using them. In the Data Engine for NoSQL solution, IBM has the FPGA programmed to act as a kind of extended memory controller to link the Power8 processor and its cache and main memory complex into the flash memory in an external FlashSystem 840. This link, which is done over the Power8 processor’s Coherent Accelerator Processor Interface (CAPI) ports, allows an in-memory application like the Redis NoSQL data store to address the flash just like main memory. In effect, it is creating a kind of single-level store between main memory in the system and flash memory in the external array, and in such a way that the differences in the speed between the main memory and flash are largely masked.

The net effect in the DataEngine for NoSQL case is that IBM says it can replace 24 X86 server running Redis in their main memory and supporting 12 TB of data resident in memory with a single two-socket Power8 server with 512 GB of memory and a FlashSystem 840 with 12 TB of flash storage–with about the same performance and at under half the cost of the X86 server cluster. This is a compelling set of numbers, and ones that show IBM is thinking very seriously about how to leverage Power8 in a constellation of compute, storage, and networking. In the DataEngine for Analytics solution edition, the FPGA card is programmed to do GZIP data compression and decompression on data stored in the GPFS file system, which results in faster file accesses and lower data requirements. IBM can crunch the data by a factor of three or so compared to running a normal Hadoop setup with its triple-replicated data in the Hadoop Distributed File System.

It would be very interesting indeed to see such data compression and flash memory extension technology brought to bear on IBM i workloads. The CAPI links for the FlashSystem 840, implemented on the FPGA card in the Power8 machine, have much lower latency and much higher performance than using the normal PCI-Express drivers to attach out to IBM’s all-flash array using Fibre Channel links. It would be good to see compression added to the FPGA to boost the effective capacity of the FlashSystem 840 and therefore lower the cost per gigabyte of the flash storage. This would be a new kind of single-level store, more tightly coupling main memory and flash memory. For a lot of customers, disk drives might be totally unnecessary with such a setup. There is probably some reason why this might not make sense, and if you can think of it, do tell. As far as I know, there were no major tweaks to the Ubuntu Server variant of Linux to allow the flash to be treated like slow main memory instead of fast disk storage.