Early Adopters Prep for New OS/400, iSeries, and Logical Partitions

The word on the street is that IBM will be giving early adopters of the forthcoming OS/400 V5R3 and Power5-based Squadron servers access to the new soft and hard wares within a few months. The information I am getting is a bit convoluted, which suggests that some customers are getting OS/400 V5R3 on existing hardware while others are getting it on new Power5-based hardware.

Some sources say they may get their hands on OS/400 V5R3 running on existing machines in May; others say they will get it in June. Considering that IBM is expected to announce OS/400 V5R3 in mid-to-late April, and that it usually ships a new operating system within 30 days of an announcement, it sounds like these companies aren’t early OS/400 adopters, but that they’re getting notice of a forthcoming announcement. Other sources say they will be getting an early release of the actual Power5-based Squadron servers running OS/400 V5R3 in June. IBM is apparently looking for early adopters who can stress-test the new OS/400, as well as Linux and AIX, on the boxes. The future Squadron machines sport a new hypervisor layer that makes these three operating systems equals on the boxes.

That hypervisor layer allows a much larger number of logical partitions to be put on a Power-based server than what was possible in the past. The pSeries Unix servers currently support only logical partitions that span a whole Power4 or Power4+ chip, which means the smallest partition that AIX can support is comprised of two whole processor cores. The iSeries Power machines have for years supported up to four OS/400 partitions per core and up to 10 Linux partitions per core. The AIX development team came late to the logical partitioning game, and the iSeries development team has once again bailed out IBM’s precious Unix line. (IBM’s Unix server line was so far behind rivals Hewlett-Packard and Sun Microsystems in the mid-1990s that, were it not for the AS/400 design team in Rochester, IBM never would have fielded the Apache and Northstar boxes, and probably would have slinked quietly out of the Unix server market. IBM Austin gets a lot of the credit for the work that IBM Rochester did.)

That said, there is still considerable confusion over how many logical partitions a Squadron box will support. I’ve gotten answers that are all over the map in the past two years, from 255 (I’ve seen that in IBM roadmaps that are several years old) to 256, to 512 to 640. If IBM’s iSeries can support up to 10 Linux partitions per processor on the Power4 today, it stands to reason that it should be able to do just as well with the Power5 machines. However, the Power4 machines topped out at maximum of 32 partitions, for some unknown reason. I’ve been thinking about this, and I think this was the case (and I am guessing here), because OS/400 had only really been implemented with microcode that knew how to juggle 32 physical processors (in this case, we are talking about cores), and that the logical partitioning code in the OS/400 microcode somehow was limited by the 32-way symmetric multiprocessing (SMP) feature in OS/400. It cannot be a coincidence that 32 partitions and 32-way processing are both top-end limits on the current iSeries.

Now, with the new Squadron machines, IBM is supporting 64-way configurations, which means 64-way SMP has to be supported in both the operating systems (OS/400 and AIX, to be sure) and in the hypervisor layer. Moreover, with simultaneous multithreading (SMT), a single operating system (whether it is OS/400 or otherwise) will see two virtual processors for each actual physical core inside the Squadron, which means the operating systems have to know how to cope with 128-way virtual SMP. The question now is how IBM will map the logical partitions to these virtual processors. Ideally, if IBM could map 20 logical Linux partitions to each Power5 core (that’s 10 partitions to each SMT-virtualized core), IBM could cram 1,280 logical partitions on its biggest Squadron box. If IBM could only map partitions to physical processors, 640 would be possible. If IBM can only do eight, instead of 10, partitions per processor, that would be 512, and only four partitions per processor would yield 256.

Getting 1,280 or 640 partitions on a single 64-way would imply that even a uniprocessor machine could be sliced up very thinly. This would be a phenomenal achievement. But there appears to be some limits that IBM cannot get beyond–at least for now. While no one I have talked to seems to be able to nail the number down exactly, even though they have been briefed by IBM, most people tell me that the number is a weird one, somewhere between 256 and 300. This doesn’t fit in the power-of-2 world of computers very well. IBM must be crimping the logical partitioning support in some way because of overhead or instability when it goes beyond a certain limit. This probably has more to do with limits in how the hypervisor layer and its dedicated service processor (which I hear is really an xSeries machine running Linux) perform running real workloads than with any theoretical limits in the server hardware.

Incidentally, I would not be surprised if the limits have more to do with how IBM is going to support operating systems to encourage mixed use on the Squadron machines. In other words, the logical partitioning limits are completely artificial, just like the governors on 5250 green-screen performance were. Say IBM is still going to impose the four partitions per processor limit with OS/400 on the Power5s, and does the same for AIX, which is about as dense. Say that IBM can do 10 partitions per processor for Linux. Look at this one scenario if physical processor cores are the limiting factor. Then IBM might put the cap for the maximum number of logical partitions for a 64-way Squadron box (that’s 256 plus 32, the closest thing to the magic powers of 2 that I can come up with that is between 256 and 300). IBM might then cap the maximum number of OS/400 or AIX partitions at 256 and the maximum number of Linux partitions at 128 or 256. IBM might also make the largest OS/400 and AIX partition span 32 or 64 processors and the maximum Linux partition span 16 or 32 processors. The new Linux 2.6 kernel supports 64 processors in a single system image, using an amalgam of SMP and non-uniform memory architecture (NUMA) approaches (just like OS/400, AIX, and other operating systems do, by the way). I am dying to know exactly what IBM does, and why.

My sources also tell me that OS/400 will be the first operating system supported on the Squadron boxes, and if history is any guide, OS/400 V5R3 was done anywhere from six to nine months ago. I also hear that Novell‘s SuSE Linux Enterprise Server will be the first implementation of Linux delivered on the box and is expected in June or July, with Red Hat Enterprise Linux 3.0 taking up the rear once again. AIX 5L 5.3 support on the Squadrons is not expected until late September or early October, probably because of the substantial rewriting of microcode to support the new logical partitions and hypervisor.