The i5 515 and 525 Versus the Windows Competition

June 11, 2007 Timothy Prickett Morgan

Now that the user-priced System i5 515 and 525 servers have been in the market for a while, the time has come to see how they stack up to the competition and to prior generations of System i5 machines. While the user-priced machines are capable of supporting legacy RPG and COBOL applications using the 5250 green-screen protocol, IBM is trying to position these machines for new workloads, like Java and PHP, as well as old ones, and is therefore trying to gauge their performance on the basis of those new workloads.

To help make the case for the i5 515, IBM ran the SPECjbb2005 benchmark test on the box. That test, which is administered by the Standard Performance Evaluation Corporation, is in essence a Java version of the Transaction Processing Council‘s TPC-C online transaction processing test with the substantial I/O requirements of the TPC-C test removed. Basically, it converts a system-level benchmark test into one that stresses the processors and main memory–which is exactly what modern computers, with their big gobs of main memory, have become.

When IBM tested the i5 515 back in March, just prior to the release of the user-priced machines, it only tested the box with one of the 1.9 GHz Power5+ cores in the server activated. The single-socket i5 515 can have two cores activated, of course. Back then, IBM did not run a test on the i5 525, which is a larger configuration of the user-priced machines that can have lots more users hanging off of it but which, somewhat paradoxically, does not necessarily deliver more performance. The i5 525 is a more expandable box, but the most important characteristic is that it can support more than 40 users–the cap on the i5 515–and can, in fact, be converted into a box that has no limits on the users attached to it. But IBM did test a p5 520 with exactly the same hardware, and in the comparisons I make, I am using this p5 performance on the SPECjbb2005 test to reckon what it might be on the i5 525. The i5 could do better or worse than this, of course, depending on the tweaks in the operating system and Java Virtual Machine (JVM), but this is a ballpark estimate that I feel pretty comfortable with.

To get an idea of how the i5 515 and 525 stack up against Windows servers, I went to the SPEC site and gathered up performance information for entry servers that are roughly in the same power class as the new System i machines. You can see the SPEC results I distilled down in this table. The table includes not only the results for the i5 515 and my estimates for the i5 525, but also for a mix of Windows, Unix, and Linux boxes. In this story, I only plan to cover comparisons with Windows. In future editions of this newsletter, I will cover how these user-priced machines compare to Unix and Linux boxes, as well as to prior generations of System i servers.

Before I get into the SPECjbb2005 comparisons, I wanted to point out that I know the limitations of benchmark tests. This is only one test, and I know that. IBM does not want people to price the i5 515 and 525 based on performance at all, but rather on user count. But a user count without some kind of reckoning of the performance available to end users is only half the story. And even after making the comparisons that I could–given the thinness of data for the SPECjbb2005 test–I find myself wishing that it was easier to make direct comparisons between i5 and Windows platforms.

The fact is, it has always been difficult to get an apples-to-apples comparison between the AS/400 and competitive platforms, and nearly two decades of the existence of an OS/400 and i5/OS platform has not changed that. The System i5, like its predecessors, has integrated software and unlimited user software licenses based on different hardware tiers (which roughly correspond to generations and processing capacity). Even with the shift to user-based pricing for i5 515 and 525 servers that were announced in April, IBM first charged for named users on the system and then changed to a concurrent user pricing model in May, after feedback from its customer base. There are many shops that have lots of users, but some companies have users that work in shifts or who are only occasional users, so charging $250 a pop for each of these users can add up pretty fast. The point is, even though IBM has moved–with these two machines so far–toward a pricing model that more closely aligns with the X64 server platform and the Windows operating system and software stack from Microsoft, that does not mean that the prices themselves for the i5 and Windows platforms have aligned. As I will demonstrate–and somewhat contrary to IBM’s own competitive analysis that IT Jungle published at the end of April–the X64 server market has moved on, from single-core to dual-core and now quad-core processors, which are offering a lot of bang for the buck.

Which means mostly that the System i line needs the Power6 processors to keep pace, of course. Now would have been a good time.

The interesting problem that I had, in fact, in coming up with the SPECjbb2005 comparisons is that I could not easily find machines that had such a low performance running this Java benchmark. While IBM was correct to point out that the per-core performance of the i5 515 was second to none, this is true in large part because of the efficiency of the Power5 and Power5+ design (which does roughly twice as much work per clock as an X64 chip) and because only one core was activated in the machine and therefore had the whole 1.92 MB L2 cache and 36 MB of L3 cache to play with. Java just loves cache memory and main memory, being an interpreted language. But current Xeon and Opteron X64 processors have big caches, often higher clock speeds, and lots more cores in an entry server configuration. There is a lot of bang for the buck in X64 hardware, based on peak performance and low iron costs.

To keep it relatively simple, I examined Windows boxes from Hewlett-Packard, Dell, and IBM, providing the actual performance on the SPECjbb2005 test or an estimate. (The performance estimates are marked in bold, italic, red.) SPECjbb2005 gauges performance in terms of business operations per second, or BOPS.

Take a look at this i5 515 and 525 to Windows X64 server comparison table. To get anything even close to the performance of the single-core Power5+ chip running at 1.9 GHz inside the i5 515 required stepping down to 64-bit, dual-core Pentium D processors, which ran at considerably higher clock speed and which cost next to nothing. Even still, as the table shows, the i5 515 that IBM tested is absolutely competitive with a Pentium D-based ProLiant ML310, Dell PowerEdge 840, or IBM System x206m server running Windows 2003 Enterprise Edition and SQL Server 2005 Enterprise Edition. (The comparisons I made have 20 users for the i5 515 and 25 for the Windows boxes, since 25 is the entry point for the Windows platform.) Somewhat surprisingly, on a per user or a per BOPS basis–pick one, it doesn’t matter–the System i5 515 platform is the lowest cost of the four different platforms.

When is the last time you saw that?

Of course, there is a wrinkle in this analysis, and I expose it in the table. No one buying such an entry X64 server puts Windows Server 2003 Enterprise Edition and SQL Server 2005 Enterprise Edition on those boxes. If you go with Windows 2003 Standard Edition and SQL Server 2005 Standard Edition, which you can get down to as few as five users and which you can configure with the 20 users I set up on the i5 515, then the Windows platform is considerably cheaper. Like anywhere from 25 percent to 43 percent in the machines I configured. Moving to Microsoft’s Windows Small Business Server, which has an embedded SQL Server database, could lower the software bill even further without having any appreciable effect on performance for such a modestly powered machine.

This brings me back to a point I raised years ago: IBM needs to have different editions of the i5/OS and DB2/400 bundle, aimed at different customer sets. While IBM has come a long way this year by offering an i5/OS Application Server edition, user-based pricing with a base software license fee for i5/OS and DB2/400, not charging a hefty premium for 5250 capacity, and not gearing down performance, the small and medium business market needs an i5/OS SBS. The current packaging is competitive with the fuller Windows stack, and that’s fine.

What the table shows quite clearly is that the i5 515 and 525 do not have enough oomph compared to the competition once you get off that first core in an entry configuration. X64 servers can easily have three or four times the Java performance, as measured by the SPECjbb2005 test, as these two i5 boxes–and deliver it at 40 percent to 50 percent less money. (That generalization compares the i5 525 with two cores activated to servers based on dual-core Opteron and Xeon processors as well as the new quad-core Xeon processors.) This difference in performance may not affect the per user cost of the machine, but no customer is going to take a vendor’s word for it that a machine rated for five, 30, 40, or 150 users (as the i5 515 and 525 are) can actually deliver performance for that number of users. Ultimately, customers need to know how much work the machine can do as well as what it costs per user to attach them to the box to get their slice of the hardware.

None of the comparisons I have made in the table include the cost of server virtualization, which is a strength that the System i platform has in its Virtualization Engine hypervisor. Moreover, the workload manager inside i5/OS can drive up the utilization of mixed workloads very high, and Windows, no matter how far it has come, simply cannot do this. If the i5 boxes can have an average utilization in the range of 40 to 50 percent, compared to the average utilization of maybe 10 to 20 percent for an X64 server, this would go a long way toward closing the price/performance gap between the i5 515 and 525 and the Windows alternatives.

That, of course, comes down to business practices, not technology. In a world where people are trying to be efficient with money, electricity, and space, having the smallest, least powerful server that will do the job with the most efficiency is the one a company should buy. If IBM can demonstrate that the utilization of the i5 machines is indeed three, four, or five times as high as Windows boxes, then it can and should command the premium that the SPECjbb2005 tests imply it deserves.

All I know is, I am glad I don’t have to make that sales call. Because that sounds like a much tougher sell than this: “Same price as Windows, better technology, and more efficiency.”

This Issue Sponsored By

Table of Contents

Content archive

Recent Posts

Subscribe

Pages

Search