Problems with Early i5 Plague Customers, Partners

Any new technology causes problems, but if the iSeries Nation “Sound Off” session at COMMON’s fall conference in Toronto last week was any indication, the May launch of the eServer i5 machines, the i5/OS operating system, and a new implementation of virtualization features caused customers serious headaches. The troubles that users and partners had in trying to get their i5 machines up and running may have contributed to the decline in iSeries sales during the past two quarters.

If there is a problem with any system, word gets around. It always does, and there is just no excuse for launching an enterprise server platform early if it’s not ready. Mike Borman, the new general manager of the iSeries Division, and his executive team got an earful about problems with the i5 machines.

As is traditional at the Sound Off session, Al Barsa, the president of Barsa Consulting Group, got the microphone first. At the spring conference, in San Antonio, Barsa waxed about how solid i5/OS V5R3 was, but five months later, at last week’s show, he railed against the new Hypervisor Management Console, the outboard Linux workstation that is used to manage logical partitions on the new i5 and p5 Power5-based servers. By having the HMC manage the partitions, IBM has been able to eliminate the need for a primary partition on an i5 server to be an OS/400 partition, and it has also eliminated a single point of failure in the system. With prior generations of OS/400 servers, logical partitions running either OS/400 or Linux were nested inside a primary OS/400 instance; if that primary OS/400 instance failed or had to be rebooted, it would kill all of the partitions on the machine. With the HMC approach, the HMC manages partitions on the i5 or p5, and if you walk up to the HMC and shoot it with a shotgun, the i5’s logical partitions keep humming along, blissfully unaware of the failure. From the sound of things, more than a few OS/400 shops wanted to shoot their HMC, considering all of the trouble it caused.

“The HMC features a keyboard that is more atrocious than any I have ever seen–even more than the System/38 keyboard–and the display shows spool files only in 80 characters,” said Barsa. “Bringing out products before they’re ready is a waste of time, money, and effort. Furthermore, it teaches customers a lesson.” He then told IBM that when he tested the HMC in March of this year, he said it wasn’t ready, and when he tested it again, in the early adopter program in May, he reiterated that it wasn’t ready. “I’ll repeat what I’ve said for the last four Sound-Offs: What were you thinking, or were you thinking?”

This put IBM on the defensive a bit, but the iSeries team did not deny there were problems. “Clearly the HMC is where we forgot the iSeries value proposition for the customers, and didn’t wait,” said Borman. “This is a very good example of where we’ve got to get back to and make sure these products are ready from the start. We’ve got to get it and other features back to where the iSeries customers think it’s easy to use and integrated.” John Reed, the iSeries product line manager during this launch, who has taken on a new job to make sure these new technologies are better integrated into the i5 boxes and i5/OS software, concurred that there were problems with the HMC, but he reminded everyone that it was a first release of a new technology. “One of the good things about the problem is that we did learn a lot with the HMC,” he said. “It was a brand new thing, and we took pretty dramatic actions with the firmware and hardware to improve it. We’ve made pretty dramatic improvements over what you saw very early on, versus what customers are seeing now.”

Neil Palmer, of Data Processing Services Canada, stood up and said that he has been doing OS/400 installs since 1988, and that there were several things that were wrong on the three machines he installed. For one, the license keys on one machine were broken and the only license key center for the iSeries is located in Denmark. The 800 number and e-mail address for the license key center were wrong on the install guide, and it took him until 2:00 a.m. to work up through the IBM support structure to get it fixed. Palmer also said that the six-inch-long serial UPS interface cable, which costs $150, didn’t work and that it needed a PTF that would not be available until November. “This all goes back to the value proposition of the iSeries,” explained Palmer to the top iSeries brass. “It’s integrated, it’s tested, and it works. We don’t want to get a PC in a box here, where we have to stick this in or stick that in, and maybe it will work, maybe it won’t. You really fell down on this. That’s something that you really need to address.”

The criticism was mostly aimed at the User Center Design team, which tests the i5 before it hits the streets. “Quite frankly, we probably were a little bit compressed on our test cycles,” said Reed. “We’re going back and re-examining that. This was not perfect in its execution, as you obviously saw,” Reed replied to Palmer. “I will not disagree with that. In fact, when Walt Ling [manager of iSeries support] and I did installs, we saw a lot of problems. We didn’t get everything packaged into the thing as it went out the door. I do apologize for that. We do take the install very seriously. We’re off and looking at what can we do differently going forward.”

Mary Kern, director of information services at the University of Toledo Foundation, backed up IBM’s claim that things have gotten better with the HMC, when she stood up and said she had installed a Model 570 server and didn’t see any of the issues with the HMC.

“It’s been a nightmare,” Kern said, explaining that she was on a first-name basis with the iSeries support people. “I have never had so many open PMRs [problem management records, a document detailing a support call into IBM] at one time in my life. Neil stole almost all my thunder, but he didn’t mention the fact that the modem that doesn’t work or that you need a PTF to make SAVSYS work. It’s been a nightmare, an absolute nightmare. But I do want to congratulate the support team. They’ve been wonderful.”

Another frequent comment is that the covers on the i5 machines were “cheap” and frequently broken when they arrived. No one wants to pay $15,000 to $40,000 for a server and have the cover broken. It may seem like a small thing, but it is an important image thing. Imagine if your BMW or Lexus arrived with a big scratch on the door. The i5 is arguably a BMW or Lexus where it counts, in its software, but IBM can clearly spend a few bucks on making a server cover that is not shoddy. Image, particularly that of the i5, matters.

One last important note that was not brought up at COMMON’s Sound Off but has just come to our attention. There is apparently an issue with the feature 2757 PCI-X RAID5 disk controllers, which were announced in January 2003, failing in the field. IBM has just sent out two PTFs to run as a diagnostic tool to identify potentially faulty 2757 cards in your machine. Those PTFs are MF33849 for OS/400 V5R2 and MF33850 for i5/OS V5R3. The feature 2757 cards are the highest-performing RAID5 disk controllers that IBM sells, with 235 MB of write cache memory (with data compression turned on, this cache memory turns effectively into a 757 MB cache). It is an Ultra3 SCSI controller that supports a RAID5 set with a minimum of three drives, expandable to 18 drives, and up to four SCSI buses, which run at 160 MB/sec, compared with 80 MB/sec of prior cards. If you have one, get this utility and run it now.