Mystifying OS/400 and i5/OS Mathematics
August 9, 2006 Howard Arner
A customer recently reported a bug in my SQL Thing product, but after investigation it turned out not to be a bug but the user’s perception of how decimal arithmetic works in the i5/OS operating system. The user was doing some arithmetic with decimal fields and then multiplying by 100 to show the value as a percent, and was getting a result of zero for all of his calculations. The real problem is in the way the AS/400, iSeries, and i5 handle promotion of precision and scale when decimal numbers are used in equations. If you are not careful, |