In case it wasn't clear, the takeaway from my last post should have
been: Pay attention to the number of decimal places in the
intermediate result (the reciprocal of the rate). If you're not
actually storing the intermediate result yourself but rather letting
RPG handle it for you within an expression, you'll typically get much
higher precision (up to 63 digits, which far exceeds Excel's); though
on occasion, the structure of an expression can result in much *worse*
precision than what is intuitive:
http://publib.boulder.ibm.com/infocenter/iseries/v5r4/topic/books_web/c0925086554.htm#HDRPRECXDF
http://publib.boulder.ibm.com/infocenter/iseries/v5r4/topic/books_web/c0925086551.htm#HDRPRCSNR
John
This mailing list archive is Copyright 1997-2025 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact
[javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.
midrange.com
