Re: %floar rounding -- RPG400-L

Subject: Re: %floar rounding
From: David Gibbs <david@xxxxxxxxxxxx>
Date: Thu, 23 Jul 2009 10:57:58 -0500
List-archive: <http://archive.midrange.com/rpg400-l>
List-help: <mailto:rpg400-l-request@midrange.com?subject=help>
List-id: RPG programming on the IBM i / System i <rpg400-l.midrange.com>
List-post: <mailto:rpg400-l@midrange.com>
List-subscribe: <http://lists.midrange.com/mailman/listinfo/rpg400-l>, <mailto:rpg400-l-request@midrange.com?subject=subscribe>
List-unsubscribe: <http://lists.midrange.com/mailman/listinfo/rpg400-l>, <mailto:rpg400-l-request@midrange.com?subject=unsubscribe>

Lim Hock-Chai wrote:

Does anyone know why the resulting dec value is 38.039099?

Floating point inaccuracies?

http://publib.boulder.ibm.com/iseries/v5r2/ic2924/books/c092508415.htm#HDRFLTFMT9

"Float variables conform to the IEEE standard as supported by the OS/400 operating system. Since float variables are intended to represent "scientific" values, a numeric value stored in a float variable may not represent the exact same value as it would in a packed variable. Float should not be used when you need to represent numbers exactly to a specific number of decimal places, such as monetary amounts. "

david

As an Amazon Associate we earn from qualifying purchases.

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.