Hi Scott!

> The result of the printf is 1152921504606847000 as it was in your example,
> but the result of the DSPLY is 1152921504606846980
>
> Which is very interesting, don't you think?

Yes indeed.  It looks as if Simon's conjecture is on the money.  Some sort
of conversion/rounding error between FLOAT8 and packed.  It looks like C
will by default only display the 16 digits that double is supposed to hold
while the RPG conversion from float to packed manages to extract that little
something extra!  :-)

It has been an interesting thread.
  --buck




As an Amazon Associate we earn from qualifying purchases.

This thread ...

Follow-Ups:
Replies:

Follow On AppleNews
Return to Archive home page | Return to MIDRANGE.COM home page

This mailing list archive is Copyright 1997-2024 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.