Re: %dec / target for a numeric operation is too small -- RPG400-L

Subject: Re: %dec / target for a numeric operation is too small
From: Barbara Morris <bmorris@xxxxxxxxxx>
Date: Mon, 26 Jul 2010 18:05:00 -0400
List-archive: <http://archive.midrange.com/rpg400-l>
List-help: <mailto:rpg400-l-request@midrange.com?subject=help>
List-id: RPG programming on the IBM i / System i <rpg400-l.midrange.com>
List-post: <mailto:rpg400-l@midrange.com>
List-subscribe: <http://lists.midrange.com/mailman/listinfo/rpg400-l>, <mailto:rpg400-l-request@midrange.com?subject=subscribe>
List-unsubscribe: <http://lists.midrange.com/mailman/listinfo/rpg400-l>, <mailto:rpg400-l-request@midrange.com?subject=unsubscribe>

On 7/26/2010 11:38 AM, Luis Rodriguez wrote:

Kirk,

As I see it, your 4:2 spec is too small for the 0.1258 value (at least
5:4)...

Luis, %dec doesn't care if there are too many decimal places. It just truncates them, so it would return 00.12 for the input '.1258'. (%dech would round to 00.13)

Kirk, what is the actual value of wrkscore that is causing the failure? The only thing I can think of is that the decimal point is not in wrkscore, so when you think you have '.1258' you actually just have '1258'.

Follow-Ups:

Re: %dec / target for a numeric operation is too small , Kirk Yates

This mailing list archive is Copyright 1997-2026 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].

Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.