|
Hi Greg
Did I do something wrong ?
I'm afraid so. As Adam said, you have to use "integer division" for it to
work, so in your example of:
Say I want to round 1461 up to the nearest thousand, which would be
2000.
So N is 1461 and S = 1000
Temp = 1461 + 1000/2
Temp = 1461 + 500
Temp = 1961
Result = (1961/1000) * 1000
Result = 1961 ?
You need to have Result calculated as:
Result = Int(1961/1000) * 1000
Result = 1000
If N had been 1641 then you would have had:
Temp = 1641 + 1000/2
Temp = 1641 + 500
Temp = 2141
Result = Int(2141/1000) * 1000
Result = 2000
The half adjustment works on the mid boundary of the scale so anything below
gets rounded down while anything equal to or above gets rounded up.
All the best
Jonathan
-----Original Message-----
From: rpg400-l-bounces@xxxxxxxxxxxx [mailto:rpg400-l-bounces@xxxxxxxxxxxx]
On Behalf Of Fleming, Greg (ED)
Sent: 10 November 2006 19:47
To: RPG programming on the AS400 / iSeries
Subject: RE: Rounding up to nearest 1000
|The rounding algorithm goes something like this:
|
|Let N be the number to round, S be the scale of the rounding (in this
|example S = 1000)
|1) temp = N + S/2
|2) result = (temp / S) * S
|
|I think this has to use 'integer division' to work.
|
|This is essentially what the H extender is about - hence the name -
|"(H)alf add".
|
|HTH,
|Adam
Adam,
I'm trying that out using my trusty calculator, and I'm not ending up
with a round number.
Say I want to round 1461 up to the nearest thousand, which would be
2000.
So N is 1461 and S = 1000
Temp = 1461 + 1000/2
Temp = 1461 + 500
Temp = 1961
Result = (1961/1000) * 1000
Result = 1961 ?
Did I do something wrong ?
Greg
As an Amazon Associate we earn from qualifying purchases.
This mailing list archive is Copyright 1997-2025 by midrange.com and David Gibbs as a compilation work. Use of the archive is restricted to research of a business or technical nature. Any other uses are prohibited. Full details are available on our policy page. If you have questions about this, please contact [javascript protected email address].
Operating expenses for this site are earned using the Amazon Associate program and Google Adsense.