Re: Where is all that CPU going?

["James H. H. Lampert" <jamesl@xxxxxxxxxxxxxxxxx>]

Let's get back to the actual problem at hand (aside from general 
sluggishness):

Would anybody care to hazard a guess about how much time the following 
SHOULD take:

On a healthy 550, with 4516G in 53 drives, and a healthy amount of main 
storage, (WRKHDWRSC shows 4 4G main storage cards, but the box has 3 
LPARs), a non-Cycle RPG program runs through a file of 132369 records.

For each of these records, if the record is of type "customer," and a 
corresponding record can be found in a 226368-record customer master file,
      the program SETLLs to that customer's first record in a 2.2M 
record "A/R journal by customer number" logical file, then runs through 
all of that customer's records in that file. For each of these records 
(but only the customer's records, and remember, the customer number is 
the leading key),
                  the program adds several numbers into digest fields 
in our controlling file, then runs through the current invoice's detail 
records in a 14.8M record "billing detail" file, to see if there is 
anything that should be backed out of the digest fields.

       After this, and before going on to the next customer, the 
program goes through the customer's records in two tracking files, one 
of them 2.2M records, 314 fields wide, the other 714k records, about 400 
fields wide, both of them logicals in which the customer number is the 
leading key (with the 3 fields on which we're digesting the data as the 
next 3 keys), and uses the customer's records in these two files to 
build up records for that customer in a digest file, checking them 
against a logical of the same 14.8M record "billing detail" file, to 
determine whether to increment a counter.

While we are currently rebuilding that digest file from scratch every 
time, there is no evidence that we ever visit any of these 19M records 
more than once (or twice, in the case of any "billing detail" records 
that are checked for both the first major step and one of the two 
"digesting" steps). If I were to change the algorithm to avoid the 
scratch rebuilds, I might reduce the number of records visited by 75%.

At any rate, given the hardware specified, how long SHOULD a non-Cycle 
RPG program take to visit some 19M records in 7 (counting both the 
billing detail PF and the LF we're using) files, updating most of the 
records in the controlling file, and writing digest records to an 8th file?

--
JHHL