Re: DATA-INTO where number of records not known -- MIDRANGE-L

Thank you Jon,

Yes it works and really appreciate your assistance, don't know where you
find the time!

I ran the json through YAJLGEN and it generated the following, just
showing the first couple of lines as the remainder of the DS is the same
as we already had.

dcl-ds jsonDoc qualified;
num_JOURNEYS int(10) inz(0);
dcl-ds JOURNEYS dim(6);
BOOKING_REF_NUM varchar(5) inz('');

So YAJLGEN has provided a count of JOURNEYS received, which if we know how
many records could be received would allow processing without the %Handler
by setting an appropriate dim..

Also it has provided a document name of "jsonDoc"

So the document name is really any name you want to use; then you have
used that same document name in the Path + journeys which matches the json
data Eg
{
"journeys": [
{
"booking_ref_num": "11111",

so the parsing starts at journeys.

I am just not following why the path was required or how it could have
been coded to not use the path ? Or is this specifically related to using
a handler ?

When the %handler is used, then number of records processed before the
%handler is called is controlled by the dim of the data structure defined
on the handler procedure interface ? Is that correct ?

Again thank you very much for the information provided and the time you
have spent.

Thanks

Don

From: "Jon Paris" <jon.paris@xxxxxxxxxxxxxx>
To: "Midrange Systems Technical Discussion"
<midrange-l@xxxxxxxxxxxxxxxxxx>
Date: 12/12/2020 01:46 AM
Subject: Re: DATA-INTO where number of records not known
Sent by: "MIDRANGE-L" <midrange-l-bounces@xxxxxxxxxxxxxxxxxx>

Don - this is working code. Set at the moment to handle 4 at a time but
can be any number.

Processing from a file but could equally be a variable.

**free
Ctl-Opt DftActGrp(*No);

dcl-ds bookingData_T qualified template;
booking_ref_num varchar(30);
journey_id zoned(9);
device_type varchar(20);
my_sa_gov_username varchar(100);
vehicle_plate varchar(10);
current_journey_status varchar(20);
start_journey_calculated_datetime varchar(25);
end_journey_calculated_datetime varchar(25);
full_journey_offline varchar(10);
satss_subsidy_from_card varchar(25);
payment_total_fare_in_cents varchar(15);
payment_subsidy_applied_cents varchar(15);
payment_passenger_to_pay_in_cents varchar(15);
payment_levy_cents varchar(15);
trumps_driver_id varchar(10);
location_start_lookup varchar(20);
location_end_lookup varchar(20);
created_on_utc varchar(25);
end-ds;

Dcl-S JSON_Input varchar(50) Inz('/Home/Paris/jsonstuff/journeys.json');

Dcl-S count Int(10) Inz; // Will contain total elements processed on
completion

DATA-INTO %Handler( HandleBooking: count )
%Data( JSON_Input : 'doc=file path=journeyData/journeys
case=any' )
%Parser( 'YAJL/YAJLINTO' : '{ "document_name": "journeyData" }'
);

Dsply ( 'Total count is ' + %Char( count ) );

*InLr = *On;

Dcl-Proc HandleBooking;

Dcl-Pi *N Int(10);
count Int(10);
booking LikeDS(bookingData_T) Dim(4) Const; // Handle 99 at a time
numElements Int(10) Value;
End-Pi;

Dcl-S i Int(10);

for i = 1 to numElements;
Dsply ( 'Booking: ' + booking(i).booking_ref_num );
Dsply ( ' Id: ' + %Char( booking(i).journey_id )); // Do your
processing
endfor;

// add current count to total in Comm Area and display current count
count += numElements;
return 0;

End-Proc;

On Dec 10, 2020, at 7:07 PM, Don Brown via MIDRANGE-L

<midrange-l@xxxxxxxxxxxxxxxxxx> wrote:

Thanks Jon,

Sorry I thought you wanted the data structure.

Here is a sample of the json

{
"journeys": [
{
"booking_ref_num": "11111",
"journey_id": 751,
"device_type": "ANDROID",
"my_sa_gov_username": "fy8420.wright@xxxxxxxxxxxxxx",
"vehicle_plate": "TAXI1234",
"current_journey_status": "COMPLETED",
"start_journey_calculated_datetime":
"2020-09-22T02:55:26.972000",
"end_journey_calculated_datetime":
"2020-09-22T02:55:45.593000",
"full_journey_offline": false,
"satss_subsidy_from_card": "PERCENT_50",
"payment_total_fare_in_cents": 11111,
"payment_subsidy_applied_cents": 2000,
"payment_passenger_to_pay_in_cents": 9011,
"payment_levy_cents": 100,
"trumps_driver_id": "FY8420",
"location_start_lookup": "LOCKLEYS",
"location_end_lookup": "LOCKLEYS",
"created_on_utc": "2020-09-22T02:55:26.827912"
},
{
"booking_ref_num": "1113",
"journey_id": 741,
"device_type": "ANDROID",
"my_sa_gov_username": "fy8420.wright@xxxxxxxxxxxxxx",
"vehicle_plate": "TAXI4019",
"current_journey_status": "COMPLETED",
"start_journey_calculated_datetime":
"2020-09-22T02:13:33.919000",
"end_journey_calculated_datetime":
"2020-09-22T02:13:48.987000",
"full_journey_offline": false,
"satss_subsidy_from_card": "PERCENT_75",
"payment_total_fare_in_cents": 552,
"payment_subsidy_applied_cents": 339,
"payment_passenger_to_pay_in_cents": 113,
"payment_levy_cents": 100,
"trumps_driver_id": "FY8420",
"location_start_lookup": "LOCKLEYS",
"location_end_lookup": "LOCKLEYS",
"created_on_utc": "2020-09-22T02:13:33.760255"
},
{
"booking_ref_num": "1116",
"journey_id": 744,
"device_type": "ANDROID",
"my_sa_gov_username": "fy8420.wright@xxxxxxxxxxxxxx",
"vehicle_plate": "TAXI4013",
"current_journey_status": "COMPLETED",
"start_journey_calculated_datetime":
"2020-09-22T02:15:14.268000",
"end_journey_calculated_datetime":
"2020-09-22T02:15:34.886000",
"full_journey_offline": false,
"satss_subsidy_from_card": "PERCENT_50",
"payment_total_fare_in_cents": 4588,
"payment_subsidy_applied_cents": 2000,
"payment_passenger_to_pay_in_cents": 2488,
"payment_levy_cents": 100,
"trumps_driver_id": "FY8420",
"location_start_lookup": "LOCKLEYS",
"location_end_lookup": "LOCKLEYS",
"created_on_utc": "2020-09-22T02:15:14.056589"
},
{
"booking_ref_num": "123456",
"journey_id": 862,
"device_type": "IOS",
"my_sa_gov_username": "fy8420.wright@xxxxxxxxxxxxxx",
"vehicle_plate": "TAXI4000",
"current_journey_status": "COMPLETED",
"start_journey_calculated_datetime": null,
"end_journey_calculated_datetime": null,
"full_journey_offline": true,
"satss_subsidy_from_card": "PERCENT_75",
"payment_total_fare_in_cents": 5555,
"payment_subsidy_applied_cents": 3000,
"payment_passenger_to_pay_in_cents": 2455,
"payment_levy_cents": 100,
"trumps_driver_id": "FY8420",
"location_start_lookup": "PLYMPTON",
"location_end_lookup": "PLYMPTON",
"created_on_utc": "2020-10-19T01:12:49.054602"
},
{
"booking_ref_num": "2",
"journey_id": 816,
"device_type": "IOS",
"my_sa_gov_username": "fy8420.wright@xxxxxxxxxxxxxx",
"vehicle_plate": "TAXI4444",
"current_journey_status": "COMPLETED",
"start_journey_calculated_datetime": null,
"end_journey_calculated_datetime": null,
"full_journey_offline": true,
"satss_subsidy_from_card": "PERCENT_75",
"payment_total_fare_in_cents": 2255,
"payment_subsidy_applied_cents": 1616,
"payment_passenger_to_pay_in_cents": 539,
"payment_levy_cents": 100,
"trumps_driver_id": "FY8420",
"location_start_lookup": null,
"location_end_lookup": null,
"created_on_utc": "2020-09-25T01:44:39.644839"
},
{
"booking_ref_num": null,
"journey_id": 780,
"device_type": "IOS",
"my_sa_gov_username": "fy8420.wright@xxxxxxxxxxxxxx",
"vehicle_plate": "TAXI4000",
"current_journey_status": "CANCELLED",
"start_journey_calculated_datetime":
"2020-09-23T05:23:29.729000",
"end_journey_calculated_datetime":
"2020-09-23T07:30:06.729000",
"full_journey_offline": false,
"satss_subsidy_from_card": null,
"payment_total_fare_in_cents": null,
"payment_subsidy_applied_cents": null,
"payment_passenger_to_pay_in_cents": null,
"payment_levy_cents": null,
"trumps_driver_id": "FY8420",
"location_start_lookup": "PLYMPTON",
"location_end_lookup": null,
"created_on_utc": "2020-09-23T05:24:05.977445"
}
]
}

Don Brown
Director

MSD Information Technology
t: 07 3368 7888
m: 0408 751 644
e: dbrown@xxxxxxxxxx

https://urldefense.proofpoint.com/v2/url?u=http-3A__www.msd.net.au&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=Kys-lxRCMpPr7up01Pp1FRjOe49ne6imWwi1b-ue8yQ&m=ZRh5z_aH-xPHHEtm6Jtl3AVodsncioW-uOqKjvQmkck&s=H009N5ESiEFkBnTfQn5Mxj3bzpb3c6yBvXnurtkkEiw&e=

From: "Jon Paris" <

https://urldefense.proofpoint.com/v2/url?u=http-3A__jon.paris-40partner400.com&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=Kys-lxRCMpPr7up01Pp1FRjOe49ne6imWwi1b-ue8yQ&m=ZRh5z_aH-xPHHEtm6Jtl3AVodsncioW-uOqKjvQmkck&s=w02WzcuBcZR-EkBJeWoTXLmLwrQkLU5d5ao3bqrPuG0&e=

To: "Midrange Systems Technical Discussion"
<midrange-l@xxxxxxxxxxxxxxxxxx>
Date: 11/12/2020 09:52 AM
Subject: Re: DATA-INTO where number of records not known
Sent by: "MIDRANGE-L" <midrange-l-bounces@xxxxxxxxxxxxxxxxxx>

Your understand of %Handler is incorrect. It will be called for every n

elements found - where n is the number of elements defined in the
handler's input parameter.

For example - this handler procedure would receive 4 at a time due to

the

Dim.

Dcl-Pr HandleCompany Int(10);
commArea Int(10);
company Char(32) Dim(4) Const;
numElements Int(10) Value;
End-Pr;

Also if you are dealing with a repeat at the top level of the document

Charles' example shows then there is no need to provide a count - RPH
already places one in the PDSD at offset 372 which is populated
automatically when the target is an array or DS array.

Dcl-Ds progStatus psds;
xmlElements Int(20) POS(372);
End-Ds;

I'll gladly show you an example of %Handler but it would be easier if

you

showed the json as I said earlier.

On Dec 10, 2020, at 6:41 PM, Don Brown via MIDRANGE-L

<midrange-l@xxxxxxxxxxxxxxxxxx> wrote:

Thanks Charles,

Ok so you have confirmed the count will provide the total number of
records received even if it exceeds the array limit.

And like you trying to find an example of %handler has not been
successful.

But my reading so far of %handler is that it would be called for each
record received.

Hopefully someone on the list may have a example of how they used
%handler.

Thank you for your reply

Don

From: "Charles Wilt" <charles.wilt@xxxxxxxxx>
To: "Midrange Systems Technical Discussion"
<midrange-l@xxxxxxxxxxxxxxxxxx>
Date: 11/12/2020 09:27 AM
Subject: Re: DATA-INTO where number of records not known
Sent by: "MIDRANGE-L" <midrange-l-bounces@xxxxxxxxxxxxxxxxxx>

Have you looked at using YAJLGEN to generate the DS & DATA-INTO call

you

need?

You can do it yourself or course, but YAJLGEN sure makes for an easy

first

pass.

Basically, in the data-into you include "count prefix" like so:
countprefix=num_

then in your DS, you include a field
dcl-ds jsonDoc;
num_SOMEARRAY int(10);
SOMEARRY char(10) dim(9999);
end-ds;

YAJLINTO will load the array and the value in num_SOMEARRAY will be how
many entries were loaded to the array.

Now if your # of entries exceeds the max array size for RPG, then I

think

%handler() is your solution.

I've only seen it used once, and that was for XML...but it only

processes

part of the doc at a time before passing the data to your handler. A
quick
serach didn't turn up a good exmaple of using it for JSON.

Teraspace and a based pointer might be another option if you're dealing
with really large documents...just a thought.

Charles

On Thu, Dec 10, 2020 at 3:01 PM Don Brown via MIDRANGE-L <
midrange-l@xxxxxxxxxxxxxxxxxx> wrote:

(Sorry, not sure if this is should be the RPG list only or the

Midrange

list)

We are using data-into with YAJLINTO to process a json file we

retrieve

from an external site using a web service.

The web service does not provide any options as to controlling the

number

of records to be returned.

The json returned does not have an element for the number of records
provided.

If I put a count at the highest level will this be set correctly if

the

number of records retrieved exceeds the array size ? And if yes how do

I

then process the records past that point ?

I don't think the %handler option helps - well the way I read the doco

it

wont.

How do I either check how many records have been returned or process a
json string that exceeds the number of elements in my receiving array

Thanks for all suggestions

Don

--
This email has been scanned for computer viruses. Although MSD has

taken

reasonable precautions to ensure no viruses are present in this email,

MSD

cannot accept responsibility for any loss or damage arising from the

use

of

this email or attachments..
--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit:

https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.midrange.com_mailman_listinfo_midrange-2Dl&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=Kys-lxRCMpPr7up01Pp1FRjOe49ne6imWwi1b-ue8yQ&m=-PWDCmLBxYYH13bcXYxWkNMenCdRQ8fUXqdj9htfCHI&s=hirmV788fhJEaASEQO0FXq6jTKco9QvzBTWHslQpkqM&e=

or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at

https://urldefense.proofpoint.com/v2/url?u=https-3A__archive.midrange.com_midrange-2Dl&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=Kys-lxRCMpPr7up01Pp1FRjOe49ne6imWwi1b-ue8yQ&m=-PWDCmLBxYYH13bcXYxWkNMenCdRQ8fUXqdj9htfCHI&s=tHYTxFLSSsI2yghAwIccJb9r99wY9LjByFQrBw79hLU&e=

.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription

related

questions.

Help support

by shopping at

https://urldefense.proofpoint.com/v2/url?u=http-3A__amazon.com&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=Kys-lxRCMpPr7up01Pp1FRjOe49ne6imWwi1b-ue8yQ&m=-PWDCmLBxYYH13bcXYxWkNMenCdRQ8fUXqdj9htfCHI&s=speJM2dL4e6S7th_1_Ly_et4cxHogMzTq9dguU0Lde8&e=

with our affiliate

link:

https://urldefense.proofpoint.com/v2/url?u=https-3A__amazon.midrange.com&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=Kys-lxRCMpPr7up01Pp1FRjOe49ne6imWwi1b-ue8yQ&m=-PWDCmLBxYYH13bcXYxWkNMenCdRQ8fUXqdj9htfCHI&s=8RBI4zBnOjf4NTdgqoZgyuvbvA1Pv3x2v6XXe1i65Tg&e=

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit:

or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at

.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription

questions.

Help support

by shopping at

with our affiliate link:

--
This email has been scanned for computer viruses. Although MSD has

taken

reasonable precautions to ensure no viruses are present in this email,

MSD

cannot accept responsibility for any loss or damage arising from the use

of this email or attachments..

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing

list

To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit:

https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.midrange.com_mailman_listinfo_midrange-2Dl&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=Kys-lxRCMpPr7up01Pp1FRjOe49ne6imWwi1b-ue8yQ&m=uRtE9vPPYLtusSRZFl6RWSl6ruCAdQc4fpqehT9MZVI&s=s-UVJvp-V7kID_eB-BZx1EcmNyGcsCSvGx1g49X5ea4&e=

or email: MIDRANGE-L-request@xxxxxxxxxxxxxxxxxx
Before posting, please take a moment to review the archives
at

https://urldefense.proofpoint.com/v2/url?u=https-3A__archive.midrange.com_midrange-2Dl&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=Kys-lxRCMpPr7up01Pp1FRjOe49ne6imWwi1b-ue8yQ&m=uRtE9vPPYLtusSRZFl6RWSl6ruCAdQc4fpqehT9MZVI&s=0I-3IGAFnDbyUKtlcjOSQvGKkWF74DQBb8g47wo2Org&e=

.

Please contact support@xxxxxxxxxxxxxxxxxxxx for any subscription

questions.

Help support

by shopping at

https://urldefense.proofpoint.com/v2/url?u=http-3A__amazon.com&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=Kys-lxRCMpPr7up01Pp1FRjOe49ne6imWwi1b-ue8yQ&m=uRtE9vPPYLtusSRZFl6RWSl6ruCAdQc4fpqehT9MZVI&s=kvWbp8tjhCS6eng4RUKKxrPa3VM-wF_540vhB2kfUwA&e=

with our affiliate link:

https://urldefense.proofpoint.com/v2/url?u=https-3A__amazon.midrange.com&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=Kys-lxRCMpPr7up01Pp1FRjOe49ne6imWwi1b-ue8yQ&m=uRtE9vPPYLtusSRZFl6RWSl6ruCAdQc4fpqehT9MZVI&s=azr1KNNR_1rfmk1WhDt4vfuD_KRJxzbglnQWcJ98LPc&e=

--
This is the Midrange Systems Technical Discussion (MIDRANGE-L) mailing
list
To post a message email: MIDRANGE-L@xxxxxxxxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit:

https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.midrange.com_mailman_listinfo_midrange-2Dl&d=DwICAg&c=euGZstcaTDllvimEN8b7jXrwqOf-v5A_CdpgnVfiiMM&r=Kys-lxRCMpPr7up01Pp1FRjOe49ne6imWwi1b-ue8yQ&m=uRtE9vPPYLtusSRZFl6RWSl6ruCAdQc4fpqehT9MZVI&s=s-UVJvp-V7kID_eB-BZx1EcmNyGcsCSvGx1g49X5ea4&e=