Re: current state of xml-into v6r1

On 2012/7/27 1:11 PM, James Franz wrote:

I am reviewing options for a project where will receive an export file from a non-i system, in xml, ftp'd to us, with a dozen tables in the single xml (order header,vendor, customer, order detail, etc, ... next order header...) where there will be perhaps half million orders. We have specs only, no file yet. The dtd seems straight forward, plus a separate excel of each column definition.
Before we head down the xml-into route, is there any limits for processing a file of millions of records? No single column will exceed perhaps 1000 bytes.
We have no xml tools (and no budget for this import).
We do have the recent v6r1 namespace ptfs.

For sure with millions of records, you'd need to use the %HANDLER form of XML-INTO, where your handler procedure would handle X records at a time.

But you might not be able to use XML-INTO for that particular XML document anyway.

If the order-details are nested within the order-header, like this,
<order-header>
<order-detail ... />
<order-detail ... />
<order-detail ... />
</order-header>
then XML-INTO would work with a %HANDLER as long as you could fit the contents of the order-header + all its order-details in one RPG data structure (max size 16MB). You would define an RPG data structure "header" with subfields for the vendor, customer, and an array for all the detail. Then you would define your handler to receive an array of header structures, maybe only DIM(1), depending on the size of the header structure. Your handler would get called over and over with a few header structures at a time.

But if the order-details are at the same level in the XML as the order-header,
<order-header ... />
<order-detail ... />
<order-detail ... />
<order-header ... />
etc
then XML-INTO would be very difficult to use because you could only get either the order-detail or the order-header in any one XML-INTO. In that case, XML-SAX or any other SAX-type XML parser would be easier/better.