CGIDEV2 doesn't like CCSID 1208?

[Adam Glauser <adamglauser@xxxxxxxxxxxx>]

I have a program which uses CGIDEV2 to create an XML file.  It reads a 
template from the root filesystem, fills in the template, and writes the 
completed XML file to the same directory (with a different filename, of 
course).  I've worked around the problem, but it's nagging me and I 
can't figure out why this is happening.

I realized as I was updating the program for the next tax year that I 
was generating the file as CCSID 819 (ISO-8859-1), but my <?xml ...?> 
header stated UTF-8.  I must have just used a text editor to convert it 
before sending it last year.

I tried changing the output file to be CCSID 1208 (UTF-8) by changing 
the CCSID parameter on wrtHTMLtoSTMF(), but I get errno=3490 (ECONVERT) 
when CGIDEV2 attempts to re-open the output file to write the output.

<goryDetails>
wrtHTMLtoSTMF does something like this:
  create a temporary filename (desired filename + 'TEMP' + job number)
  unlink tempfile if it exists
  open tempfile to create it (empty with proper CCSID) with:
     C                   eval      FileHandle = open(
     C                             StmfTemp:O_CREAT + O_WRONLY +
     C                             O_TRUNC + O_TEXTDATA + O_CODEPAGE +
     C                             O_LARGEFILE:
     C                             S_IRWXU + S_IROTH + S_IXOTH:CodePage)
where CodePage is 1208, which I've passed as a parameter.

  re-open tempfile for writing:
     C                   eval      FileHandle = open(
     C                             StmfTemp:O_CREAT + O_WRONLY +
     C                             O_TRUNC + O_TEXTDATA:
     C                             S_IRWXU + S_IROTH + S_IXOTH)

At this point, I get the ECONVERT error.  I found a forum post* from 
Scott Klement stating that CGIDEV2 doesn't support multi-byte CCSIDs for 
writing to STMF, but I don't think UTF-8 counts.  I'm also not sure why 
the above would fail for multi-byte CCSIDs, the documentation for 
O_TEXTDATA doesn't mention such a limitation.
</goryDetails>

If I simply change the CodePage parameter to 819 and my template to use 
<?xml version="1.0" encoding="ISO-8859-1"?>, everything works fine.

FWIW, I one thing I tried was making the template file UTF-8 encoded.  I 
learned about the BOM bytes** that some software writes at the beginning 
of multi- or variable-byte encoded files.  CGIDEV2 doesn't like the BOM 
bytes, but seems to work fine with a template file that is CCSID 1208.

I don't know where to look next - any suggestions?
Adam


* 
http://forums.systeminetwork.com/isnetforums/archive/index.php?t-47517.html 
(or http://tinyurl.com/6d4wk8 )

** 
http://en.wikipedia.org/wiki/Byte-order_mark#Representations_of_byte_order_marks_by_encoding 
(or http://tinyurl.com/5rojua)