Re: Character Encoding

[Bruce Vining <bvining@xxxxxxxxxxxxxxx>]

I am totally unfamiliar with MQSeries from an actual usage point of view, but if you want to determine the encoding that is currently in use then:
 

send a message of known contents to the MQSeries queue
dump that entry (or the whole queue if there isn't an entry specific mechanism provided by MQ) in hex
 
If you can then post the original message and the dumped message it should be a snap to determine the encoding being used.  Please make sure the message does contain extended Latin-1 characters such as the Ñ as that will provide an easy codepoint to distinguish between CCSIDs 819 and 1208.
 
Based on your reported results of using Notepad and specifying 8859-1 I suspect however that you are indeed using UTF8 encoding.  The reason is that in UTF8 the Ñ would be encoded as x'C391' which, if processed as 8859-1 (CCSID 819), would be treated as à followed by an undefined code point (8859-1 reserves x'91' as a control) with who knows what graphic character.
 
So I suspect (again out of a certain amount of ignorance) the real question comes down to what this Biztalk is keying off of in order to understand the encoding of the message.  It does not appear to be 1208/UTF8...

Bruce
Bruce Vining Services
507-206-4178

--- On Thu, 12/11/08, Fleming, Greg (ED) <GFLEMING@xxxxxxxxxxxxxxxxxxxx> wrote:

From: Fleming, Greg (ED) <GFLEMING@xxxxxxxxxxxxxxxxxxxx>
Subject: Character Encoding
To: "Midrange Systems Technical Discussion" <midrange-l@xxxxxxxxxxxx>
Date: Thursday, December 11, 2008, 2:54 PM

I hope I can ask this question in a way that makes sense.

I'm trying to get my head around character encoding and CCSID's with
regard to data being transferred off of the iSeries.

We send orders in XML format  to an MQSeries queue, and use Microsoft Biztalk
to pick those up from the queue. 

If we include characters like Ñ, Ü, etc, Biztalk doesn't recognize them
and chokes.  

We are NOT specifying the encoding in the XML header, so it is apparently
defaulting to UTF-8.

Using notepad, I altered our document to include the encoding in the XML header
and tried ISO-8859-1, which allowed the document to go through OK, but the
character was changed from Ñ to something else that looks like an 'A'
with a tilde on top, and a square box following that. 

I read on a website that randomly assigning ISO-8859-1 could solve an immediate
issue, but might cause different issues later if my document isn't actually
ISO-8859-1 encoded.

 

If I leave the message on the MQueue, and browse it using the MQueue Explorer,
the Ñ is still intact, so I don't think the translation from EBCDIC to
ASCII is actually causing the problem.  It seems that the issue is telling
Biztalk what kind of encoding the document is actually using. 

 

So the meat of my question is this: How can I tell what the actual encoding of
the document is ?   Is there a correlation between the IBM CCSID (37 for the job
that is sending these messages to the queue), and the resulting PC encoding ? 
Or is there a system value on the iSeries that determines what encoding a
document will have when the iSeries translates it from EBCDIC to ASCII ? 

 

Thanks

 

Greg Fleming

Senior Programmer/Analyst

Everglades Direct, Inc.