On 18 Apr 2013 07:11, Åke Olsson wrote:
Problem is as follows:
A file with text field in Unicode (UTF-16) is read by a job running
under CCSID 870 (Czech).
The text field has a character value that is valid in many language
locales but not in Czech.
As a result these characters are replaced with X'3F' in the
conversion process between Unicode and EBCDIC. 3F causes lots of
problem when it is passed on later in the chain.
Question: Is there anywhere where you can configure the system to
replace non-existing characters with some other hex code (preferably
X'40' since that will not cause problems later in the chain)?
I am not aware of any ability to override the 0x3F as the EBCDIC
substitution character. I expect that given a database file field for
which the data will not properly convert into the desired CCSID, the
condition will need to be addressed after-the-fact. Basically, what
Bruce said. But FWiW, even if the OP already is aware, for the archives:
I believe native\RLA informs of the possibility of substitution
characters upon the request to Open a file, but I do not recall what
specific feedback occurs for any particular row. I think maybe a data
mapping error CPF5029 occurs, and a preceding error message CPF5035 RC21
diagnoses "21: The data has been converted from one CCSID to another.
However, substitution characters were used for characters which could
not be converted." to inform of the condition on any particular field(s)
for that record; but the data had been copied to the buffer, with the
substitution character.
The SQL informs of the condition with a warning that can be used to
condition a /recovery action/ such as another SQL request to perform the
following [or similar]:
Set :stringVar REPLACE(:stringVar, x'3F', x'40')
http://pic.dhe.ibm.com/infocenter/iseries/v7r1m0/topic/nls/rbagssqlconversrules.htm
_i SQL conversion rules i_
"When two strings are compared, one of the strings is first converted,
if necessary, to the coded character set of the other string.
...
A warning occurs if a character of the string is converted to the
substitution character.
..."
http://pic.dhe.ibm.com/infocenter/iseries/v7r1m0/topic/rzala/rzalaccl.htm
_i Listing of SQLSTATE values i_
"The tables in this topic provide descriptions of SQLSTATE codes that
can be returned to applications...
...
Table 2. Class Code 01: Warning
...
SQLSTATE value: 01517
Meaning: A character that could not be converted was replaced with a
substitute character.
SQLCODE Value: +335
midrange.com
