David FOXWELL wrote:
Our system is set to CCSID 65535. A programmer wrote some code containing the <!!> characters to replace <concat>. [snip] The program actually running the SQL instructions saw something other than <!!>.
David:
I _think_ that your fundamental problem is buried in those words.
AFAIK, the earlier programmer placed '||' (vertical bars; "pipes";
x'7C7C'?)into the SQL statement and those are what are in the SQL
statement now. I _think_ that a hex dump of the physical bytes
should show the correct hex values. If '!!' (exclamation points or
'marks'; x'2121'?) had actually been in the physical statements,
they would have been rejected as syntax errors. However...
When the bits that make up those two bytes get converted into
visible characters for you to look at, either on a display or
printer, the bits get converted according to whatever conversion
tables are in effect for the display or print operation. The earlier
programmer could have had the same setup as you have; he would
indeed have typed the exclamation points even though they would be
stored as x'7C7C'. The same inappropriate conversion could have
happened back then.
With a system QCCSID of 65535, conversions are allowed to look like
almost anything. A display device that has a particular
keyboard/language setting being used by a user profile with
particular language identifier, country/region identifier, CCSID,
etc., might show different character "shapes" for a given sequence
of (65535=no conversion) bits than what would be _seen_ on a display
device sitting next to it. Exact same bits, but different "shapes"
drawn on the two screens.
One user in a given set of circumstances gets x'7C' drawn as a
vertical bar; the next user sees the same bits drawn as an
exclamation point.
It's not particularly different from using FTP to do a BINARY
transfer of an ASCII text file and then trying to view it under an
EBCDIC CCSID. The BINARY transfer avoided doing the appropriate
translation/conversion at the bit level when the bits were written
to storage.
The hex values are the same. But when viewed through an
inappropriate character set/code page conversion to visual
representation, the characters become... garbage.
You could probably see the characters as vertical bars if you
adjusted your display session appropriately. Maybe it's in the *DEVD
that you're using; maybe it's in the conversions defined in your
emulator settings; maybe it's even in the country/region or language
settings on your PC. There are a lot of possible points depending on
what all the bits travel through.
If humans would read everything as hex or binary values, there might
not be such a thing as 'variant' and 'invariant' characters. But
because we require our machines to draw bits as visible characters
with different shapes in different languages, just for our
convenience, we have to make sure that we tell it the right ways to
do the drawings (or light up pixels or deposit ink droplets).
We do that by ensuring that CCSID tags go along with the collections
of bits. The machines can then manipulate bits at all input and
output points, as well as during all collating/sorting/comparison
points, wherever those bits get processed.
Having a major intermediate process that skips conversion (QCCSID
65535) is trouble. As soon as a device gets attached that expects a
different interpretation of bit patterns, things get ugly.
Now that I've confused everything beyond hope, I'll go back to
work... I'm sure that some discussion will bring clarity eventually.
Tom Liotta
midrange.com
