Re: Allowing accented characters in the database

On 2023-08-29 8:28 a.m., Patrik Schindler wrote:

Hello Barbara,

Am 29.08.2023 um 01:57 schrieb Barbara Morris <bmorris@xxxxxxxxxx>

1. If the job CCSID is 65535, database doesn't do the conversion, but RPG (by default) assumes that the conversion was done.

Which is a side effect of the default QCCSID system variable never being touched, if I understand correctly, yes?

Yes, I agree.

2. The UTF-8 data used by the current job should all be able to be converted to the current job CCSID.

Example: a 👍 is never to be found in the 37 code page. Although I think it would be awesome beyond belief to actually see this in a 5250 session. :-D

It's not just emojis. The 37 code page has Latin characters, so it supports all the European languages, but it doesn't have Cyrillic, or Chinese, or Thai etc characters. The Unicode field might have data from many character sets. It would be ok if the application was setup so that a particular job only needs to look at records that can convert to the job CCSID. But if the application was going to look at all the records, then it should avoid conversion to the job CCSID.

Again, this is only an issue with UTF-8 data in database files. There is no similar issue with UCS-2 and UTF-16 data in database files.

Why should one mess with UTF-8 in the i database then? As long as automatic conversion works towards the end user UI, I see no reason to introduce workarounds to prohibit any conversion just for the sake of "UTF-8 in the database"? To me, this smells quite a bit like the ever-touched-QCCSID issue.

As far as I know, this issue with UTF-8 in the database is an RPG issue, mainly due to the fact that RPG did not become "aware" of alphanumeric CCSIDs until V7R2.