Re: Content-type issue with application/json and application/javascript

Mark - As Henrik said, we can't really dictate what content types are
valid. There's a lot of stuff that "works" in the world of HTTP servers
that isn't right. While browsers are usually flexible, not everything is.
Some frameworks are getting very anal about this kind of stuff because
there are too many unpatched exploits floating around to make assumptions.

In our case, we are looking at writing an API that will be consumed by
various frameworks and programming languages. The programming libraries
for consuming JSON data are going to verify the data is in a valid format.
Some will let things slip and others will not. The standards say that the
official mime type of JSON is "application/json" and that the body should
be have UTF-8 encoding. Probably a good idea to follow that so I have a
greater chance of the API working everywhere it needs to. Believe it or
not, I have ran it to issues when XML was not UTF-8 encoded.

I would rather deal with this and get right from the start. I would rather
not deal with an emergency fix down the road when some giant, new client
can't get our API working.




On Tue, Aug 25, 2015 at 1:50 PM, Mark Murphy/STAR BASE Consulting Inc. <
mmurphy@xxxxxxxxxxxxxxx> wrote:

Just out of pure ignorance, since application/json causes conversion
issues, and text/json appears to work properly, why do you want to use
application/json? Are there issues that text/json causes on the client side?

Mark Murphy
STAR BASE Consulting, Inc.
mmurphy@xxxxxxxxxxxxxxx


-----Matt Lavinder <mlavinder@xxxxxxxxxxxxxxxxxxx> wrote: -----
To: "Web Enabling the IBM i (AS/400 and iSeries)" <web400@xxxxxxxxxxxx>
From: Matt Lavinder <mlavinder@xxxxxxxxxxxxxxxxxxx>
Date: 08/25/2015 11:42AM
Subject: Re: [WEB400] Content-type issue with application/json and
application/javascript

Henrik -

No worries. I have no intention of changing our CGIConvMode, but I did
want to make sure I understood the implications. The bulk of our CGI
programs produce text/html and they work perfectly fine.

I plan to tweak the programs that produce JSON output and leave everything
else alone.

On Tue, Aug 25, 2015 at 11:36 AM, Henrik Rützou <hr@xxxxxxxxxxxx> wrote:

btw if you use binary or recieve application/xxxx content type the input
also
is recieved as binary without conversion to ebcdic

On Tue, Aug 25, 2015 at 5:34 PM, Henrik Rützou <hr@xxxxxxxxxxxx> wrote:

matt

you are right

On Tue, Aug 25, 2015 at 5:31 PM, Matt Lavinder <
mlavinder@xxxxxxxxxxxxxxxxxxx> wrote:

Kevin -

It's OK because I know enough to get what he was saying. It makes it
pretty clear in the HTTP Server documentation that if you add
"charset=utf-8"
(really, charset=anything) that no conversion occurs on the body.

I have never tried CGIConvMode %BINARY%, but wouldn't that mean that

all

your content has to be UTF-8, including output with

"content-type:text/*"?

On Tue, Aug 25, 2015 at 11:23 AM, Kevin Turner <
kevin.turner@xxxxxxxxxxxxxxxxxxxx> wrote:

Henrick

I know English is not your first language, but I couldn't understand

that

at all :)

But anyway, my observations are not relevant because we always use
"CGIConvMode BINARY"

Kevin

-----Original Message-----
From: WEB400 [mailto:web400-bounces@xxxxxxxxxxxx] On Behalf Of

Henrik

Rützou
Sent: 25 August 2015 16:19
To: Web Enabling the IBM i (AS/400 and iSeries) <

web400@xxxxxxxxxxxx>

Subject: Re: [WEB400] Content-type issue with application/json and
application/javascript

apache doesn't neither converts from an ebcdic body if you send then

the

body also has to be pre-encoded in utf-8

Content-Type: text/html; charset=utf-8

On Tue, Aug 25, 2015 at 5:14 PM, Kevin Turner <
kevin.turner@xxxxxxxxxxxxxxxxxxxx> wrote:

I am pretty sure we always ensure the data is in UTF-8 when

sending

content that has a content type that is anything but "text/...."



-----Original Message-----
From: WEB400 [mailto:web400-bounces@xxxxxxxxxxxx] On Behalf Of

Matt

Lavinder
Sent: 25 August 2015 16:12
To: Web Enabling the IBM i (AS/400 and iSeries) <

web400@xxxxxxxxxxxx>

Subject: Re: [WEB400] Content-type issue with application/json and
application/javascript

Hello Henrick. That's sort of what I suspected. Do I have to

change

the

CGI program to produce ASCII output, or is there a way to tell

Apache

to do an EBCDIC to ASCII conversion on an "application/javascript"
content type?

On Tue, Aug 25, 2015 at 10:56 AM, Henrik Rützou <hr@xxxxxxxxxxxx>

wrote:

apache converts ebcdic to utf-8 if the content type is text/...
othervise not.


On Tue, Aug 25, 2015 at 4:52 PM, Matt Lavinder <
mlavinder@xxxxxxxxxxxxxxxxxxx> wrote:

We have several CGI programs that generate javascript, usually

JSON.

For years we have set the content type to text/plain or
text/javascript. I'd like to start using the more standard
"application/json", but I am

hitting

a major snag. The output is garbage. I get the exact same

output

when I set the content-type to application/javascript.

Just to verify this was related to CGI output, I used AddType

to

change

the

content-type of static JavaScript files to

"application/javascript".

Those

files look fine, so this must have something to do with it

being

CGI output.

I don't really know what is up with the output, but I know it
isn't EBCDIC. It doesn't convert. Garbage in, garbage out.
Changing the

content

type back to text/javascript sets everything right.

Anyone have any guesses what is going on here? We use CGIDEV2
under IBM i 7.1. The default CCSID of our CGI jobs is 37.

Just so we are clear on one thing. We had the following

settings

in our HTTP Conf:

DefaultFsCCSID 037
CGIConvMode %%EBCDIC/EBCDIC%%

I commented both settings out and it had no impact on the

output

whatsoever. All of our CGI programs that use content type of

"text/*"

seemed to work. The ones using "application/javascript" or
"application/json" for content type still looked like garbage.
The only guess I have is that mod_cgi sees that the

content-type

starts with "application" and treats the output as binary,

but I

have no idea how I

can

tell mod_cgi to treat "application/javascript" as text, or if

that

is

even

the issue.

I am hoping someone has an idea. I know Apache pretty well,

but

this one has me stumped.

--
Matt
--
This is the Web Enabling the IBM i (AS/400 and iSeries)

(WEB400)

mailing list To post a message email: WEB400@xxxxxxxxxxxx To
subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx Before posting, please

take

a moment to review the archives at

http://archive.midrange.com/web400.

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)
mailing list To post a message email: WEB400@xxxxxxxxxxxx To
subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx Before posting, please

take a

moment to review the archives at

http://archive.midrange.com/web400

.

--




*Matt Lavinder Programmer AnalystData Management Inc.Phone: (336)
573-5045Fax: (336) 573-5001*
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)
mailing list To post a message email: WEB400@xxxxxxxxxxxx To
subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives at
http://archive.midrange.com/web400.


___________________________________________
This email has been scanned by iomartcloud.
http://www.iomartcloud.com/


________________________________

NOTICE: The information in this electronic mail transmission is
intended by CoralTree Systems Ltd for the use of the named

individuals

or entity to which it is directed and may contain information that

is

privileged or otherwise confidential. If you have received this
electronic mail transmission in error, please delete it from your
system without copying or forwarding it, and notify the sender of

the

error by reply email or by telephone, so that the sender's address

records can be corrected.

----------------------------------------------------------------------

----------


CoralTree Systems Limited
Company Registration Number 5021022.
Registered Office:
12-14 Carlton Place
Southampton
Hampshire
SO15 2EA
VAT Registration Number 834 1020 74.
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)
mailing list To post a message email: WEB400@xxxxxxxxxxxx To
subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives at
http://archive.midrange.com/web400.

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)

mailing

list To post a message email: WEB400@xxxxxxxxxxxx To subscribe,
unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives at
http://archive.midrange.com/web400.


___________________________________________
This email has been scanned by iomartcloud.
http://www.iomartcloud.com/


________________________________

NOTICE: The information in this electronic mail transmission is

intended

by CoralTree Systems Ltd for the use of the named individuals or

entity

to

which it is directed and may contain information that is privileged

or

otherwise confidential. If you have received this electronic mail
transmission in error, please delete it from your system without

copying or

forwarding it, and notify the sender of the error by reply email or

by

telephone, so that the sender's address records can be corrected.

--------------------------------------------------------------------------------

CoralTree Systems Limited
Company Registration Number 5021022.
Registered Office:
12-14 Carlton Place
Southampton
Hampshire
SO15 2EA
VAT Registration Number 834 1020 74.
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--




*Matt Lavinder Programmer AnalystData Management Inc.Phone: (336)
573-5045Fax: (336) 573-5001*
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400)

mailing

list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>

--
Regards,
Henrik Rützou

http://powerEXT.com <http://powerext.com/>
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing
list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.

--




*Matt Lavinder Programmer AnalystData Management Inc.Phone: (336)
573-5045Fax: (336) 573-5001*
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing
list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.
--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing
list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: http://lists.midrange.com/mailman/listinfo/web400
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at http://archive.midrange.com/web400.