Re: [EXTERNAL] Re: Rise of Node

"I'd be curious to learn what you end up with concerning deploying of compiled RPG objects."
I hadn't got round to tackling that one yet, I was hoping that the RPM announcement might help in that respect, so that it might be possible to package source and binaries up in RPM. The client I'm looking at this for has no change management whatsoever except what I've done for then in BitBucket and does everything by hand so even having an automated build that created a save file they could transfer to their live machine by hand would be a considerable help!

"Could you post an example of how you do this?"
So, first off I'm referring to business logic in the sense of basic interactive "CRUD+" stuff, not batch processes and, in the applications we most frequently do, the business logic is usually making calculations over sets of data or enforcing business rules so it's nothing fancy or clever and mostly stuff that can be done in a single SQL statement. The CRUDdy parts are broken out into individual stored procedures, plus usually an additional one for the paged queries and any that make sense as standalone APIs, like CUST_SP_Block(...), CUST_SP_UnBlock(...), CUST_SP_AddAddress(...) etc., etc.. So each SP is usually quite compact, any shared logic, like validation is written as a separate SP and called from where it's needed (like insert, update or just on its own if the client wants to validate etc.). Our webservice simply executes the stored procedure POSTed to (after authority checking of course!) and passes the parameters in the request body, for example:
/web-service/CUST_SP_Query --> executes call CUST_SP_Query(PAGE => 1, PAGE_SIZE => 15, SEARCH_CRITERA => "Smith")
/web-service/CUST_SP_Add --> executes call CUST_SP_Add(....)
/web-service/CUST_SP_Delete....etc.
/web-service/CUST_SP_Block
/web-service/CUST_SP_AddAddress

The webservice does nothing more than provide an interface between the RESTful(ish!) API and the stored procedures and the request body always has the same format {"inputParms": {"PAGE": 1, "PAGE_SIZE": 15, "SEARCH_CRITERA": "Smith"}}, where the input parameters match those of the stored procedure. The output is also always the same structure, basic data about the SP called, the input and output parameters and then an array of result sets, each containing the column meta-data and the rows themselves. This makes life very easy for the client, because you always know what shape the data will be so you can often make quite generic code to manipulate and display it. It also makes it very easy to add new web services, just write a SP and it's automatically available. as an API endpoint.

We consider that the client of the API is dumb and shouldn't manipulate the data other than for presentation purposes. This means, for example, if the database contains a net and VAT amount, then the SQL will calculate the gross amount if it's needed, rather than have the *script client do this. Similarly, if the client needs subtotalling then this is also done in SQL not by the client summing the data themselves. Also, most interpretation of the data is done concretely in the server side SQL, so if "older than 30 days" means overdue, then the SQL will look up the definition of overdue and provide a flag in the data that the UI will use.

When it comes to updating/inserting/deleting rows in the database, this is again encapsulated into a stored procedure which uses the "begin atomic" statement to make sure that multiple updates are encapsulated and done safely.

Where I've written "client" above, you could read "server" if your server was a RESTful NodeJs application, for example, firing basic SQL queries at the database. I would argue that the NodeJs server side code should not manipulate the data at all either but the SQL should do it. Therefore each RESTful endpoint in NodeJs just executes a stored procedure, which if you boil that down further you come to the approach above because you might as make it generic then.

Another advantage I see with this way is that it is more secure. If your server can fire arbitrary SQLs at the database then it has to have sufficient authority to the database and thus any compromise of the server gives a free reign to the intruder. By encapsulating all of the logic in stored procedures and securing the stored procedures correctly the user id under which the web service is running only has to have *USE rights to the SPs and nothing else, the SPs can run under *OWNER, meaning you never have to give the webservice access to the raw database.

"I wouldn't have such a distaste for this approach if SQL stored
procs were easier to debug. I am a big fan of step debuggers."
I guess you already know this, but SPs are just C programs under the covers, so the ordinary debugger can be used to step through the SQL code or the underlying C code, you have to use a service job though. In any case, I always develop and test my SQL components interactively before pasting them into the SP body, which cuts down on the debugging effort in the first place, I can't remember the last time I actually had to debug a SP.

So far our applications haven't required us to use RPG stored procedures for the backend, apart from in the encrypting and decrypting of user tokens, which uses the system APIs. In terms of complicated business logic that you would see as being problematic done in SQL, give me an example and I'll try to explain how I'd deal with it (or suddenly realise I never thought it it 😲!)


________________________________
From: WEB400 <web400-bounces@xxxxxxxxxxxx> on behalf of Aaron Bartell <aaronbartell@xxxxxxxxx>
Sent: 20 March 2018 14:02
To: Web Enabling the IBM i (AS/400 and iSeries)
Subject: Re: [WEB400] [EXTERNAL] Re: Rise of Node

recently we use Git even for our IBM i sources

We're doing similar things. I look forward to the day this becomes the
norm. I'd be curious to learn what you end up with concerning deploying of
compiled RPG objects. That's what I love about deploying scripting apps;
put stream file(s) in new directory, point webserver at it, done.

That's a really good question, and I don't think we're as far as way as

you think.

Agreed.

I'd still strive to keep as much of the core business logic wrapped up in

SQL stored procedures but I'd try to split the functionality out to where
it's done best, e.g. financial calculations and record processing in SQL

Could you post an example of how you do this? I am all for business logic
like referential integrity, but I am guessing you are doing full-on logic
in SQL. I wouldn't have such a distaste for this approach if SQL stored
procs were easier to debug. I am a big fan of step debuggers.


Aaron Bartell
IBM i hosting, starting at $157/month. litmis.com/spaces


On Tue, Mar 20, 2018 at 7:42 AM, Tim Fathers <X700-IX2J@xxxxxxxxxxx> wrote:

"I agree the decimal math isn't optimal in Javascript, but it seems like
such a small thing compared to adopting an entirely other language to do
the business logic "

Hmm, I'm not sure the inability do basic financial calculations is that
small a thing, but it's one of many issues I think JavaScript has, dates is
another example of where it's weak. As I've mentioned, the lack of static
typing (Typescript aside) and it's dynamic nature make it very easy to
write code that's brittle.

"And then there are two separate change management and
deployment processes that need to be coordinated. And maintaining knowledge
in both languages."

Actually, on quite a few the projects I've been involved with recently we
use Git even for our IBM i sources and I'm currently working to try to
create a decent build chain using Jenkins and either BOB (
https://nam03.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.s4isystems.com%2Fbetter-object-builder%2F&data=02%7C01%7C%7Cac10858d2c4644f1cf2408d58e62ddab%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636571477625066578&sdata=2JHVs2nBlzCfqxvaJFQZ8exZz3N5bJM2FPcFu%2F0zSKw%3D&reserved=0) or a derivative of it
to automate builds in a similar way to our front-end. I can't help
wondering why IBM didn't just give us a version of BOB way back when ILE
was introduced...but that's another story...

"If tasked with building an application from the ground up, what would your
language stack look like?"

That's a really good question, and I don't think we're as far as way as
you think. The answer depends on what the starting point was. If I was
looking at an IBM i application, especially if it was an extension or
migration of an existing application I'd do what I've done already, that is
implement the business logic in the language best suited to the platform
and closest to the database (SQL and RPG in my case) because I think that's
where many of the IBM i's strengths lie. It also means that the existing
knowledge people have can be leveraged, there doesn't have to be a mass
retraining and relearning exercise and you don't have the fatal combination
of a team of inexperienced (in the new methods) developers writing a new
application from the ground up using an unfamiliar programming paradigm.
Having worked on similar Smalltalk project back in the day, I know from
experience that this is doomed to failure.

For a web app that wasn't IBM i based but still had to talk to a
relational database then I probably would use Typescript, simply because
most other platforms don't have the integration between database and
programming languages that the IBM i has, so there's not a readily
available backend language to use - and I like Typescript 😊 I'd still
strive to keep as much of the core business logic wrapped up in SQL stored
procedures but I'd try to split the functionality out to where it's done
best, e.g. financial calculations and record processing in SQL, token
decryption etc. in Typesript.

If the web app were to use an object database like MongoDB then of course
there would be no real choice but to implement everything in *script of
some sort, but again, in both the latter cases it makes sense dealing with
the Idiosyncrasies of *script because it's the obvious choice and bringing
something else into your mix would most likely make your life harder not
easier

To be clear, my criticism of backend *script is mostly in the context of
the IBM i, which already has built into it languages that are specifically
designed to efficiently express typical database orientated business
functionality. It's an integrated platform and it doesn't make much sense
to me to wrestle *script into submission to do something that can already
be done much more eloquently with another language. Once you start talking
about not taking advantage of the IBM i's integrated nature then I would
question why you'd use it at all. If you want to develop NodeJs
applications then there are probably plenty of other platforms that are
cheaper and better suited to the task - containerisation using Docker, for
example, springs to mind.


________________________________
From: WEB400 <web400-bounces@xxxxxxxxxxxx> on behalf of Aaron Bartell <
aaronbartell@xxxxxxxxx>
Sent: 20 March 2018 12:40
To: Web Enabling the IBM i (AS/400 and iSeries)
Subject: Re: [WEB400] [EXTERNAL] Re: Rise of Node

I'm not suggesting you can't write typical "business logic" in Javascript,

but I'm questioning the wisdom of it and your workarounds demonstrate why
IMO.

I agree the decimal math isn't optimal in Javascript, but it seems like
such a small thing compared to adopting an entirely other language to do
the business logic (i.e. RPG, Java). While there are tools to span the gap
(iToolkit), it is still a gap that needs to be crossed for each business
logic call. And then there are two separate change management and
deployment processes that need to be coordinated. And maintaining knowledge
in both languages.

Regardless of our different views on possibilities of where business logic
should be, I am curious of the following given that you have a decent
amount of experience in a variety of languages...

If tasked with building an application from the ground up, what would your
language stack look like?

Aaron Bartell
IBM i hosting, starting at $157/month. litmis.com/spaces


On Tue, Mar 20, 2018 at 4:33 AM, Tim Fathers <X700-IX2J@xxxxxxxxxxx>
wrote:

I'm not suggesting you can't write typical "business logic" in

Javascript,

but I'm questioning the wisdom of it and your workarounds demonstrate why
IMO. The solution below is like something from a 1980's Windows program
before preemptive multitasking was around you had to remember to yield to
the OS and I'm sure you don't need me to explain the potential drawbacks.
The fixed point function doesn't round properly, as far as I remember,

the

only safe way to do fixed point maths is to shift the numbers to integers
first, either way, it's not as straightforward as you claim. Even if it
were, having to wrapper all of your basic maths operations in a function
call is error prone and ugly and simply clutters up the program with

noise.

The benefits of using JS or even Typescript to express your business

logic

over a language designed specifically for that purpose would have to be
significant to outweigh the disadvantages and in it's current state, I
don't think they are.

--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing
list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://eur01.safelinks.protection.outlook.com/?url=
https%3A%2F%2Flists.midrange.com%2Fmailman%2Flistinfo%
2Fweb400&data=02%7C01%7C%7C15cca215f9184070397908d58e5768e6%
7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636571428425000686&sdata=Dhf%
2FoRSlCaSn4VTNp7mqWYf4d6hOUrv5XvCBivGuDbk%3D&reserved=0
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://eur01.safelinks.protection.outlook.com/?url=
https%3A%2F%2Farchive.midrange.com%2Fweb400&data=02%7C01%7C%
7C15cca215f9184070397908d58e5768e6%7C84df9e7fe9f640afb435aaaaaaaa
aaaa%7C1%7C0%7C636571428425000686&sdata=YZ7UP3O4dXjs%2BY0PH4v9ZS%
2FNNVuordpSPWC2Hhrwiek%3D&reserved=0.

--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing
list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.midrange.com%2Fmailman%2Flistinfo%2Fweb400&data=02%7C01%7C%7Cac10858d2c4644f1cf2408d58e62ddab%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636571477625066578&sdata=799%2FH3piTHuVTrVrpFFpzj6ku1Y9OBE4Ui2E68o%2FZJU%3D&reserved=0
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Farchive.midrange.com%2Fweb400&data=02%7C01%7C%7Cac10858d2c4644f1cf2408d58e62ddab%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636571477625066578&sdata=Ld8GRc4fVOCn4YMobvudCHrlD1G4E1cHNirFutRX7Ek%3D&reserved=0.

--
This is the Web Enabling the IBM i (AS/400 and iSeries) (WEB400) mailing list
To post a message email: WEB400@xxxxxxxxxxxx
To subscribe, unsubscribe, or change list options,
visit: https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.midrange.com%2Fmailman%2Flistinfo%2Fweb400&data=02%7C01%7C%7Cac10858d2c4644f1cf2408d58e62ddab%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636571477625066578&sdata=799%2FH3piTHuVTrVrpFFpzj6ku1Y9OBE4Ui2E68o%2FZJU%3D&reserved=0
or email: WEB400-request@xxxxxxxxxxxx
Before posting, please take a moment to review the archives
at https://nam03.safelinks.protection.outlook.com/?url=https%3A%2F%2Farchive.midrange.com%2Fweb400&data=02%7C01%7C%7Cac10858d2c4644f1cf2408d58e62ddab%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636571477625066578&sdata=Ld8GRc4fVOCn4YMobvudCHrlD1G4E1cHNirFutRX7Ek%3D&reserved=0.