Seize deadlock: threads, fair seizes, file extends; PRTLCKRPT; IBM confidential

IBM is researching whether our issue is fixable at our current release
V5R5M5. I will share a portion of IBM's diagnosis.

<IBM quote>
"- The issue is a seize deadlock:
- Job A tries to extend file File and waits behind readers and "fair
seize" kicks in
- Job B is reading file File, has low seize. It fires a UDF
- Job B's secondary thread is running and needs to seize file File, due
to "fair seize", it waits.
Job B is generally cancellable, there's a "wait time out" message that
wakes it up looking for cancellation.
. . . [He] can't go into more detail since it is IBM confidential. It
is all operating system level stuff anyway.
This isn't related to the commitment control levels, this is all within
the OS."
</IBM quote>

ISSUE:
A QZDASOINIT job running Java ran this SQL SELECT statement but did not
complete.

select count(*) as countDiff from CRMSAT.F55QUOTE left join
AMSDTA.F4211LA on AMSDTA.F4211LA.SDKCOO=QUCMPY and
AMSDTA.F4211LA.SDDCTO=QUTYPE and AMSDTA.F4211LA.SDDOCO=QUID and
AMSDTA.F4211LA.SDNXTR <> ? left join CRMSAT.F55QURV on JOBID=QUJBID and
REVISN=(select max(REVISN) from crmsat.f55qurv where JOBID=QUJBID and
CRMSAT.F55QURV.SDKCOO=AMSDTA.F4211LA.SDKCOO and
CRMSAT.F55QURV.SDDCTO=AMSDTA.F4211LA.SDDCTO and
CRMSAT.F55QURV.SDDOCO=AMSDTA.F4211LA.SDDOCO and
CRMSAT.F55QURV.SDLNID=AMSDTA.F4211LA.SDLNID) and
CRMSAT.F55QURV.SDKCOO=AMSDTA.F4211LA.SDKCOO and
CRMSAT.F55QURV.SDDCTO=AMSDTA.F4211LA.SDDCTO and
CRMSAT.F55QURV.SDDOCO=AMSDTA.F4211LA.SDDOCO and
CRMSAT.F55QURV.SDLNID=AMSDTA.F4211LA.SDLNID where QUJBID=? and QUID=?
and QUTYPE=? and
(COALESCE(AMSDTA.F4211LA.SDKCOO,?)<>COALESCE(CRMSAT.F55QURV.SDKCOO,?) or
COALESCE(AMSDTA.F4211LA.SDDCTO,?)<>COALESCE(CRMSAT.F55QURV.SDDCTO,?) or
COALESCE(AMSDTA.F4211LA.SDDOCO,?)<>COALESCE(CRMSAT.F55QURV.SDDOCO,?) or
COALESCE(AMSDTA.F4211LA.SDLNID,?)<>COALESCE(CRMSAT.F55QURV.SDLNID,?) or
COALESCE(AMSDTA.F4211LA.SDSRP1,?)<>COALESCE(CRMSAT.F55QURV.SDSRP1,?) or
COALESCE(AMSDTA.F4211LA.SDKTLN,?)<>COALESCE(CRMSAT.F55QURV.SDKTLN,?) or
COALESCE(AMSDTA.F4211LA.SDUORG,?)<>COALESCE(CRMSAT.F55QURV.SDUORG,?) or
COALESCE(AMSDTA.F4211LA.SDLITM,?)<>COALESCE(CRMSAT.F55QURV.SDLITM,?) or
COALESCE(AMSDTA.F4211LA.SDDSC1,?)<>COALESCE(CRMSAT.F55QURV.SDDSC1,?) or
COALESCE(AMSDTA.F4211LA.SDDSC2,?)<>COALESCE(CRMSAT.F55QURV.SDDSC2,?) or
COALESCE(AMSDTA.F4211LA.SDUPRC,?)<>COALESCE(CRMSAT.F55QURV.SDUPRC,?) or
COALESCE(AMSDTA.F4211LA.SDAEXP,?)<>COALESCE(CRMSAT.F55QURV.SDAEXP,?) or
COALESCE(AMSDTA.F4211LA.SDAN8,?)<>COALESCE(CRMSAT.F55QURV.SDAN8,?) or
COALESCE(AMSDTA.F4211LA.SDITWT,?)<>COALESCE(CRMSAT.F55QURV.SDITWT,?) )

Other QZDASOINIT jobs that attempt to insert records into F55QURV needed
to extend F55QURV because it was full. File extend was not possible
because the first job would not complete its task. E.g.
B00FE00008498000 IS WAITING FOR A EXCLUSIVE SEIZE

I tried to ENDJOB the first QZDASOINIT job repeatedly and it finally
ended after 2 hours. The hundreds of QZDASOINIT jobs attempting to
insert completed immediately thereafter because F55QURV extended in
size.
I am very glad that we did not have to power off and IPL the system in
order to correct a deadlock condition.
We occasionally encounter mysterious slowness on our system and I
typically correct the matter with these commands:
ENDHOSTSVR SERVER(*DATABASE) ENDACTCNN(*DATABASE)
ENDPJ SBS(QUSRWRK) PGM(QZDASOINIT) OPTION(*IMMED)
I have just recently decided to see whether seizes and locks are a
frequent problem on our machine:
STRPFRTRC SIZE(*CALC)
ENDPFRTRC MBR(JUNE15)
PRTLCKRPT MBR(JUNE15) RPTTYPE(*ALL)