Breathing Room

 

When I left my last contract, I had an idea for software that was — to use an expression — 'half-baked'.  It was an idea for a problem reporting-responding system for your local friendly tools maven — someone like me.  Someone, perhaps, like you.  I thought there was a need for a vehicle for programmers — the folks who do the real work of producing software that generates the reports and files that management uses on a day-to-day basis to run the companies that produce the wealth that pays our salaries — to log that some particular tool exhibits unusual behavior or to suggest that some tool needs additional function.

The problem is that to develop mainframe software, one really needs a mainframe, and when I say 'mainframe', what I really mean is an 'IBM-ish operating system'.

There aren't too very many of those around, and what few there are are generally devoted to revenue-producing activity, as opposed to blue-sky and pie-in-the-sky ventures... like problem-reporting systems.

It's not easy to find such things — IBM-ish mainframes — but it's not impossible, either, and I managed to find one that didn't require me to pay $50/hr of computing time.

As a result, I've managed to get that problem reporting-responding system pulled together into a form that seems to me ready for some real-world testing.  So...

I have put 'FIXTOOL' out on my alternate source code page, http://frankclarke.dx.am/REXX/listindx.html, and now invite all and sundry to steal a copy and run it through its paces, reporting any errors back to me at my main email address.  You will also need to snag copies of several other routines that are needed in one way or another, and FIXTOOL also requires that SMTPNOTE (an IBM-originated routine) be active on your site.

• Routine DFLTTLIB should be downloaded and adjusted to fit your local environment.  It designates the local default ISPTLIB dataset that will house all the ISPF tables used by my software.  Every user of FIXTOOL must be able to WRITE to its table.
• Routine ICEUSER should be downloaded and customized to identify the 'special' users who are expected to manage the tools. 
• Before running FIXTOOL, you must run TBLMSTR.  TBLMSTR will create an AA-form table (AAMSTR) with a single entry describing itself, and write that table onto the default ISPTLIB. 
• You, the maintainer, must add a row onto AAMSTR for the IT-type table so that TBLGEN will know how to build it when ordered.  The easiest way to do this is to invoke FIXTOOL with the single parameter 'LOADQ'.  This only has to be done ONCE.  Subsequent invocations of FIXTOOL need not (should not) use LOADQ.

So, here is the component list of all the software that must be in place in order to run FIXTOOL: 

• DFLTTLIB
• FCCMDUPD
• FIXTOOL
• ICEUSER
• LA
• TBLGEN
• TBLMSTR
• TRAPOUT

Several of these routines reference SYSUMON, a usage logging routine that uses RXVSAM to update a KSDS where tool-usage statistics are kept.  You can either disable it or build the appropriate KSDS and let it count.  Up to you.

If you decide to fix any problems you find rather than have me fix them, I would still appreciate hearing from you what those problems were and how you addressed them.

 

ISPF Tables — Copying

For any number of reasons, you may decide you need a copy (usually partial) of some ISPF table.  Possibly, the subset of the table you wish to operate on may not be easily parsed on the basis of the table's data content.  You will have to build a temporary table and load it from the original table, perhaps row-by-row.  Careful, now...

TBQUERY will tell you what an existing table looks like.  As soon as the table is opened, TBQUERY can be invoked to give a list of the keys and names.  From this, you can TBCREATE a replica table.

    "TBOPEN SAMPLE"
    "TBQUERY SAMPLE KEYS(knames) NAMES(nnames)"
    "TBCREATE NEWCOPY KEYS"knames "NAMES"nnames 

The column names returned from TBQUERY are enclosed within parentheses.  Unless you have other uses for the lists, it isn't even necessary to trim them off.  At this point, you are ready to write a duplicate or subset table, save for one easily overlooked aspect.

Remember 'extension variables'?  Unless you ask when reading each input row you will not know whether there are extension variables attached to the row, nor will you know their names.  Not knowing their names means not being able to specify which names should be attached to each output row.

   do forever
      "TBSKIP" intbl "SAVENAME(xvars)"
      if rc <> 0 then leave
      "TBMOD " outtbl "SAVE"xvars
   end

Notice that the extension variables themselves did not have to be operated on in any way.  We did not have to assign them, for instance.  Because the row was read (the NOREAD parameter was not specified), all the row's variables: keys, names, and extension; were populated to the variable pool.  They are therefore immediately available to be loaded to the output table row.

In fact, because of this behavior: populating every variable regardless of type; it might be good practice to never read a table row without acquiring the list of extension variables.

You can get fancier than this, of course.  For an idea of just how much fancier you can get, take a look at TBCOPY over on my REXX page.  TBCOPY doesn't use TBQUERY.  It relies on an external subroutine, TBLGEN, which itself relies on a special table in which is stored the DNA of all permanent tables, sort of a data dictionary for tables.  TBLGEN is much more elaborate than is strictly necessary, because it is intended to do all of an installation's TBCREATEs for permanent tables.  TBCOPY is likewise much more elaborate than is strictly necessary because it is intended to do all TBCOPYs for permanent tables within an installation.

ISPF Tables — Extension Variables

Sometimes we just have to deal with tables we didn't design (and would have designed differently had we had the opportunity).  Sometimes we just need a tiny little tweak to make that table perfect (reg., U.S. Patent Pending).  Sometimes we want to let everyone know that this table row is unusual either for what it has or for what it hasn't.  Enter the extension variable.

Every table row in a given table has room for every KEY and NAME field originally specified when the table was TBCREATEd.  Flexible little devils that they are, table rows also have unlimited^* space for other stuff, and every table row can be unique as to what other stuff it holds.  Observe:

    "TBCREATE BILOMAT KEYS(BMITEM) NAMES(BMDESCR BMONHAND BMECOQTY)"

our Bill-of-Materials table.  It is keyed by the item number (BMITEM), and has fields for a description, the number on hand in inventory, and the economic order quantity.  Wha...?  Where are the 'materials' that make up the 'bill'?  They're in extension variables, naturally.

    "TBVCLEAR BILOMAT"
    bmitem   = 'ER4590T'
    bmdescr  = 'Whatsis, large, left-handed, tan'
    bmonhand = qty.whatsis_l_lh_tan
    bmecoqty = ecocalc.whatsis_l_lh
    xvars    = 'DD114 DF33 DF34 DF36 DF38 EM2030'
    dd114    = 6
    df33     = 1
    df34     = 1 
    df36     = 1
    df38     = 2 
    em2030   = 4
    "TBADD BILOMAT SAVE("xvars")"

The row for 'ER4590T' in table BILOMAT has all four canonical fields filled, and there are six additional extension variables, one each for the six components of every ER4590T.  When the supplier for the EM2030 component advises you there will be an unavoidable 40% increase in the price of that component, you're going to want to know which of your products (BMITEMs) is going to be affected and by how much.  The catalog entries for each of those products will have to be adjusted to correctly show the new price of everything that relies on the EM2030, right?  Of course!

    "TBVCLEAR BILOMAT"
    em2030  = 0
    "TBSCAN BILOMAT ARGLIST(EM2030) CONDLIST(GT)"
    "TBDISPL BILOMAT PANEL(SRCHPART) AUTOSEL(NO)"

TBVCLEAR zaps all the 'regular' names mentioned in the definition of table BILOMAT, variable 'EM2030' is set to zero, TBSCAN specifies that we seek all rows where an extension variable 'EM2030' has a value greater than zero.  Panel SRCHPART should, if coded correctly, display a scrollable list of all rows (all BMITEM keys) associated with subassembly EM2030.

Zounds!  This is almost DB2!  Except that it didn't take us eight months and require the approval of two vice-presidents.  And if they decide later that they really do want to implement it in DB2, you already have the proof-of-concept ready to demonstrate.

 

(*) — You didn't think when I said 'unlimited' I actually meant 'without limit', did you?  Tsk.  Of course there are limits.  It's just that you're unlikely to hit them before your table is so large your system chokes trying to open it.  I've seen that happen with complex tables as small as 6,000 rows, but I have also seen 50,000-row tables that worked just fine, and I suspect they could have grown much larger without causing a problem.

ISPF Tables — Searching

There's no telling how many programmers have simply given up using ISPF tables because 'the @#$% table services never work the way you expect them to work!'  They actually work just fine.  It's the @#$% IBM manuals that don't make their operation clear — don't make the operation of some very complex services as clear as they ought to be.  Whether this is simply a matter of writing style or whether IBM does so deliberately, the delicate interrelationship between TBVCLEAR, TBSARG, TBSCAN, and TBDISPL rarely gets to a programmer's conscious level.  The lack of examples showing TBVCLEAR, TBSARG, and TBDISPL working in concert also doesn't help.

The most important ISPF table service related to table searching is neither TBSARG nor TBSCAN;  it is TBVCLEAR.  The IBM manuals (almost) completely gloss over the fact that when searching an ISPF table, every element that has a value, whether it is part of the explicitly-named search argument or not, participates in setting the search argument.  Every element that has a value.  The service that makes sure elements do not have values is TBVCLEAR.

If you carefully read the manual text for TBSARG, there are subtle clues to this behavior as it relates to setting search/scan arguments:

A value of null for one of the dialog variables means that the corresponding table variable is not to be examined during the search.

and

To set up a search argument, set table variables in the function pool to nulls by using TBVCLEAR.

and

The search arguments are specified in dialog variables that correspond to columns in the table, and this determines the columns that take place (sic) in the search.

You could be forgiven for thinking that specifying this variable or that one in a TBSARG might have some bearing on which variables get used for setting the search argument.  Alas, it has less 'bearing' than might seem obvious.

Let us assume that we have a table, TBL, and that this table is defined as

    "TBCREATE  TBL  KEYS(KEY1 KEY2) NAMES(NAME1 NAME2 NAME3) WRITE REPLACE"

We have read a particular row, either by TBGET or TBSKIP, and now we wish to find all the rows in TBL with the same NAME2 value.  These will be displayed via TBDISPL for further processing.

    "TBSARG TBL NAMECOND(NAME2,EQ)"

is insufficient.  It is insufficient because KEY1, KEY2, NAME1, and NAME3 all have values and are not null.  In fact, a subsequent TBSCAN or TBDISPL will return a single row, the one that was just read.  That's the only row on the table with a matching KEY1 and KEY2.  In order to expand the search to the remainder of the table, we must zap all those variables:

   nval = name2     /* preserve the value */
   "TBVCLEAR TBL"
   name2 = nval     /* load NAME2 */
   "TBSARG TBL NAMECOND(NAME2,EQ)"

In fact, since "EQ" is the default and all table variables that are not null participate in the search (and NAME2 is the only non-null variable at this point), it should be sufficient in this case to cast the TBSARG as

    "TBSARG TBL"

Since we're working in REXX, some may be tempted to DROP the elements that are not to be searched-for.  A REXX 'drop' is not the same thing as setting a table variable to null via TBVCLEAR and the results may be disappointing.  Likewise, setting the variables to null via 'closed quotes'

    parse value "" with key1 key2 name1 name3 .

also won't get you what you need.  This will cause the search to be limited to those rows for which KEY1 and KEY2 are empty.  There can only be one of those and there may not be even that.

Before TBSARG, before TBSCAN (if that's how you're setting the search argument), TBVCLEAR.  Voila!

ISPF Tables — Closing

How do you like that?  I wrote this and never published it.  This was supposed to be the blog entry for 11/22, and it seems to have been forgotten.  Ah, well...  better late than never...

When you've finished doing whatever needed to be done with the table, it must be TBCLOSEd.  As with TBOPEN, a LIBDEF is needed, but not to ISPTLIB.  ISPTLIB is the input side of table processing.  ISPTABL is the output side of table processing.  Since data will be written to ISPTABL, ISPTABL cannot be a concatenation.  Only one dataset may be allocated to ISPTABL:

   if noupdt = 1 | sw.0tbl_chgd = 0 then "TBEND  " $tn$ /* don't save */
   else do
        "LIBDEF  ISPTABL  DATASET  ID("isptabl")  STACK"
        "TBCLOSE" $tn$
        "LIBDEF  ISPTABL"
        end 

If there have been no changes to the table, it is not necessary to TBCLOSE (which writes the table to disk).  A simple TBEND is sufficient.  Both TBEND and TBCLOSE flush the in-storage copy of the table, but TBEND doesn't write to disk first.  Since it doesn't write to disk, the LIBDEF for ISPTABL is not necessary for TBEND.

As with other LIBDEFs, this one only has to exist long enough to close the table.  After that, it can be (and should be) released immediately.

ISPF Tables — Defining and Building

Building an ISPF table is done by the TBCREATE service.  The table may or may not have a key or keys.  The table may or may not have one or more name (data) fields.  It will almost certainly have at least one of either, but 'almost' is not 'absolutely'.  In fact, an ISPF table may be TBCREATED with no specifically-named fields at all...  but you had better know what in Hell you are doing in that case.  Very likely, before doing so, you would very definitely know what you had in mind.  The uses for an 'extension-variables-only' table are breathtakingly esoteric.

    "TBCREATE" table_name "KEYS(" key_field_names ")",
                         "NAMES(" data_field_names ")" other_table_qualities
    "TBCREATE TEMP  [ KEYS() NAMES() ] WRITE REPLACE"       

To populate a new table row, each of the named fields, whether KEYS or NAMES, is assigned a value (or not — more later) and the TBADD service is called.  The row is created with all KEYS and NAMES fields, whether populated or not, plus all extension variables referenced by the SAVE parameter.  Extension variables are potentially unique to each row, and are only loaded to the table if specifically requested in the TBADD/TBMOD service call.

    "TBCREATE ZORK KEYS(INDEX OFFSET) NAMES(PLACE TYPE VALUE)"
    index   = index + 1
    /* OFFSET not populated */
    place   = left
    type    = normal
    value   = total
    oper    = userid()
    time    = Date("S") Time("N")
    "TBADD ZORK SAVE(OPER TIME)"

This added one row to table ZORK.  All fields except OFFSET have values, and there are two extension variables, OPER and TIME, which are not part of the canonical definition.  There is a field for OFFSET, but it is null, and here null has a different meaning than it does in REXX.  Here, it means: doesn't exist.

That's it.  That's all there is to populating a table.  Updates to the table thereafter can be done via TBADD (for new rows) or TBMOD (for replacement rows — or new rows if the key does not presently exist).

ISPF Tables — Opening

When using an ISPF table for input, the table library must be available via ISPTLIB.  The library containing the table to be used has to be either part of the original ISPTLIB allocation or LIBDEF'd into a preferential position.  If LIBDEF'd, the LIBDEF can reference either a DATASET or a LIBRARY.  If a LIBRARY, the library itself must have been previously ALLOCATEd under a unique DDname.  This latter method is fraught with hazard and I do not recommend it.  I recommend instead using the DATASET route as being much less likely to give you an ulcer:

   "LIBDEF  ISPTLIB  DATASET  ID("isptlib")  STACK"
   --- use the dataset ---
   "LIBDEF  ISPTLIB"

Once the dataset is in place, you must open it with TBOPEN...  unless it's already open...  unless it's not even present.  Well, heck, how do we know that?  TBSTATS:

   "LIBDEF  ISPTLIB  DATASET  ID("isptlib")  STACK"
   "TBSTATS" $tn$ "STATUS1(s1) STATUS2(s2)"
   if s1 > 1 then do
      say "Table" $tn$ "not available."
      zerrsm = "Table" $tn$ "not available."
      zerrlm = "Table" $tn$ "not found in the ISPTLIB library chain"
      sw.0error_found = "1"
      end; else,
   if s2 = 1 then do                   /* table is not open          */
      "TBOPEN "   $tn$   openmode.NOUPDT
      if rc > 4 then do
         sw.0error_found = 1
         zerrsm = "Table did not OPEN"
         zerrlm = "Table" $tn$ "cannot be opened due to prior",
                  "enqueues."
         "SETMSG  MSG(ISRZ002)"
         end
      end
   else "TBTOP" $tn$
   "LIBDEF  ISPTLIB"

STATUS1 generally addresses the data-content of the library.  STATUS2 references the OPEN-state of the named table, in this case, variable "$tn$".  A value of "1" in STATUS1 indicates that the named table exists in ISPTLIB.  A value greater than one signals trouble:  the table you've named isn't where it can be opened.  This certainly constitutes 'an error'.

A value of "1" in STATUS2 signals that the named table is not presently open or otherwise in use.  If this is not the case (the table is already open - STATUS2 returned a value greater than "1") there is no need to re-open the table.  ISPF will ignore such a request.  With the table presently in a not-open state, a TBOPEN can be issued for the table.  The example here shows that the mode of opening is determined by the current setting of variable "NOUPDT" (set in REXXSKEL's TOOLKIT_INIT section).  The table will be opened for WRITE if NOUPDT is off, and NOWRITE if it is on.

A table which is already open need only be TBTOPped to make it ready for our use.

As soon as the table is determined to be open, the LIBDEF for the ISPTLIB is no longer needed — a full copy of the table now exists within the user's region.  It's good practice to immediately drop the LIBDEF the instant it has served its purpose.

That's it.  That's all there is to opening an ISPF table.  From here on in until the table is TBCLOSEd or TBENDed, the complete contents are available for your use.  Unfortunately, that makes them unavailable for anyone else's use if the table was opened WRITE, so be a good user: get in, get done, get out.

If you as the programmer discover that some of your users are hogging the table, you may have to take drastic action to prevent that.  'Drastic action' may include TBCLOSEing the table after every update, TBENDing it after every inquiry, and consequently repeatedly TBOPENing it for each new cycle.  In between, you will have to maintain a record of what the original looked like when it was first fetched so that you'll know whether it was changed in the meantime.  Lots of CPU cycles wasted because of undisciplined users.  You may even wind up rewriting the app to eliminate ISPF...  but you will have had the experience of prototyping it (easily) in ISPF and will get the benefits Dr. Brooks promised when he said:

When planning a program, plan to do it twice.  You're going to do it twice;  you might as well plan for it.

ISPF Tables

Most of our uses of ISPF tables are very simple.  Every once in a while, however, we might bump into a relatively-more-complex situation where the uses are not so simple.  Without fail, such times are opportunities for us to learn some very difficult and painful lessons.  It's much better to learn them on a blog than while sitting before a terminal pumping code in a vain attempt to make a deadline.  The most difficult lesson is this: reading a table row populates any REXX variables that have matching names (and creates them otherwise).  This is most dangerous when you have to work with more than one table simultaneously.  Let me illustrate:

   address ISPEXEC 
   "TBSKIP ORDERS" /* sets vendor, product, date, qty, invoice */
   "TBGET  INVENT" /* keyed by 'product', sets qty, descrip */ 

The value for 'qty' from 'ORDERS' has been overlaid by 'qty' from 'INVENT'.  To preserve the original 'qty', it has to be copied to a different variable name.  Code-writing time is the wrong place to have to suddenly rename variables wholesale.  That is why I recommend that every table variable name should have a two-character prefix unique to the particular table:

   address ISPEXEC 
   "TBSKIP ORDERS" /* sets orvendr, orprod, ordate, orqty, orinvc */
   "TBVCLEAR INVENT"
   inprod = orprod
   "TBGET  INVENT" /* keyed by 'inprod', sets inqty, indesc */ 

There is now 'orqty' and 'inqty' and they can be compared, perhaps to see whether it's time to reorder.

Problem solved?  Hardly.  ISPF tables have more tricks up their sleeve(s).  Behold... extension variables.

Extension variables are unique to each row, and only the programmer exercising care in their naming can prevent clashes such as above.  As with ordinary table variables, extension variables also populate REXX variables when the row is read, and there is no warning that a simple TBSKIP may have clobbered other variables... unless the programmer is smart enough to ask if any extension variables were transferred with the rest of the row:

   "TBSKIP ORDERS SAVENAME(xvars)"
   parse var xvars  "("  xvars  ")"

which lets us know that 'ornotes', 'ormgrnam' and 'ormgrtel' were set without warning.  If we were to update that row without also commanding that those three extension variables be included in the update, they would simply disappear, also without warning.  It's a quick way to shoot holes in table data.  That's not us, thankfully.  We're smart enough to:

   "TBMOD   ORDERS  SAVE(" xvars ")"

thus preserving the data in those extension variables.

As with everything else, a few small proof-of-concept routines can be an invaluable learning aid, and (of course) you need to test the logic to death.