Thursday, September 24, 2015

Screen Data Loading in Progress

Any Progress Programmer who has worked with the QAD ERP package is familiar with the "CIM format" of loading screens from specially formatted data files. I placed the phrase in quotes because although QAD can lay claim to the format they call CIM (originally an abbreviation for Computer Integrated Manufacturing) by enhancing this format to include deletion and enabling a batch mode system based on it, the format itself is native to standard Progress ABL functionality.

Many Progress Programmers who do not have familiarity with QAD do not know about this format for data loading. It's not like this format is necessary to load data into the database. It is sort of a strange format, and using it with a standard screen is a lot less straightforward than writing

create credit.
import delimiter "," account_code effect_date cr_amount.

which could load a table from a CSV file in short order. But the nice thing about using CIM format, or as I like to call it, Screen Data Loading format, is that
  1. you can very closely mimic the actions of someone entering or updating data via a Progress input screen,
  2. that way all input screen code can be reused and also 
  3. all referential integrity is maintained and
  4. the formatting is very similar to standard, delimited formats and just as readable.

To illustrate what I mean and how CIM format works, imagine you have a Progress program named mainmenu.p which is the main screen for an accounting system. This program brings up a credit input screen when you choose the code "CRINP". The input screen contains three fields and runs in a repeating block until END-ERROR is encountered (F4 in Unix, ESC in Windows). At this point a field labeled "Commit?" comes up and if the user types a "Y" then the program commits all the entered data into the credit database table and returns to the calling program.



Here's what a program would look like to use the Screen Data Loading format available in Progress for that screen:

/* Screen Data Loading (SDL) sample code */
input from sdl-credit.txt.
output to sdl-credit.log append.
run mainmenu.p.
output close.
input close.

The contents of a typical sdl-credit.txt would look like the following.

"CRINP"
"2500" 05/25/14 1433.70
"5004" 05/26/14 500.00
- 05/24/14 100.00
"2500" 05/29/14 50.34
.
"Y"
.

Here's what happens upon executing of the Screen Data Loading code:
  • CRINP would trigger the credit entry screen.
  • In the next 4 lines, the first field would go into account code, the second into date and the third into amount.
  • The single unquoted hyphen in the third input line cause the first field to be skipped over. This is especially useful to bypass unnecessary fields or to accept defaults in an input program.
  • The single period on a line designated and END-ERROR condition which brings up the "Commit?" question.
  • The "Y" commits the data and control is returned to mainmenu.p at which point the second period causes that program to exit.
  • All output is appended to the specified output log file. This file can get rather long since each input field causes the screen to be rewritten in its entirety, but it is good for trapping errors.
As I explained earlier, QAD deserves a lot of credit for utilizing this format in their own batch loading system, popularizing it, adding a method for deletion functionality and extensive error trapping and reporting. But I was gratified to learn that this method works on any system designed using Progress ABL. I used it for one client who owned a home-grown system without user-friendly input screens. Using the Screen Data Loading method I've outlined here, I was able to instruct users to fill out spreadsheets which then I transformed into the proper format and loaded into the system.

Once a programmer becomes familiar with the format, the part of the development which requires the most intense work is what I call the pre-validation of the data. For ongoing processes, I always make sure each of the fields being loaded will not flag an error ahead of time. For one-time data loads, I usually just "let 'er rip" and check out the error file after the fact. Of course one always needs to watch out for data values going into the wrong fields. But this danger exists in coding for more standard formats, so it is not a pitfall unique to this particular format.

Contact me for all your Progress/QAD Development requirements.

Email: paul.fry@flexresourcing.com
Phone: 216-496-9915