?

Log in

Previous Entry | Next Entry

Redevelopment

By the end of the 1970s the Customs computing facilities were under great strain. Not only had cargo and passenger traffic continued to grow but computing requirements had roughly doubled every three years. At the same time, maintaining and improving applications like SEARCH and INSPECT was a daunting task, especially when parts of the source code were missing. The old network with its 16 GA minis (each with 48K of RAM and two 1Mb disk packs) was also feeling its age. The old ICL 4/72 mainframes were sold to a scrap metal dealer who melted them down for their gold content. They were replaced by a pair of ICL 2960s, which emulated the 4/72s. The lack of support for this setup led Customs to consider the possibilities of a new machine.

Having reached a technological dead end, the decision was taken to redevelop on a new architecture. Tenders were put out for a new network and a new mainframe. The network contract was awarded to PRIME for an X.25 network. The mainframe contract was awarded to Univac (now UNiSYS) which proposed to use a Univac 1184. The one mainframe was intended to run all of Customs new applications plus separate development and test environments as well. The new applications would be written in a high level language - COBOL.

Network

As it turned out, the network was the first thing to be redeveloped. Prime found it too hard to replace the old network so it was assembled together in the computer centre and connected to the new. The new network was driven by micros called TPs (Telenet Processors) and controlled by a Prime minicomputer. A second minicomputer ran a new application called SESS - Session Establishment and Security System - which handled user logon and off. Connecting the new network to the Univac machine was more difficult. Univac had to put several man years of effort into getting their mainframe it to talk to the X.25 network and the old terminals.

PASS II

The first of the new systems was a rewrite of PASS, but PASS II was completely different system in every way. The proposal called for, in fact, the most elaborate passenger processing system in the world. It would be a twenty-four hour system with powerful facilities for inputting and processing passenger alerts. Elaborate retrieval subsystems were provided and there were numerous luxuries. The new system had excellent facilities for generating beautiful and complicated screens and these were used with abandon. The database software contained many superb facilities, especially for crash recovery. Users would include not only Customs and Immigration, but the Federal Police, ASIO, the National Crimes Authority, and even the Tax Office and Department of Social Security. With its links to PICS (at Foreign Affairs) and TRIPS (at Immigration), PASS II has been described as the biggest computer threat to privacy in the Australia.

PASS II was implemented in late 1984. To say that implementation did not go smooth would be an understatement. Few parts of the system worked well at first. Most of the problems were network related. Connections were constantly lost. The elaborate screens sometimes took minutes to refresh. It could take up to ten minutes to log on through SESS. SESS fell over frequently and when it came back up, it would forget today's password changes or it would stop talking to Univac. There were also hardware problems with the new network.

If you were lucky enough to get on to PASS II, things were not much better. For one thing, PASS II was running on the same machine as the development programmers and they frequently slowed or even crashed it. For another, PASS II had been rushed into production and still had plenty of errors left in it. At the heart of the problem was inexperience with the new environment and techniques. The problem was made more acute because of a tight schedule. The elaborate testing plan had been discarded to save time. The result was transaction aborts, producing millions of pages of octal dumps which the PASS II programmers could not read. The systems programmers rapidly became buried under mountains of octal dumps which were wheeled in on trolleys.

Worse still, the PASS II database usually deadlocked and had to be redesigned. This occurred because the database analysts and designners had, through inexperience, failed to understand the significance of the fact that hundreds of air passengers tend arrive on the same plane at exactly the same time.

Worst of all, PASS II strained the resources of the machine to the limit. The tender had stated that PASS transactions would require 50,000 machine instructions, the same as PASS I. PASS II couldn't even refresh a screen with that - 5,000,000 would be more like it. There was just no way that the machine could support two systems if one of them was PASS II.

Customs had seen bumpy implementations before, but that was in the days when the alternative was doing it by hand and when it still could be done by hand. To the users at the airports faced with long queues of tired, impatient travellers this was just too much.

PASS II was taken out in Sydney before Christmas and was not reintroduced until the end of March 1985. Elsewhere, many other users went back to PASS I, which remained the system of choice until the plug was pulled on it in July 1985. The system still ran poorly until the machine was upgraded at the end of 1986.

Eventually PASS II was acknowledged as a success, but it was a user relations disaster. User confidence in and goodwill towards the computer systems was lost forever. For years afterwards, PASS remained very sensitive industrially and even the smallest problems could rouse senior Customs and UNiSYS management out of their beds at 3:00am.

Trace

The proposal report for TRACE - Total Retrieval and Analysis of Customs Entries (the abbreviations having by now become a tradition) - foresaw the most impressive system yet. More than just a replacement for SEARCH, it would be menu driven and simple enough for the most inexpert user to use, but powerful enough to search and sort a 40 gigabyte Customs Commercial database on many different keys and produce elaborate executive reports on local printers. The proposal was packed full of fancy features, such as the ability to compile an extraction for later use and to run off statistical analyses of the data.

Initially it was intended that TRACE would precede PASS in 1983, but it soon slipped into 1984, then 1985 and it continued slipping. For a while it was slipping by two or three months every month, prompting some people to wonder if it might not be better if they stopped work because then it would only slip by one month each month! Clearly, TRACE was going to sink without one if something was not done, and like Phineas Phogg in "Around the World in 80 Days", they started to burn the ship in order to make their deadline.

As delivered in early 1986, TRACE was the planned TRACE stage I minus most of the non essential features (and not a few of the essential ones).

At a cost of over $3 million, TRACE was possibly the most expensive "in house" extraction system written up to that time. However, in 1986-87 alone, Customs officers using TRACE uncovered $53 million in unpaid Customs and Excise duty. Somehow the authors had succeeded in creating a CPU bound extraction system - one which displayed a disturbing tendency to chew up all the CPU the machine could muster. A new machine, a Univac 1192 - the most powerful Univac machine in Australia at the time - was purchased for TRACE but it was clear that if it would have to be throttled if was to ever live with other systems.

Compile II

COMPILE II was intended to take over all the data entry functions of both COMPILE and INSPECT. With the problems with PASS II and TRACE in the back of their minds, the designers aimed for the smallest possible system that they could deliver. Of all the new systems, COMPILE II provided the least improvement over the existing system. It was deliberately made to look like COMPILE I to require no retraining for the agents. They found the new system easier to use. They were pleasantly surprised when they discovered that it did not lose their work in progress overnight and that it produced prints immediately.

COMPILE II's major problems were its successes rather than its failures. Huge entries (up to 999 lines) could now be generated with attendant performance problems. One new feature was the ability to generate refunds. Some agents were quick to realise the potential of this. Take one of the dusty boxes of old entries from the basement and key in the entry numbers. COMPILE II would give you an instant opinion on whether you could get refunds on the entries. A good box could yield several hundred dollars worth. A few carpetbaggers even went around buying up old entries and processing them. Refunds soon accounted for 10% of all entries. This was not the sort of computerisation that Customs had had in mind!

Clear

Taking over the Customs House processing side was CLEAR - Clearance of Lodged Entries using ADP Resources (although some say that it actually stands for Constantly Losing Entries At Random). This was originally envisioned as a major attempt at automation of Customs Houses. As such, the proposal ran into industrial relations difficulties, being regarded as an attempt to standardise work practices (which, in fact, it was). This led to CLEAR being drastically cut down. Even so, there was trouble. CLEAR implemented a system whereby entries were assessed for risk through a complex process of profile matching. Entries rated as "green" could be paid immediately. Those rated as "red" had to be checked out by hand. The aim was for all green entries to be processed that morning or afternoon. In Sydney and Melbourne, this was regarded as the best idea yet. Elsewhere, it was seen as threatening jobs. The Adelaide Customs House walked off the job. In the end though, the red/green system was implemented.

Reference Files, Utils and FPS

The old commercial systems required files of reference data such as quota, owner, supplier, tariff and other details. Usually these were maintained as card decks. In the new systems, these formed a separate package of some twenty mini applications called References Files. Reference Files wound up costing more ($2.7 million) than COMPILE II ($2.5 million) or CLEAR ($2.4 million).

Users also found another surprise packaged with the new systems - the Utils system, a cluster of even more unrelated features. Foremost amongst them was FPS, an entire database system to support the enormous volume of print generated by the systems, to the tune of 600,000 pages a day. Agents found that the Utils Print Control facility allowed them to look at their print queues and manipulate the prints, a feature they now claim that they can't do without. Other functions included the ability to send messages to other terminals and Broadcast, an unpopular third party product which replaced the News facility. High security users could access the Network database, a copy of the SESS database on the host. When it was decided to scrap SESS in 1987, the Network database was upgraded to become a full replacement.

That COMPILE II and CLEAR were implemented on schedule in July 1986 was something of a minor miracle which was the result of a great deal of work by Customs and UNiSYS staff, often at absurd hours of the morning.

Personal Computing

With so much going the redevelopment effort, an application backlog developed because there was no longer the time nor the personnel to implement small systems on the mainframe. These needs, which continued to multiply like rabbits, were taken care of through the introduction of personal computers which were often bought in bulk and below cost. Many, many small applications were developed by the sections which used them, and some large ones too, such as SIERRA, which replaced IRIS. For users with bigger needs, a UNiSYS package called MAPPER which could be used for spreadsheet and database type applications was. A MAPPER application replaced SCRIBE, for example.

Going Blue

When the redevelopment effort was at its height in 1985, when PASS II was not working and the programmers were working at night because PASS II was using the machine exclusively during the day, the idea of dumping UNiSYS took on some appeal. Some people argued that having a sole supplier made Customs vulnerable. Accordingly, it was decided to buy an IBM machine, an old 3084 - roughly equivalent to the UNiSYS 1184 in processing power, but huge in physical size and nicknamed "Thomas the Tank Engine". An old MAPPER application, the Diesel Fuel Rebate system known as REEF - Rebate of Excise on Eligible Fuel - was redeveloped on IBM in a fourth generation language called Natural. It was felt that 4GLs were the way to go, but several were tried and none found satisfactory. The move towards IBM, both at Customs and elsewhere, has provoked a backlash however. As the memory of the old ICL systems has faded away, user loyalty has been transferred to the new systems and there has been a great deal of opposition. Spirited debate over the merits of an all blue public service was carried out through the letters page of the Canberra times and the Customs acquisition project was scrapped before it even began - a new record.

The network, however, was redeveloped a second time, this time into an SNA (IBM's Standard Network Architecture) network. This enabled UNiSYS to remove the non standard code from the proprietary software and reliability improved, although SNA was stretched by the task of the wide area network and driving it became the main task of the tank engine. One superb feat of user relations was persuading the customs agents to upgrade all their terminals to IBM standard terminals (often by adding an SNA board to their PCs) for no real benefit to themselves at all, without mass protest.

Customs no longer had the leadership that it once held in information technology, but this was in some ways a blessing; for the first time, a number of complete systems were bought "off the shelf" from other departments: Cobra for asset control; Registry for the files; and Nomad2 for personnel management.

Customs continued to attempt to forge links with other departments. A second network, TRADEGATE, was established to link Customs up with the rest of the export community. Electronic funds transfer was implemented so that agents no longer had to show up at the Customs House to pay for their entries. Indeed, by 1990, there was little reason to show up there at all, because their computer terminals were like having their own Customs officers on site. Efforts were also made to forge links with the airlines and shipping companies with the ultimate aim of bringing the entire import-export industry together electronically. Similarly, on the passenger processing front, strenuous efforts were made to strengthen the links with Foreign Affairs, Immigration and New Zealand Customs. These other organisations remained backward compared with Customs, however.

Coastwatch

Perhaps nothing illustrates the latest trends as Coastwatch, the coastal surveillance program which was transferred to Customs in 1988. Planes fly up and down the coasts of Northern Australia reporting anything unusual. These sightings were then telexed to Canberra. The Federal Police had tried to computerise the process to no avail by having a DPO key certain details from the telexes into a simple, structured database. Customs computerised it successfully with a device which could read telexes and writes them away where they can be accessed by a commercial text search program. An elegant "off the shelf" solution.

Conclusions

In terms of the strategic use of IT, the role played by economies of scale is clear. As far as the import/export industry goes, if Customs hadn't done it, it probably would not have been done. Only Customs had the resources to attempt such a daunting task; the customs agents had neither the organisation nor the money.

Customs maximised its strategic benefits by computerising its major functions. The strong leadership of the organisation must be praised here. Other departments would have started with SCRIBE instead of INSPECT and might never reached COMPILE. Later, these sorts of minor systems like Cobra, Registry and Nomad were purchased at low cost from such departments.

With no real "markets" or "demand", the Customs effort was technology driven. I hope that I have demonstrated how one idea led to the next. Certain breakthrough ideas seem obvious in retrospect only because we are so familiar with the technology. The only way that the organisation could become familiar with the technology at that time was by utilising it. It this way, knowledge was acquired the hard way.

Was the IT effort cost justified? Throughout the period covered - from the early 1970s to the early 1990s, Customs has not grown at all in size - the same number of personnel now process four times the volume of cargo and passengers. Whether or not this is by working harder or smarter is immaterial; productivity is much higher and this is an important point when you remember that Australia's productivity figures are low compared with other nations largely because the public sector is assumed to have had no measurable productivity increases. The faster clearance of cargo has also improved productivity elsewhere. It must be admitted that not all of the systems are strictly cost justifiable, however, for a variety of reasons. One reason why the effort has slowed is that Customs is no longer so generous, having less to gain.

The principle that congruence between task and technology is essential was confirmed by the redevelopment fiasco. Customs became overconfident of its mastery of the technology. The result was expensive and painful, but through intelligent persistence, Customs achieved most of its goals.

The spiralling complexity of the modern computer systems was not fully understood. It seems that we now require a dump truck to do the work which was once done by a wheelbarrow. Operating systems in particular have become massive edifices which too often provide very little more for the average user.

The redevelopment, especially of the time critical PASS, drove home the importance of speed to the users. This largely explains why 4GLs have not become accepted at Customs. Considerably more powerful hardware still will be required before these can do the job effectively.

PASS II demonstrated that getting users to accept a new system is not an easy task, even when the new system is superior, and its nearly impossible when its not. The situation became even more difficult when the system was being used by the system owners as a deliberate means of changing the users' work practices, as CLEAR was. The users of the systems also were often hostile to each other: the TRACE users saw their job as arresting COMPILE II users!

In shaping the systems, the relationship between management outlook and IT outcome has, I hope been made clear. Some say the the failure to do great things since 1986 has been largely a failure of management to articulate their vision for the future.

Finally, the case shows the progression from systems developed "in house" to ones largely bought "off the shelf", but it is too early to say whether or not the Systems Division, as an entity, has a future. Some say that it is now redundant and that the owners can roll their own applications, but this is still some way off. The redevelopment showed that organisational wisdom has to be renewed; it cannot just be carried forward in such a turbulent environment.

Comments

( 1 comment — Leave a comment )
kerravonsen
26th Oct, 2005 20:06 (UTC)
Interesting, interesting.
( 1 comment — Leave a comment )