by Clay Carr, AI Expert, May 1992.
There's a new wrinkle in the way we look at the field of human performance improvement - something called performance support systems (PSSs). These systems offer a fertile field for knowledge-based technology. In fact, this may well be an opening to a far broader application of expert systems than any but the most optimistic proponents of them envision. Why do I say this? Expert systems represent a solution looking for a problem ("Say, is there somewhere here we can use an expert system?"). PSSs begin by identifying how performance can be improved, then use the technology that best accomplishes this goal. The performance support arsenal consistently includes expert system technology-- but now this technology apppears as the solution to an identified problem. I believe this difference is critical.
In 1988, Feigenbaum, McCorduck and Nii heralded the imminent explosion of expert systems in the United States with their book The Rise of the Expert Company: How Visionary Companies are Using Artificial Intelligence to Achieve Higher Productivity and Profits (New York: Times Books). As they examined American business, they saw that "almost everywhere, expert systems were speeding up professional work by at least a factor of 10. Even speedup factors of 20, 30, or 40 were common. And today's expert systems, powerful as they are are still Model-Ts. For instance, if we look at the economy, the gains in white-collar productivity are very low, almost zero. Considering that economists and business planners tell us that we should strive for productivity gains of five percent or seven percent per year, even the "small" speedups we saw were huge.
Four years later, the reality is that the explosion never happened. Whatever the successes, many of us have experienced resistance to attempts to implement expert systems. For all too many organizations, the expert system revolution has yet to happen.
Expert System's Problems
I have been a manager for more than 20 years, and though I'm more familiar with AI than most managers and thus more open to it, I still judge it by its worth for improving the performance of my organization. AI may be a superb research topic, but that's irrelevant to me. I want to know how it will help me accomplish my performance goals and whether it can do it more effectively and efficiently than other alternatives. In other words, I'm a typical manager. From a manager's point of view, then, why have expert systems so far fallen short of their promise? Among the many possible reasons, five seem particularly significant.
Strangeness. "Artificial intelligence," "expert systems," and "neural networks", whatever their virtues may be, sound strange. They may be exciting and very sophisticated, but they also sound terminally techie. Most aspects of AI bring with them the trappings of a world very different from that of day-to-day organizational life. It takes so much time to explain what expert systems are that it's hard to get to what they can do for somoeone. Consequently, the supporters of expert systems in an organization tend to end up a small coterie, convinced of the value of these systems but without a chance to implement them.
Lack of a constituency. Strangeness has another cost: expert systems have no natural constituency to serve as sponsors in most organizations. Since expert systems are a major computer techonolgy, you'd think that the management information systems (MIS) department would be a natural supporter. Not so. MIS is supposed to manage the organization's menagerie of computers, from mainframes to palmtops. To MIS departments, expert systems are just another splintering technology. From their point of view, even if the technology is worthwile (which is doubtful), it's difficult to control and means yet another bunch of applications someone will want MIS to maintain.
Unless an individual with influence happens to support the technology, expert systems are just as foreign to the rest of the organization. Sales, finance, accounting, marketing, human resources--all of them have their own concerns and problems that expert systems seem to have little to do with. For most of them, expert systems are a distraction rather than a help.
Threatening. We can take the two points above one more step-- expert systems appear threatening to many organizations. As a result, they must be "sold" at least one and often two to three levels above the one that will use them.
Here's an example, one that I'm sure has been repeated hundreds (or thousands) of times. One of the individuals in an organization I'm familiar with proposed an expert system for the organization's help desk. It never succeded. This was because she communicated (unintentionally) to the individuals on the help desk that the system would take over their most interesting and skilled work and turn them into little more than data input clerks.
Of course, that isn't necessary. Feigenbaum, McCorduck, and Nii stress that successful expert systenis should serve as assistants to individual performers. That sounds great, but the reality is somewhat different. Expert systems sold as assistants all too often turn into the real experts, thereby deskilling their human users.
Rigid. Expert systems are often threatening even when they don't need to be, because in isolation they represent a rigid technology. By this, I'm not referring to the structure of rule bases, frames, or other constituents of the system. I mean instead the identification of the funtion of the system as an expert. "Expert" is a social role; someone who is an expert knows more than other people and, presumably, is competent to tell them what to do. At least that's the understanding of an expert in an AI context.
Few performers function as experts in this sense. Instead, they may be coaches or mentors, helping others learn to perform certain tasks. They may be assistants, taking care of the routine work for others. Or they may be "dofers" (coined by Anthony Putman), implementing the decisions of others, and so on.
The point is, an organization has many social roles. "Expert" is just one of these roles, and not necessarily a particularly important one. However, AI has largely associated itself with this one role, to the neglect of the others. As long as this is the case, the range of solutions offered by expert systems will be unnecessarily constrained and rigid.
A solution looking for a problem. As significant as the other four drawbacks are, none of them compares with this one: expert systems represent a solution trying to find an appropriate problem. There's nothing new about this. Total quality management, self-managed teams, and just-in-time inventory have all been treated as solutions in search of problems. The CEO decides that the firm needs a dose of self-managed teams or cycle-time reduction and calls in an appropriate consultant. They work out the details and embark on the program. Typically, the effort fails. Why? New technologies flourish when they answer specific business concerns, not when they're imported as solutions in themselves.
Expert systems are even more apt to be seen as solutions in search of problems. While total quality management or cycletime reduction are often treated as roving solutions, they began as responses to legitimate business problems. At least they have the potential to solve the same kind of problem in other organizations. Expert systems, on the other hand, never arose as solutions to business needs. They were created because they were interesting to researchers.
True believers, shells in hand, go looking for someone who will let them implement an expert system. This makes for a very hard sell. Even worse, once it's sold it's apt to fall far short of its goal. And, if you think it's hard to sell an expert system to someone who's never seen one, try selling it to someone who's seen one that didn't work.
Lest you think that I'm over-stressing this "solution in search of a problem" aspect, let me give you just one example of what it can mean. General Motors spent $650 million to convert one of its Michigan factories to a high-tech plant, using the latest in equipment and computer support. The new plant managed to produce a car in the same amount time with only slightly more defects per car than it had as a low-tech plant. (At the same time, the NUMMI plant that Toyota runs for GM in Fremont, Calif., produces cars in 60% less time with 60% fewer defects, using traditional technology but dramatically different work processes and human resource management practices.) In short, starting with the solution rather than the problem can be disastrous, whether in an Al context or not.
Performance Support Systems
How does the switch to an emphasis on performance support systems counteract the problems associated with expert systems? Before we can answer the question, we need to look at what these performance support systems are. Several years ago, a number of us who were learning about this new technology called "expert systems" began thinking about how it might be applied to performance problems. Back then, we didn't have a name for what we were describing. I preferred to call it an "intelligent support environment." That name didn't stick; "performance support system" did.
A PSS is a computer-based system that uses knowledge-based systems, hypertext, on-line reference, extensive data bases, and allied technologies to provide support to performers on the job, where they need it, when they need it, in the form most useful to them.
Because of this emphasis on performance improvement, PSSs differ significantly from expert systems. PSSs aren't strange: improving performance on the job is an everyday concern. No'one ever gets threatened by a system that promises to help them do their job better. PSSs aren't rigid because they're defined, not by the technology used, but by the effect achieved. Finally, an effective PSS is a specific solution to an identified problem. (They also have very different sponsorship. Their problem isn't that they lack sponsors but that they can easily attract too many of them.)
Remembering that "expert" and "expert assistant" are really social roles, we might best understand what a PSS is by looking at four of the basic roles that it can fill:
Since this may sound a bit abstract, let me give some specific examples. Intel, like any chip-maker, needs to identify defective chips before they get shipped to customers. The firm has a PSS for its technicians that helps them do just this. Specifically, it lets them access pictures and descriptions of defective chips to compare these with the chips they see in their microscopes. The PSS speeds up the identification of defective chips significantly; before it was implemented, technicians had to remember what they had just seen, leave the clean room, look through a series of references to find a picture that matched the chip as they remembered it, then return to the clean room and make a final decision. For them, the PSS more than anything else performs as a librarian-helping them find relevant information rapidly.
Amdahl makes mainframes, to which their customers often attach hardware from other manufacturers. When a problem arises, the range of its potential causes is truely mind boggling. Amdaht has developed a prototype PSS that supports the technicians who must identify and solve these problems. It can perform as an advisor; if the problem is a known one, it simply explains the solution to the technician. If the problem is unknown, the PSS becomes a combination advisor and librarian and helps the technician find the documentation needed to solve it.
Retail Home Centers uses a very different kind of PSS: a combination advisor and dofer that designs decks for do-it-yourselfers. A salesperson gets basic information from the customer and enters it into the PSS. The system then uses the information to develop the dimensions of the deck, design the railing, locate any steps required, and otherwise create the finished plans for the deck. Because it builds a three-dimensional model, the customer can "walk around" the deck and even look at it from below. When the design is complete, the PSS displays the dollar cost and then, on request, prints out a three-dimensional drawing and a itemized bill of materials.
One of the earliest PSSs developed is still in use by the Caterpillar Co. It began, more or less, as standard CBT, but then moved to the worksite itself. A performer can access the system there, either to get information on a particular step or process or to access a specific training module. In other words, the system operates as a combination librarian and instructor.
Why Switch to a PSS?
These sound OK, but you're probably thinking that PSSs are nothing but expert systems with some bells and whistles added. From a technological point of view, that's an apt characterization. Most (though not all) of the PSSs mentioned do use expert systems but also employ other technologies--hypertext, CBT, databases, and so on. If you focus on the technology, though, you miss the point.
It is critically important whether one begins with a solution and looks for a problem or begins with a problem and then moves to a solution. Let me illustrate the difference. If we begin with the solution, expert systems, we might ask:
In each case, the answer is "yes." In each case, though, an expert system would have fallen short of what the PSS produced. Here's why:
Note the difference. If each firm had begun with the technology-- an expert system--it would have produced a system that met only part of its requirements or was more expensive than it needed to be. By beginning with performance needs, though, the company could select the technology that best met these needs. This focus on direct performance support is the critical difference between PSSs and expert systems.
How a PSS Helps
Using expert systems as part of PSSs helps avoid the five major problems of expert systems. We've looked at the first advantage of PSSs: They focus first on the performance problem and only then on the solutions. Now let's look at how they bypass the other four problems.
Avoiding strangeness. PSSs may not sound as appealing as AI and expert systems, but they don't sound as strange either. When we talk about performance, we're talking the organization's language. Just the change in language alone almost guarantees you will get your foot in the door.
However, this is more than just semantics. PSSs begin with performance problems and come into being as a means to effective solutions for these problems. Managers could not care less whether the solution contains the latest expert system shell, complete with hypertext, frames, and perhaps even case-based reasoning. Managers want to know what all this stuff will do to help their performance; we can worry about just what it is and how to use it.
If, however, we start by saying, "I can build you a system that will give your people just the support they need to perform better, just when they need it," we don't get mired in the technology. We can stick with what it accomplishes--and speak the language the manager understands best.
Removing the threat. By focusing on performance rather than AI technology, you evade both the threat and the unrealistic expectations that AI can create. This makes a PSS easier to sell not only to managers but to the performers who must use it.
Remember the example of the expert system that was going to "help" the help desk personnel? They rejected it, because it so clearly would deskill them. Suppose, however, that the had proposed a real performance support system--one that could support the help desk personnel as a librarian, instructor, advisor, assistant, and dofer (if necessary).
There's even more. Properly designed, a PSS could not only support performance but develop the skills of the performers. True, individuals learn from expert systems, even when learning wasn't part of the design. This is true with expert assistants as well. But why build a rocket so we can have Corning Ware? Why not set out to build a system that develops performers as they use it? Skilled workers are more flexible than any machines will be, at least in our lifetimes. Why not capitalize on this flexibility and on the commitment that workers can bring to the job? It creates more demanding design problems, but the potential payoff for the firm is much greater. And worker response is far, far more positive.
Gaining sponsors. Unlike expert systems, PSSs have a built-in clientele. We've seen that expert systems can't even count on the support of MIS personnel. In fact, MIS may be their strongest opponent.
The problem with PSSs is the reverse. When an organization decides to look into PSSs, it may find that:
In short, the problem is not too little sponsorship but too much. And it is a problem. Everyone wants a piece of the action. Even more, everyone would prefer to keep the action all to themselves. A successful expert system often requires no more than a sponsor willing to try it in a small corner of the organization. A successful PSS may require a sponsor that can persuade different groups to work together. Getting a PSS off the ground is typically harder than launching an expert system. Once airborne, though, a PSS will usually have much more organizational muscle behind it. And it is more apt to succeed precisely because of this broader sponsorship.
Gaining flexibility. By this point, it should be clear that PSSs are much more flexible than expert systems. PSSs are more tightly wired to the needs of the performance situation, and they perform in a much broader variety of roles.
Lest you're concerned that all this talk of roles is anthropomorphizing a mechanical system, let me say that expert systems aren't really "experts," and not even PSSs can be real librarians or coaches. Expert systems, PSSS, and other computer-based agents now fill roles that human beings have filled. In the words of Anthony Putman of Descriptive Systems, these systems are "artificial persons." It doesn't matter whether we want them to be artificial persons or not; they play those roles, and the humans involved with them react to them in those roles. It simplifies things to accept this and use it to your advantage.
The role flexibility of PSSs has an additional advantage, and it's an extremely important one. Large organizations that implement expert systems often surface a very real but previously hidden problem: workers don't do their jobs the way that higher levels of organization think they should. They don't follow established policies or procedures.
When a knowledge engineer develops an expert system to support what workers really do, it's often unacceptable to the part of the organization that sets the policy and procedure. But, if the expert system is modified to conform to policy, workers generally won't use it. This is where the flexibility of a PSS comes in handy. In this kind of situation, workers and policymakers can often agree on a set of tools (a PSS) that can facilitate more effective performance without having to choose sides in a debate.
The Bottom Line
As a manager, though, I know all too well how dangerous it is to implement solutions that aren't clear responses to real problems. Like most of my compatriots, I'm burned out on the "flavor of the month" programs that characterize American organizations. I want you to bring me something that helps, or I don't want you to bring me anything at all.
I think PSSs can help. As a manager, I like the idea of performance support provide just when and where it's needed. As a designer, I like the flexibility that PSSs give me for meeting performance needs. If these are true, PSSs are worth a look.
Clay Carr is the chief of the Defense Logistics Agency Civilian Personnel Service Support Office, which is developing an in-house performance support system capability.