® InfoJur.ccj.ufsc.br
Paul Bratley[1], Jacques Frémont[2], Ejan Mackaay[2] and Daniel Poulin[2]
[1]Département d'informatique et de
recherche opérationnelle
[2]Centre de recherche en droit public
One enduring characteristic of the law is change. To be effective in regulating the lives of those subject to it, the law must be certain, and it must be known. Yet it must be amenable to change when its rules need revising, either because blemishes need to be corrected, or because it has fallen behind social reality. For designers of legal expert systems, coping with constant change creates a major problem, since every change in the law can have repercussions on the knowledge base. In the worst case, a change in the law might require the entire knowledge base to be rebuilt.
A review of the literature on the subject of legal change provided little practical guidance. The present paper is therefore an attempt to pose the problem rather more concretely than it has been treated in the past, to summarize the attitudes to change we found in the literature, and to explain why more work is needed.
Our conclusion is that although most published work on legal expert systems pays lip service to the idea that they must be designed to cope with change, the problem is still entirely open.
One enduring characteristic of the law is change. To be effective in regulating the lives of those subject to it, the law must be certain, and it must be known. Yet it must be amenable to change when its rules need revising, either because blemishes need to be corrected, or because it has fallen behind social reality. Moreover change, always a fact of law, is increasingly frequent in the modern world.
For designers of legal expert systems, coping with constant change creates a major problem, since every change in the law can have repercussions on the knowledge base. In the worst case, a change in the law might require the entire knowledge base to be rebuilt. The challenge is to find ways by which this can be avoided so that only part of the knowledge base has to be reconstructed; better still, we might hope to design an expert system which can suggest ways to react when given information about necessary changes.
In earlier work [Mackaay 89] we dealt with time as it occurs within the context of legal rules. This paper deals with the different, but complementary, perspective of how legal rules themselves change over time. The law is created and evolves in four ways: through statutory changes, through bilateral or multilateral instruments, through the application of customs, and through precedents. For the purposes of this paper, we restrict ourselves mainly to the first form of creation of the law, that brought about through legislation or regulation.
The need for a detailed study of the problems created for legal expert systems when the law changes was brought forcibly home to us last year. Our previous work concerned unemployment insurance. In 1990 the Canadian Unemployment Insurance Act was substantially amended, and we are presently faced with the task of bringing our systems up to date. Towards the end of the year we had just finished an exhaustive study of some 1000 cases concerning claimants who voluntarily left their employment. We hoped to use this collection for experiments using case-based instead of rule-based reasoning. However the changes in the law mean that overnight almost all of them became irrelevant. In such a situation, what should we do -or perhaps, what should we have done when we first designed our system- to make it easier to cope with change?
A review of the literature on the subject of legal change provided little practical guidance. The present paper is therefore an attempt to pose the problem rather more concretely than it has been treated in the past, to summarize the attitudes to change we found in the literature, and to explain why more work is needed. We first outline the kind of changes we have encountered in our own field. As mentioned above, this is unemployment insurance, where the relevant Canadian statute was substantially amended last year. The amendments were studied by an expert, and we were thus able to construct a basic inventory of the types of legal change encountered, and to arrive at preliminary conclusions concerning how experts integrate such changes into their existing knowledge.
Next, armed with this experience, we survey the suggestions in the literature on legal expert systems and artificial intelligence in general to see how well they could be expected to handle the problems we encountered, and how the integration of changes in the law might be handled in a computer system.
Our conclusion is that although most published work on legal expert systems pays lip service to the idea that they must be designed to cope with change, the problem is still entirely open. We have no magic solutions to suggest; but we feel that bringing this problem to the forefront of designers' minds is a necessary step if ever we are to have real, working systems. At least one recent paper [Bench-C. 91] shows that others share our concern.
We review and comment upon a series of changes ranging from simple to rather complex.
Compare the old and the new versions of section 5, par. 1 of the Act :
New s. 5(1) :
Major attachment claimant means a claimant who qualifies to receive benefit and has been employed in insurable employment for twenty or more weeks [...]
Old s. 5(1) :
Major attachment claimant means a claimant who has been employed in insurable employment for twenty or more weeks ...
The only change is the addition of the expression "qualifies to receive benefit and," a change which has no effect, since the nuance was implicit in the old version.
Another example of a minor change with little or no effect is that of s. 6(2) which required a person to have twenty weeks of insurable employment before qualifying for a certain category of benefits; in the new s. 6(2), the twenty weeks rule is replaced by a table. The changes required to the knowledge base are of minor importance.
The order of the paragraphs of s. 6 has been inverted between the old and the new provisions. What is considered the main rule now precedes the exception. The terminology has been made "gender-neutral." These changes do not affect the expert system.
In some cases the knowledge base is modified by using, in a different context, tests or provisions already present. For instance, s. 10(4) has recently been modified to add a provision already found in s. 37(3) of the Regulations dealing with interruption of earnings. In such cases, parts of the existing knowledge base can be imported more or less as is into the new knowledge base.
Under the old Act, section 30 provided that where the applicant refused work without good cause, lost his employment as a result of misconduct or left voluntarily, benefits would not be payable for a "penalty period [...] not exceeding six weeks." This provision has been changed to read that the number of weeks of disqualification "shall be not less than seven and not more than twelve."
On the surface, this looks like an insignificant change, whose implementation requires no more than replacing 0-6 by 7-12. For the expert, this is not the entire answer. Since the disqualification now lasts at least seven weeks, even the minimum constitutes a rather severe penalty. The expert therefore expects that authorities deciding whether work was refused without good cause, whether the applicant misbehaved or whether he left voluntarily will interpret these clauses more restrictively than before to avoid imposing what they see as an unduly harsh penalty. Technical changes aimed at "tightening up" the Act thus lead to unintended conceptual changes.
Section 28(1) of the Unemployment Insurance Act reads:
A claimant is disqualified from receiving benefits under this Part [...] if he voluntarily left his employment without just cause.
It is implicit in this provision that the applicant meets the normal conditions for qualifying for benefits. Special circumstances warrant a penalty, which was dealt with earlier. The special circumstances, the voluntary character and the absence of just cause, are concepts that are not directly observable but are left to be specified by decisions rendered in individual cases. All are what lawyers call "fuzzy concepts." They require "tests" in terms of directly observable facts. As a result, the fuzzy concept in the expert system becomes either the final consequence of a set of rules upon rules which can reach considerable complexity as the law develops, or of a case-based reasoning system. Handling such fuzzy concepts may require separable sub-systems within the knowledge base.
Section 28 was changed by the addition of a new paragraph (4), which reads:
(4) For the purpose of this section "just cause" for voluntarily leaving an employment exists where, having regard to all the circumstances, including any of the circumstances mentioned in paragraphs (a) to (e), the claimant had no reasonable alternative to immediately leaving the employment:
(a) sexual or other harassment;
(b) obligation to accompany a spouse or dependent child to another residence;
(c) discrimination on a prohibited ground of discrimination within the meaning of the Canadian Human Rights Act;
(d) working conditions that constitute a danger to health or safety; and
(e) obligation to care for a child.
Superficially this provision merely codifies existing case law and should be easy to integrate into the knowledge base. However, the test developed by case law has been rendered more severe by the addition of the condition preceding the colon: "the claimant had no reasonable alternative to immediately leaving the employment." This condition will have to be added to the knowledge base. It contains two further fuzzy concepts: "reasonable alternative" and "immediately." Thus it entails a restructuring of the sub-system which implemented the old test.
If legal norms change, so does the expert's perception of the legal world. In updating his comprehension of the legal norms, he has recourse to many techniques to evaluate the impact of such changes. These can be extrinsic or intrinsic. Extrinsic techniques include recourse to general common law rules of interpretation as well as rules imposed by the Interpretation Act; but they also include elements of common sense. Intrinsic aids imply recourse to rules specific to the field, whether specific interpretation rules or understanding of the general aims of the normative scheme. The basic working tool of the expert analysing modified rules is semantics: the meaning of the words used by the norm.
We are led to think that any formalization of the techniques by which experts update their knowledge requires a better understanding than we have, not only of the way norms change, but also of the way experts integrate and understand these new rules.
On another level, the preceding examples allow us to comment on the fundamental nature of legal change. Among the few things which are clear is that law generally tends to change incrementally, which means, in a majority of cases, without destroying the knowledge base on which it functions. When this happens, the law builds on the existing base, accumulating and superposing rule upon rule. Thus as the law evolves, it requires us to use an ever more complex knowledge base, where the first rules are overlaid by newer ones, and these others are overlaid in their turn.
Moreover, in coping with legal change, it appears that statutory changes of the nature previously discussed do not effectively change the law overnight. In many cases, the expert has no clear answer to the legal difficulties raised by a modifying provision. In such cases, he integrates the new rule into his knowledge base, but without necessarily drawing definite conclusions on its real impact, waiting for subsequent judicial or administrative determination. Uncertainty, in such cases, necessarily follows.
The design of a computerized system usually requires that the procedures to be computerized be relatively stable. There are however examples where computer systems are constructed to handle situations whose duration is minimal: in some cases one has no choice, or else the profit to be realised makes the game worth the candle. Similarly, certain branches of the law, although subject to frequent modification, are nonetheless worth studying by researchers in artificial intelligence. This is particularly the case where the law affects large numbers of people, where the sums of money involved are enormous, and in areas which have resisted attack by more traditional techniques.
In such rapidly-evolving systems the methods used in artificial intelligence are often preferable to the procedural techniques of traditional computer programming; they are easier to adapt to less stable problem areas. In particular, artificial intelligence uses an explicit representation of the necessary knowledge, making it relatively easy to determine those areas of the knowledge base which correspond to a particular part of the problem domain. Thus when this domain is modified, it is easier to modify the corresponding structures in the knowledge base. In a comparison of the two approaches, McCarty pointed out the difficulty of modifying the kind of procedural system typified by the CORPTAX program [Hellawell 80] and contrasted this with the possibilities offered by TAXMAN.
[Finally, and closely related,] there is the difficulty of modifying the CORPTAX system whenever the legal rules are modified. By contrast, in a system such as TAXMAN, the representation of legal rules is explicit, declarative, and modular, and a rule can be modified in most cases by modifying a single data structure. [McCarty 83, p. 283]
While it is certainly true that modifying a knowledge base is easier than modifying a program where the rules are represented implicitly, this does not mean that such modification is simple. Moreover a number of legal areas of great interest for applications in artificial intelligence, such as tax law and much social legislation, are among those where change is most frequent. This provides an additional challenge to be met in the design of expert systems. The following paragraphs survey some proposals which have appeared in the recent literature.
The area handled by the NSA is particularly stable and well-defined. No doubt for this reason the techniques used in its construction pay no special attention to managing changes. The rules used in the system are constructed on the basis of a thorough analysis by an expert of all the relevant cases. For the NSA, only 74 cases in all are deemed to be relevant among all the cases recorded this century in Australia, Canada, Great Britain, and New Zealand [Smith 87, p.88]. Such a system might almost be designed never to change. Using the techniques employed by Smith, which require a global analysis of every single case to establish the necessary legal rules, any new decision modifying the law would imply that the whole analysis must be reworked. If such techniques are acceptable in stable, narrow areas, it is clear that they are of little utility in many contexts where expert systems are most urgently required.
Quite different is the area chosen by Schlobohm and Waterman for their EPS system, [Schlobohm 87]: the rapidly changing area of the law to do with estate planning. In EPS, a rule-based system, the knowledge used is provided by the practical expertise of an expert in estate planning. However as tax law is an area which tends to evolve rapidly, this approach led to problems. Schlobohm and McCarty subsequently described them in the following terms:
[EPS] does not contain any knowledge about how the rules were obtained [...] As a result, human experts would have to modify the heuristic rules whenever the law changes, and the entire system containing the new rules would then have to be debugged. Finally, EPS [...] would be of little use to expert estate planners, since experts only need assistance when the law changes or when dealing with a novel client situation. [Schlobohm 89, p. 9]
This difficulty is inherent in any system which makes use of the heuristic knowledge of a human expert without conserving a link between this heuristic knowledge and the primary sources of the law. Therefore such systems are ill-adapted to handling areas where change is frequent. For as Schlobohm underlines, any change in the law must first be assimilated by the expert and integrated into his conceptual map of the field in question, and only afterwards can the rules be produced and merged into the expert system.
More recently Schlobohm and McCarty have proposed a new design for an estate planning system: EPS II [Schlobohm 89]. This system is to be built using the Language for Legal Discourse (LLD) [McCarty 89]. In EPS II, the expert's heuristic knowledge is complemented by an explicit representation of the rules laid down by the Internal Revenue Code. Both kinds of knowledge are expressed in the form of propositions in LLD. When the system is used, prototype solutions furnished by the human expert can be modified or "deformed" to satisfy the needs of the client and the constraints of the law.
The authors of EPS II, aware of the difficulties caused by changes in the law, underline the advantages of their design in this respect, and present an example showing how certain kinds of change to the Code can be handled by EPS II. They do not claim that all changes can be taken care of in this way. In particular any modification of the Code which adds a new concept or radically changes an existing one is beyond the powers of EPS II. In general, the problems of maintaining a system will become even more difficult for systems possessing a conceptually rich and complex representation of the application area precisely because of the deep analysis they require. We doubt that such an approach will be practical in an area which changes as rapidly as that of tax law. The situation will be even worse if indeed "experts only need assistance when the law changes or when dealing with a novel client situation." [Schlobohm 89, p. 9]
This team are working on the problems involved in maintaining legal expert systems, which they call Advisory Systems on Legal Questions (ASLQs), [Weusten 89; Smits 89]. Their papers provide a reasonably complete characterization of the changes which may have to be made in a legal expert system. Weusten draws up the following list: strictly textual changes in the expert system; changes in the law which do not entail changes in the knowledge representation; changes which imply that the system's rules have to be modified; and the case where the formalization must be completely restructured following legislative changes.
The measures suggested by these authors to facilitate system maintenance are relatively elementary, and furthermore they depend on the idiosyncratic fashion in which ASLQs are implemented. In these systems the necessary knowledge is first assembled into a data base, and auxiliary programs then compile this data into a Prolog program. The authors propose that the associations between the rules and the sources of the law (cases and statutory provisions) should be kept in separate ASCII files. Tools are provided to allow rules to be edited in an explicit form. No useful guidance is available for handling changes in the last two categories above; yet this is exactly where we most need help.
Quast and de Wildt, of the Leidraad project, have developed a knowledge base to deal with the concept of "commensurate work" in Dutch unemployment insurance law [Quast 89]. Their rule base was built on an analysis by an expert of 150 cases decided by the Central Court of Appeal. The rules use a weighting of the characteristics of employment judged to be "suitable." On the subject of changes in the law, these authors remark that "updating the knowledge base must be feasible without a complete reorganization." They believe the tree structure used in their system will suffice to attain this objective. We are less sanguine. Should the legislature tighten the criteria defining "commensurate work," this will at best require that the analysis of the 150 cases be carried out again; at worst it could make them all irrelevant.
In her doctoral thesis [Oskamp 90] Oskamp devotes much of the penultimate chapter to the problem of maintenance, noting that it has so far attracted little attention. She points out that besides translating legal changes into corresponding modifications of the knowledge base, it is often necessary to keep one or more older versions. The system may need rules of transitional law telling it which version is relevant to a given problem. Since statutory changes do not come labelled as "simple" or "complex," maintenance is not a mere technical job, but as much a problem of expert interpretation as the original creation of the knowledge base.
In the area of statute law, the most favourable approach to obtain an easily-modifiable system is to use a formalization which makes the articles of the statute correspond to the rules in the knowledge base. This is the line taken by the researchers at Imperial College. Every rule in such a system is supposed to be a formal paraphrase of some provision of the legislation and to have an explicit linkage back to the statute.
From the point of view of ease of maintenance, this is ideal. Modification of a paragraph in the text of the law simply implies that a corresponding modification has to be made in the associated rule in the knowledge base.
While such a straightforward formalization of the law is easy to maintain, it is less clear that it is useful in an expert system. The question thus arises whether the system can be made practical without losing the ease of maintenance. Two remarks can be made. First, a myriad little conventions are necessary to transform a legal text into a logic program. Secondly, such a logical formalization cannot serve as the base of an expert system unless one adds a layer of pertinent heuristic knowledge and other general knowledge of the kind used by jurists. This is exactly what Kowalski and Sergot described recently. Taking their own work as an example, they admit that their model represents "a layman's reading of the provisions. This in itself renders our British Nationality Act program of limited practical value" [Kowalski 90, p. 207]. They explain that such a program will need the help of a human expert, to verify that the formalization is correct, to modify and augment the formalization to handle vague concepts, to suggest the best way to apply the rules of the system in particular cases, and so on.
Once the formalization is structured, explained and augmented in this way, modifying the rules of the system following a change in the law is no longer straightforward. For it is now also necessary to modify implicit or explicit rules which do not correspond directly to paragraphs in the text of the law, and we find ourselves once again in the same situation as with more heuristic systems.
Perhaps the most explicit acknowledgement of the problems of coping with change can be found in the work of Bench-Capon, now at the University of Liverpool. In [Bench-C. 91] the authors state that "the greatest barrier to the routine use of knowledge based systems techniques for practical legal applications lies [...] in the problems associated with the maintenance of such systems." Like the researchers at Imperial College, Bench-Capon's group insist that the knowledge representation must mirror the sources of knowledge. The MAKE project is investigating issues connected with maintenance of regulation-based systems in the area of compensation for work-related injuries. The aim is to provide tools for "minor maintenance", that is, to deal with day to day changes due to modification of the legal texts and the application of the law. The project is too recent to judge the value of its results.
Our own work on the Chomexpert system [Poulin 88] allowed us to experiment with changes in the law and their effect on an existing knowledge base. In particular the important modifications made recently to Canada's unemployment insurance law provided a challenging test case.
In Chomexpert, the knowledge base is organized into modules, each corresponding to a legal concept identified by the human expert. We asked the expert to follow the structure of the text of the law as closely as possible when he defined what had to be done by the system in each particular case, only deviating from the text of the law to the extent necessary to obtain a usable knowledge base. This was intended to allow us to cope with change easily. We also provided a labelling technique by which each rule in the knowledge base could be related to the articles of the law which gave rise to this rule, and vice versa.
When the law was changed, our methods turned out to be quite incomplete. Needless to say certain quantitative changes, which are simply changes to numerical parameters, are easy to handle. It suffices to identify the rules concerned and to modify them slightly. However other changes in the law make the precautions we took look very inadequate. Recently, for instance, we have been working on the problem of handling an ill-defined notion mentioned above, namely voluntary departure. Every rule which, in the opinion of our expert, represented the state of the law on this point will have to be revised. The facts that our formalization of the law adheres closely to the text, and that the rules are related to the paragraphs of the law or to cases which support them, turn out to be of no help whatever.
The case-based approach complements the rule-based approach of the systems discussed above. In the legal field, this avenue has been explored principally by Rissland's group at the University of Massachusetts. One aim of this group is to improve the performance of legal expert systems using case-based reasoning (CBR). According to Skalak [89a], the weaknesses of legal expert systems that can be alleviated by CBR include their shallowness, their tendency to overreach their expertise, and their inertia, that is, the difficulty they have in assimilating new knowledge. Here we consider only how using CBR might reduce the inertia of a legal expert system.
The CBR systems proposed for legal applications are of the type known as "precedent-based.'' They use precedents not only to find a solution (for example, when applying an "under-defined rule'' to a new set of facts [Skalak 89a, p. 676]), but also to justify and explain the solution which is found. Skalak describes the way they function as follows:
If one wants to emphasize the appropriateness or relevancy of past case to a new one, one concentrates on their similarities; if one wants to block such a view, one concentrates on differences. [...] The key idea is to reason from cases similar to the current case in order to argue for a particular interpretation in the current case and to justify the reasoning in terms of the past cases. [idem, p. 681]
To do this, these systems use several sources of knowledge, in particular a collection of cases, an index to this collection, and metrics for measuring similarity and relevance.
The addition of this type of system to a rule-based legal expert system could improve its performance when the law changes. The essential idea is that it is easier to add new cases to a collection than it is to add new rules to a rule-base. No doubt this is true. It is less clear how a mixed system based on both rules and cases would work in a changing context. When new cases are added to the system, the latter may well be able to include them automatically in its collection, and this is certainly worthwhile. However when the law changes, the problem of inertia must still be faced. For instance, many modifications of the law springing from case law cannot be directly introduced into the system in this way either. New "dimensions" of a legal concept may make their appearance. Skalak [89b] has studied the inconsistencies between cases (which he calls "incursions") observed when an essential dimension is missing. If such incursions may be manageable in a small collection of cases, what are we to expect in systems with a much bigger collection such as will be needed in practice? Furthermore, once the incursion has been identified and the new dimension added, this addition of a dimension will make it necessary to re-index all the cases using this new criterion. Lastly, a statute modifying an "under-defined rule" may render obsolete a number of cases already in the collection.
As far as the rules are concerned, they too may need revising when the cases change. Skalak and Rissland propose using ID5, a classification algorithm [Utgoff 89], to induce new rules corresponding to the modified case base. They have carried out a number of experiments with this in view [Skalak 90], so far with mixed results. We are forced to conclude with Rissland and Skalak that "there is no quick cure for this fundamental problem."
Legal expert systems are not the only ones which have to deal with change. An increasing body of literature is devoted to the formal study of knowledge base revision. The fundamental problem is how to revise the knowledge base when new, contradictory information is obtained, or when we wish-for whatever reason-to reject some conclusion produced by the system. If we want the knowledge base to remain consistent, the conflict must somehow be resolved.
As Makinson [1985] points out, this revision process may be tricky. On the one hand there may be several ways to resolve the conflict, and then we need a criterion for choosing one; on the other, Makinson shows that the formulation of the knowledge base is important too, in the sense that two equivalent formulations of the same knowledge may behave differently under revision. Nevertheless some progress has been made on the general problem.
Much of this work starts from a set of postulates for rational revision of a knowledge base first proposed by Gärdenfors, but most conveniently set out in Makinson [1985]. Dalal [1988] added additional principles such as Irrelevance of Syntax (that is, a revised knowledge base should not depend on the syntax or representation of either the old knowledge or the new information). Katsuno and Mendelzon [Katsuno 1989] give an excellent introduction to the area, and review a number of proposals from the literature.
More recently, Gärdenfors [1989] has proposed that "the key concept [...] is the notion of normative entrenchment." By this he means that the rules in a system do not all have the same status, since some are regarded as more basic than others and therefore less prone to amendment. Hence when a system is revised, the rules that are given up first are those with the lowest degree of entrenchment. Gärdenfors has legal systems particularly in mind when he makes this suggestion.
None of these proposals seems sufficiently developed to be applied in a real-world legal expert system. However they offer hope that someday it may be possible for a legal knowledge base, when presented with new and conflicting rules, to suggest possible revisions to the user, pointing out rules or groups of rules which might be changed to restore consistency. The idea of entrenchment, too, has obvious applications when changes in the law leave lawyers uncertain about how the jurisprudence will develop: those areas where there is the most room for uncertainty will be the least entrenched, while the basic rules of legal interpretation will go unchallenged.
Legal expert systems must be able to face up to frequent and sometimes far-reaching changes in the law. A brief survey of the proposals for handling change, made by researchers in this area, as well as by others with a more global view of change in knowledge representations, suggests that although many have recognized the existence of this problem, no-one has yet found a practical way to attack it.
The work done at Imperial College and in Bench-Capon's group underlines how important it is that the rules in the knowledge base maintain links to the statute or in general to the primary sources of the law. A number of systems, including ours, hew to this line. Case-based reasoning does not provide a definitive solution, but can contribute to the development of systems better able to deal with change when this is the result of changes in case law which do not introduce new concepts. Systems which use the richest and most complex representations of their knowledge are likely to be those with most "inertia." Indeed it may be that building an elaborate model of the law takes longer than the time between changes. Formal approaches are promising, but not yet practical. Lastly, a better understanding of how legal experts manage and integrate change in the law is vitally necessary.
We propose in the coming months, using our Chomexpert system as a testbed, to see whether it is possible to design and include mechanisms specially intended to cope with change. Such mechanisms would ideally include techniques by which the system itself might suggest possible ways to react to changes in its rules and conclusions, leaving the human expert to make a final choice. However we are still far from this ideal.
The work described in this paper is supported by a grant from the Social Sciences and Humanities Research Council of Canada and by the Fonds FCAR of the Québec government under its programme of Actions structurantes. We thank the referees for pointing out some very recent work on the subject of change.