® InfoJur.ccj.ufsc.br
Instituut voor Taal- en Kennistechnologie
Institute for Language Technology and Artificial Intelligence
T.J.M. Bench-Capon
Department of Computer Science,
University of Liverpool,
Liverpool,
England
Abstract
This article discusses some criticisms of the use of rule based representations in the construction of legal knowledge based systems. The conclusion will be that these criticisms are misplaced; either because they misunderstand the nature of support which such systems can offer to legal decision making, or because they misunderstand the motivation and ambition of the systems. I will argue that rule based legal KBS can provide useful support, provided users can both understand and carry out their proper responsibilities when using them.
In [15] Ronald Stamper wrote:
The consensus among legal philosophers is probably that rule-based legal expert systems leave much to be desired as aids in legal decision making.
When discussing the approach to building legal systems which involves the deduction of consequences from a formalisation of legal knowledge as a logic program, of which [13] provides the best known example, Robert Moles wrote in [12]:
most of this work has involved important, but erroneous, assumptions as to the nature of legal reasoning and law.
Moles' arguments would apply not only to logic programs, but to any form of rule based legal knowledge based system. These two quotations are illustrative of a growing conviction that rule based representations are inadequate, or even harmful, when it comes to building legal knowledge based systems. As I have long advocated the use of logical models in connection with such systems, I want in this paper to consider a couple of the specific objections. In doing so my objectives are twofold: to indicate that we should not abandon the use of the rule based approach altogether, and to come to a better appreciation of the role of systems built using this approach.
The specific objections I shall consider are those made by Moles, and those that Stamper characterises as ``application problems''.
Moles' objections can be summarised as ``Law is not rules'' and that in consequence any representation of law as a system of rules will traduce the law. Moles first made this point not in the context of legal knowledge based systems, but in the context of jurisprudence: his book Definition and Rule in Legal Theory [11] is largely a critique of Hart and his view of jurisprudence which suggests that rules do have an important role in law [9]. It is indeed possible to see his assault on rule based legal knowledge based systems as a continuation of his attack on this jurisprudential view. Moles' own position emphasises the importance of ``consequentialism'' whereby the consequences of a decision are giving over-riding importance in deciding cases. He writes ([11], p271) of
``the complex decision making which is involved in any legal case, ... it involves a process of complex negotiation which can only be carried out in the light of the many considerations obtaining at that time. ... In stating that legal rules can be applied without further judgement; that they apply in an all or nothing fashion; that legal decision making follows the form of the syllogism or that it is a pattern matching routine, the modern positivists, joined now by the computer scientists take us along a dangerous road.''
I do not wish to attempt to arbitrate this jurisprudential dispute. None the less I will make the following comment. Whilst Moles has made out a persuasive case that cases are, on occasion, decided in a way which appears to conflict with previously accepted rules, and with previous decisions, and, moreover, that such cases may rightly be so decided, this does not mean that deduction has no role to play. Any suggestion of a dichotomy is misleading. For a full, and to me convincing, advocacy of the case for deduction in legal reasoning see MacCormick [10].
Some personal anecdotal experience may, however, be of interest. Although not a lawyer or a jurisprude, I did work for several years for the UK Department of Health and Social Security, in a position which involved the formulation of policy and legislation. When formulating legislation it was always recognised that it would be impossible to foresee every circumstance to which the legislation would be applied, and hence it was not desirable that the legislation be too tightly drafted. That is, the intention was to lay down certain very specific conditions, such that a benefit would not be payable to anyone who was under 16, or over 70, or not resident in the United Kingdom, but that certain other conditions should be specified in such a way that the adjudicator would have a fair degree of room for manoeuvre so that a just decision in the light of actual circumstances could be reached. One specific example concerns the now happily defunct benefit of ``Housewives Non-Contributory Invalidity Pension''. This was a benefit payable to married women, who had failed to establish a connection with the labour force through the payment of National Insurance contributions, but who were, through reasons of ill health incapable both of paid employment, and of ``a substantial extent of their normal household duties''. The use of the term ``substantial'' here was to allow just the kind of flexibility, and to avoid the impossible task of prescribing a definition of ``household duties''. In practice the wording was too loose and there was great variation in the treatment of people, depending on the views of individual adjudicators. When ultimately the matter was determined by a Tribunal of Commissioners issuing a definitive interpretation it was rather more liberal than the original intention of the legislators, and the legislation was accordingly amended. Two points are worth noting: that the legislators intended to allow flexibility to take account of individual circumstances within a framework of more rigid conditions, and that when the amount of flexibility was demonstrated to be too great they felt able to alter the situation by further legislation. In this case the interplay between deduction from rigid conditions and judgement as to circumstances seems clear, at least from the perspective of those responsible for drawing up the legislation.
Moving from the nature of law to computer science issues, there are two points which Moles seems to believe that I wish to discuss: first that use of a rule based representation involves a commitment to the belief that the law does comprise a system of rules; and second, that use of a rule based legal expert system requires one to believe that the rules ``can be applied without further judgement'', and that they apply ``in an all or nothing fashion''. I believe that neither of these statements is true.
Suppose first of all that a fairly extreme version of Moles' position was the case: that the judge could decide a case, say a benefit claim, in whatever way seemed right, unfettered either by the written legislation, or by the precedents of past cases, but that the onus of establishing the claim lay on the claimant. We could record this state of affairs in the following way:
award_benefit_to(Claimant,yes):-
adjudicating_claim(Judge,Claimant),
seems_deserving_to(Judge,Claimant).
award_benefit_to(Claimant,no).
Note that we must identify the judge responsible for making the decision, since the opinion of other judges can carry no weight here. Note too that the burden of proof is placed on the claimant, since if the judge cannot answer affirmatively to the second condition, the decision will be made in favour of the defendant. Here we have a rule based system, even though no rules, other than those deriving from the procedures, are of relevance.
Admittedly the utility of such a system will be small, although it does make clear that the matter is one of free discretion, for the appropriate judge. Moreover, note that the second condition can only be determined by the appropriate judge: any other person will be able to produce no more than an informed guess, based on facts and knowledge of the judge and his past behaviour.
In practice, even where wide discretion is permitted to the judge, there will be certain conditions which the legislators desire to be satisfied. Thus, for example, the benefit may be intended for persons over a certain age, resident in this country, and these conditions may be prescribed in the legislation. The situation will now be:
award_benefit_to(Claimant,yes):-
adjudicating_claim(Judge,Claimant),
age(Claimant, A),
A > 65,
resident_in(uk,Claimant),
seems_deserving_to(Judge,Claimant)
award_benefit_to(Claimant,no).
Here we have what is the usual case: a mixture of relatively clear cut conditions, and a degree of flexibility, although the flexibility is still rather extreme. The effect of the system now is to direct attention towards two specific points: under-age claimants and those living abroad would see little future in claiming, and the judge would need to be satisfied as to the claimants fulfilling these two conditions before exercising discretion. The problem, on the consequentialist view, would occur only when a judge felt that the claimant ought to be awarded the benefit, but was either not old enough, or not resident. Here the judge is able to decide in favour of the claimant, although the justification of the decision may prove tricky: ``resident_in'' is something of a term of art, and so gives scope for creativity, but if the claimant was born only 55 years ago, the judge might need to resort to some fairly extreme interpretation, perhaps holding that such a person should be considered as 65 on grounds of health, the toughness of their life, or something similar.
What this shows is that the role of the judge is crucial, and that the rules of the expert system cannot be interpreted in an all or nothing fashion. In effect, the predicates, no matter how clear cut they may seem, require the assent of the appropriate judge, and so the correct formulation should perhaps be:
award_benefit_to(Claimant,yes):-
adjudicating_claim(Judge,Claimant),
holds(Judge,
age(Claimant, A)),
holds(Judge, A > 65),
holds(Judge(resident_in(uk,Claimant)),
seems_deserving_to(Judge,Claimant)
award_benefit_to(Claimant,no).
Alternatively, one might consider this an unnecessary complication: the need for the appropriate judge to subscribe to the various decisions might be held to be capable of remaining implicit in the representation, to be brought out in the use of the system. Thus if it were the judge using the system, he would need to bear in mind at all times that he was making a legal decision, with due regard to the consequences, and so that the answers to the questions should be given in the light of this. Similarly if it were the advisor of a claimant making the decision, he would need to remember that the judge would take these considerations into account, and so the answers given by the advisor could never be more than informed predictions of the view that the judge would take. So construed, of course, both users recognise that the rules of the system cannot ``be applied without further judgement'', and that they do not apply ``in an all or nothing fashion'': such users must recognise that they do no more than identify issues, which must be decided in a jurisprudentially appropriate fashion.
So far we have considered only a system that might come from formalising legislation alone. To make a truly useful system we will want to incorporate expertise, knowledge of how these conditions are likely to be interpreted. For a general discussion of how we can graft expertise onto a formalisation of legislation see [2]. In the simplest case this may just be the inclusion of some arithmetical ability, so we might add:
Age(Person, A):-
date_of_birth(Person, Date)
difference_in_years(Date, Today,A).
Such a rule would provide a convenient way of calculating the age of the person, but not the only way: it is not intended to curtail the freedom of a judge to arrive at a different answer, nor to preclude an age being attributed even where the date of birth of the person is unknown. We must, however, distinguish sharply between the failure of the first condition, which indicates that the date of birth is unknown, and the failure of the second condition which indicates that the claimant's date of birth is known to be too recent. For the rule to be evaluated correctly by the user, the reasoning of the system must be transparent. This point will be further discussed below.
We can, moreover, extend the scope of expertise to give guidance even on discretionary predicates. Suppose that widows had always been found deserving of this benefit. We could add a clause:
seems_deserving_to(Judge,Claimant):-
widow(Claimant).
A rule such as this must, however, be clearly flagged to the user, so that it is distinguished as a generalisation from case law. It does not supply a sufficient condition, but only a generalisation of past experience, the sort of generalisation that experts may reach by reflecting on their knowledge of past cases, but one which has no status and may well turn out to be false. A judge will not be obliged make this induction from the past experience, and so is under no obligation to follow this rule. An advisor using the system must be fully aware of the judge's freedom to make some different induction and so to ignore the rule.
This places the relation between past cases and a decision in a particular case in an interesting light: the adjudicator must, it seems, make the induction of the rule afresh each time a case is decided. Only when the adjudicator commits to the rule in the context of a case can deductions from that rule be made. This helps to explain why, when presenting a case, it is not the rule that is put forward, but the decisions, albeit in such a way as to encourage the induction of the desired rule by the adjudicator.
If therefore we wish to incorporate such rules in a knowledge based system, it is essential that we present to the user the backing for the rule, indicating that it is a possible induction from a set of cases, rather than being an expression of legislation. In this way the users can commit to, and hence accept the rule, or reject the induction and hence the rule and the conclusions that may be derived from it, as they see fit.
For these reasons it is important that the results of the system must be presented to users in a way which enables them to distinguish the various rules employed, and to see the backing for, and status of, such rules, so that they can commit themselves to the rules employed and to the base facts which license the application of those rules. The users will therefore require more than a simple proof trace which constitutes the standard ``How?'' explanation of typical expert systems. Rather the user must be presented with an argument, with the steps made clear. Techniques for presenting the output of the system in the form of an argument susceptible to such a critique based on the argument schema of Toulmin [17] can be found in several previous papers I have written: [5], [6] and [7].
Thus when examining the output from such a system, the user must be aware that every assent to a predicate, and every commitment to a rule represents a separate legal decision, which must be made in the context of the case under consideration. Where the user is an adjudicator, the user will have the power to make, and the responsibility to make, these decisions. Where the user is an advisor, the user must predict the decisions that the adjudicator will make. Where the user is an advocate, the user must persuade the adjudicator to make these decisions in a way favourable to the client. Thus the various potential users of a system stand in quite different relations to the rules of the system, and must take their relation to these rules into account in their use of the system.
We can now see that the problems that trouble Moles stem not from the use of rules as such, but from the use that a rule based system might be put to, if used in a simplistic and unreflective way. The users must be able to understand their relation to the rules of the system, and different types of rule and their authority must be distinguished. Toulmin's argument schema can be used to make explicit the underlying argument which allows the users to commit appropriately to the various steps.
Moles' arguments thus have power only against naive rule based systems, used naively. His challenges may be fair in the case of the original British Nationality Act system [13], which can be seen as representing a rather extreme view, They are not, however, powerful when directed against the bulk of subsequent work which takes a much more subtle view of the role of the user, and nature of the representations, whilst remaining firmly in the rule based tradition.
Properly used, rules are not ``all or nothing'', nor does building a rule based expert system in law rely on identifying rules which have some pre-existence in a Platonic world of jurisprudence. Rather the rules are justified by the artefact they constitute: a matchstick model of St Paul's is a model of St Paul's, and no criticism on the grounds that St Paul's is made of different materials can be adduced. A rule based system must provide support for a class or for classes of user, and those users must be aware of several things: of their relation to the system; of the way in which the conclusion of the system is reached; and of the part they must play in enabling the conclusion to be reached. A rule based system thus needs a fair degree of sophistication, both in the presentation of its reasoning and, perhaps more importantly, on the part of its users. Given the required sophistication, a rule based system can provide effective support to a user carrying out a task.
In [15] Stamper writes:
``The mechanical nature of the classical logic approach ... introduces semantic problems. They were classified as either theoretical, or as by-products of applications ... The application problems that commonly arise when predicate logic is used for building expert systems were seen to be numerous. They are almost entirely caused by the freedom to formulate fat predicates. This liberty leads the analyst to invent a private language, introducing complexity, system rigidity, artificially enforced uniformity, the illusion that the machine uses natural language, and explains many of the difficulties in maintaining and merging systems.''
I shall not discuss the theoretical problems here, since they will take the debate into too deep waters. I do, however, want to say something about the application problems and ``fat'' predicates. By ``fat'' predicates are meant the ``elongated symbols that look like natural language'' such as ``X acquires british citizenship by section Y on date Z''.
The point is not new: Stamper made it in 1988 [14] and I wrote a short reply to these criticisms in [1]. It appears, however, that my arguments there were not accepted, so I shall make some further argument here.
I will begin by pointing out is that there do exist useful systems, both in law and in other domains, which are based on no more than propositional logic. If this is so, the move to predicate logic, however fat the predicate, cannot make them less useful.
Doubtless Stamper would also reject propositional logic. The root of his rejection is, however, a difference in motivation. What Stamper wants is to uncover some of the semantics of the domain. This aim is interesting and laudable, but it is not the aim of the majority of those working on legal knowledge based systems. Perhaps semantic understanding would be a good thing, although that could be seen as giving the machine the capability to decide questions most would think better left to appropriately appointed people, but that would not mean that syntactical approaches could contribute nothing. The support may be far less reaching, but it would still be support.
It is interesting that Stamper identifies fat predicates as the particular villains of the piece. Most of the problems are essentially software engineering problems, and yet the use of meaningful identifiers - than which fat predicates are nothing more - is recommended practice in software engineering. If it is good practice with regard to conventional systems why is not also good practice in regard to knowledge based systems? It seems to me that Stamper's real objections are those expressed in [14], the creation of a private language and the deception practised on the user, and the others are mainly beside the point. Which is not to say that knowledge based systems do not pose software engineering problems of their own - see Coenen and Bench-Capon [8] for a full discussion of these - but only that what is good practice with building one sort of computer system is likely to be good practice for other kinds of system too. Recent work by Stamper [16] suggests that he thinks his approach is best for conventional systems also, but that is as yet unsubstantiated.
Let us consider the heart of Stamper's objection - that rule based systems tend to be written with fat predicates, and so involve the use of a private language that deceives the user, and perhaps the programmer too, into thinking that the system ``understands'' a natural language. These points are directly addressed in [1]. Here we may say first that a private language is problematic only if it is logically private (see Wittgenstein [18]): if it is shared, or can be shared, by a community there is no harm - and may be some use - in tailoring a language to specific needs. In building knowledge based systems is is clear that a shared language, with purpose defined meaning for special terms, is what is at issue. Systems are rarely built by an individual: for a system of any reasonable size a team is needed. And, of course, that team will agree on how they will develop the language for the system - the meaningful identifiers (or fat predicates) that the system will use. For this system vocabulary they will define some terms by means of other terms in the system vocabulary (this is essentially what the rules of a knowledge based system do) and others will be left for the user to determine. These latter terms will be chosen with a view to communicating with users using the natural language of the user, employing whatever communication skills the team has. For such a team effort discipline and co-ordination is needed and this inevitably requires the formation of a vocabulary more or less specific to the system.
Moreover, the predicates used in rules will typically be internal to the system. In some cases, such as [13], the interface will construct questions from the system predicates, but more normally the questions will be asked in a specific form stored as ``canned text''. In so far as the user is misled into thinking that the system ``understands'' natural language, the fault is not that of the system, but of the lack of sophistication (and perhaps the lack of training) on the part of the user. The text used to communicate with the user is chosen by the programmer in the same way that an author of a book chooses text to communicate with the reader: the program is only the passive medium through which programmer and user communicate. Any attempt to impute understanding to the system - whether on the part of the user or of the programmer - is a clear mistake, but a mistake made by few knowledge based systems builders. For a full discussion of this point see Bench-Capon and McEnery [4].
To be fair to Stamper, he does agree that systems of the sort he objects to might be employed with advantage by bureaucrats who ``will ascertain the facts and apply the rules to them in order to discover the consequences for the case in hand'' [15]. Although Stamper wishes to contrast such bureaucratic systems with legal systems, it should not be forgotten that most knowledge based systems in the legal domain are directed not at lawyers but at just such bureaucrats, perhaps best typified by the clerks charged with adjudicating social security benefits [3]. Even such bureaucrats will have some discretion, and scope for creativity, but they must work within the constraints not only of the law but also those imposed by their organisation. For such an application, given appropriate training of the users in a proper understanding of the system and their proper role, it may be that the standard expert systems advantages of consistency of decision making, speed of decision making, availability of information, error free deduction and the like, may well outweigh the disadvantages consequent upon any simplification introduced by representation in rules. Moreover, from the perspective of the organisation which employs the bureaucrats and commissions their systems, the structure and conformity imposed by a system is an advantage in itself.
Arguments against rule based systems in law can be highly effective if we consider only simplistic systems used in a simplistic way. A system is always the combination of user and machine; both must have their role defined, and the users must be aware of the role they are expected to play. The problems identified by critics are, for the most part, failures of the user rather than defects of the machine, or of the system design. We can use rules to represent a Molesian position as easily as a Hartian position. We agree that rules cannot be applied in an all or nothing fashion, but assert that few builders of serious legal knowledge based systems would expect them to be. They do require a sophisticated interface, and a sophisticated user, but given this appropriately trained users can make the appropriate commitments as they execute the system. Rule based systems no more provide perfect support than does a calculator for an accountant. But provided the users understand their role in using the tool, they can provide some support, as does the calculator for the accountant. Whilst we may envisage better, more comprehensive and more exciting support systems, we need to recognise that this does not make the support we can provide through rule based systems useless. Moreover, in the use of legal knowledge based systems, we may not wish to shift much of the burden of responsibility from the user to the machine. Rule based systems at least cannot - and typically do not - pretend to more than limited systems designed to support a user who remains responsible for the decision. Thus the responsibility remains firmly where it should be: with a properly empowered, human, decision maker. [an error occurred while processing this directive]