Using Artificial Intelligence in Legal Computer Assisted Instruction



R P Jones, Liverpool John Moores University



Legal Computer Assisted Instruction


As with television, the micro-computer, through the vehicle of Computer Assisted Instruction (CAI), has been heralded as the means to democratise education by delivering the same high- quality educational information to all learners. Unlike television the computer can provide individualised and self paced instruction. The earliest efforts to produce CAI materials can be traced back to 1960, but it was in the early 1970's when the main funded projects of PLATO and TICCIT began in the United States. These projects were funded in order to evaluate the following claims which were being made for CAI.


It provided the potential for self pacing, adaptive and individual tuition.

Programs could provide instant feedback and correction.

During program development tutors carefully articulate questions, answers and responses thereby enhancing their understanding of the subject area.

It provided a means for equalising levels of achievement.

The program could provide links with other technological developments: video, databases and expert systems.

Programs could rigidly control the sequence of topics made available to the user.

Programs allow for lower educational costs in the long term.

Additionally CAI promised the following advantages over its immediate predecessor, workbook programmed learning.


Sophisticated branching, which is impossible with workbooks.

Data collection; difficult with workbooks.

Easier control of cheating.

Random generation of problem situations thus making the programs more interesting and varied for the user.

Easier routing of students according to their previous responses and consequent adaptation to students' requirements.

By the late 1970's disenchantment with CAI had begun to settle over many educators as well as the key funding agencies. It was realised that the bulk of the educational software produced was only of minimal use and some was probably detrimental. Critics were able to claim that CAI trivialises and over simplifies a subject with its simplistic questioning and control strategies and is in consequence an ineffective teaching method. A recent report of the OECD (Lesgold, 1986) was able to state;


"... education stands at a crossroads. The reification of ineffective teaching practices into computer artefacts will not improve education, nor will handing potentially useful information tools to teachers who have not been trained in their use ... a major effort is needed to refine and demonstrate new educational science and technology and to train teachers to use the tools."


Yet CAI is particularly appropriate to legal education for it can give experience in expressing law-related propositions, not just in recognising them. Within the limited legal vocabulary, free form responses can more easily be anticipated by the CAI author thereby reducing his dependence upon multiple choice techniques which are suggestive to the student and can mislead. However the general difficulties encountered in CAI in the early 1970's did much to discourage the development of programs in law. One notable exception was the EDUNET project in the USA which helped to encourage CAI production in a range of disciplines, including law from as early as 1967 (Burris et al, 1979). This project led directly to the formation of the Centre for Computer Assisted Legal Instruction (CCALI) based at the Universities of Minnesota and Harvard (Burris, 1985).


One cause of the deficiencies within the then existing CAI programs can be traced to the software tools used. CAI programs may be produced using a wide range of software tools; the most popular being authoring systems or languages. An authoring system is a complete interactive environment in which the tutor may develop CAI materials. By their nature authoring systems must attempt to cater for the complete education market providing a tool for the teacher at primary, secondary and tertiary level. Facilities provided are generally related to screen design, limited graphics and basic routing. Unfortunately most systems encourage poor CAI method by leading authors into premature program production without encouraging thought as to design or function. Authoring systems often have inadequate facilities for response matching and screen handling and, additionally, use of such systems means a lack of standardisation between faculties with the consequent lack of portability and a lack of co-operative effort. Authoring languages are computer languages specifically oriented towards the development of CAI materials. They allow greater flexibility within the program (but may lack functions standard to a computer language such as Pascal). These languages require time to master and suffer from similar problems of standards and portability as authoring systems. Both authoring systems and languages have failed to evolve with the needs and practices of CAI.


These difficulties led CCALI to develop an alternative authoring system. The Pascal Instructional Language (PIL) establishes predefined operators to remove machine dependency, it also contains a number of procedures that assist in basic areas of string manipulation and screen processing (Burris, 1985). The adoption of a standard language and operating system meant that the full facilities of the computer were available to the author, the programs became portable between machines and costs of development of the system were spread between all the users of Pascal. The system claimed significant benefits including modularity of program design, the use of structured program logic and the availability of data structures. The main difficulty with PIL is the requirement for both an author and a competent Pascal programmer, the system requires the establishment of an organisation that can receive text files from authors and have them programmed. To meet this deficiency CCALI have themselves produced a series of authoring systems that enable the author to program his own CAI materials. This can encourage the development of smaller and more specific programs for use by the individual author 'in house'.


Even with the use of systems such as PIL the criticism of legal CAI continues. Programs are claimed to merely replicate the same skills as taught by traditional legal methods and to simplify and trivialise the law into a series of yes/no or multiple choice scenarios.


Two initiatives in the UK were aimed specifically at this problem of program quality. Jones (Jones, 1987) proposed a methodology to provide a step by step guidance to authors in the preparation of legal CAI materials. Authors were encouraged to consider the form and type of program needed, justification and place in the curriculum. Leith (Leith, 1988) with the aid of an IBM study contract developed the Lexical system as a purpose built legal authoring system for use on IBM PC's and compatibles.


AI Techniques in Legal CAI


Although the standard of programs may improve as tutors begin to gain experience, there is a limit to the sophistication one can attain using the present software. In addition to the drawbacks of authoring systems presented above, traditional CAI programs suffer from the problems all traditional systems face, namely;


an inflexible algorithm - the knowledge about how to teach is bound to the particular algorithm of the program.

data dependency - the algorithm manipulates data supplied by the student rather than the knowledge of the student.

An attempt to overcome these deficiencies was generative CAI. These systems had the capability to generate new problems from the combination of different elements in a database (Uttal et al., 1969). Unfortunately, this adaptivity was limited and often unrelated to the individual student needs.


An alternative line of research is to consider whether the techniques and tools used in Artificial Intelligence can be used to enhance CAI programs. AI research has made available the programming languages LISP and PROLOG, expert system shells and many insights into knowledge representation, inference methods and the learning process.


The interface of Artificial Intelligence and CAI is referred to as Intelligent Computer Assisted Instruction (ICAI) (Kearsley ed., 1987). ICAI brings together work on artificial intelligence, cognitive psychology and educational research. ICAI systems must look at user performance and evaluate it with respect to expert performance, deciding what the user can already do and what could lead the user closer to expert performance. This requires three types of knowledge: knowledge of the domain expert, knowledge of how to recognise the specific capabilities of the user (student model) and knowledge of the course and mechanisms of learning (pedagogical expertise). With such abilities the ICAI system can improve on the richness of feedback and the degree of individualisation offered to the user.


The LEX Project


With a number of legal expert systems under development it is tempting to develop an intelligent tutor out of an embryo expert system. The LEX project provided such an opportunity. It was one of the European based projects investigating legal expert systems from both a professional and teaching perspective. It was a co-operative project between the University of Tuebingen and IBM Germany. The author had the opportunity of working with the LEX project in the development of an intelligent tutoring system (ITS).


It had always been the intention of the LEX research project to investigate both the development of:


a tool for giving legal advice to lawyers and helping them prepare a case (consultation system) and

an intelligent tutoring system for law students.

The consultation system supported a lawyer in analysing case descriptions and advising a client. As far as possible the dialogue of the lawyer with the system should take place in natural language. The lawyer should be in a position to feed the description of a case and the request for a solution into the computer in natural language. The system then attempted to solve the case by means of applying its knowledge. In order to do so it analysed the case description and the request and combined these with statements from its knowledge base in order to form a series of arguments. Wherever certain facts were lacking it was intended that the system would ask for them.


The intelligent tutor should support a user in an interactive learning environment where the system incorporated all knowledge needed to teach the area of application.


The approach of developing an intelligent tutor from an existing system has a number of problems; it imposes the existing expert system methodology on the learner, the student is viewed as a deficient system, it assumes that the result is more important than the means (this is totally inappropriate in law where the result is of importance only to the litigants) and finally the existing expert system fails to provide adequate explanations to enable the user to determine why a particular rule is corrrect or what is the strategy behind the goal structure. This approach necessitated the development in LEX of a number of additional knowledge bases to deal with pedagogical and assessment issues. A similar experience is found in the discipline of medicine where the GUIDON and NEOMYCIN projects have attempted to develop an intelligent tutor out of the MYCIN expert system (Clancey, 1983).


LEX has its area of application in a small number of practically significant offences of German traffic law as the first field of application. At their head is a regulation (paragraph 142, German Penal Code) according to which whoever is involved in a traffic accident and subsequently fails to fulfil certain obligations, for example to remain at the scene of the accident, is punishable. The reasons for this choice were, on the one hand, the great practical significance of this area of the law and, on the other, the complicated structure of the norms which also make generous use of general clauses and indefinite legal terminology. For this reason it was believed that many of the problems that can arise in the formalisation of rules are to be encountered paradigmatically in this field.


Since the LEX system had been designed as a consultant for a lawyer, the task of conversion aimed to make the LEX system become much more active in its actions with the user. LEX uses its expertise to access the knowledge base (faces inwards); the tutoring system must use its expertise to assess the student (face outwards), allowing the student to become the most important part of the system.


Design Decisions


Experience in developing legal CAI programs in the USA (Burris, 1985), UK and at the University of Tuebingen led the project group to isolate the following design decisions in relation to the tutoring program;


the system should be based on a natural language dialogue between user and system and not be restricted (as present legal CAI programs are) to YES/NO, multiple choice answers or keyword matching. LEX offers a limited natural language capacity that would certainly enhance the ITS program.

the ITS should have a student model so that it could adapt its teaching to the individual user's needs.

this student model would enable the system to choose not only appropriate teaching material but also the most appropriate discourse method and CAI technique.

this model would be easily accessible for tutors to evaluate both the system and the user's performance.

the system should have the capability of presenting the same legal material in differing CAI formats.

user understanding would be compared with that of the system but firm conclusions as to user knowledge would not be solely dependent upon this comparison (Clancey, 1983).

problem solving methodology would be judged rather than the result, this was made possible by being able to compare the system's inference process with that of the user.

the system could act as an intelligent front end to a legal database (JURIS) and as a research tool, users should be able to switch between the tutor and consultancy mode.



Implementation required the development of a number of additional rule-bases, designed to assess and route the user and components to hold the additional teaching materials. The system was able to pose case facts to users, compare their response to its own and ask further responses based upon that analysis (Haft et al., 1987).


ICAI in the Academic Environment


The author's experience with the LEX Project provided an opportunity to consider the development of similar systems with Leicester Polytechnic. Computing facilities and expertise dictated that development work should be based in the PROLOG language. Components could therefore be developed individually and alternative reasoning strategies could be used, neither would have been possible if development had been attempted in an expert system shell.


System Architecture


Three different Intelligent Tutorial System (ITS) architectures were studied. These were the ACT principle pioneered by John Anderson and his colleagues at Carnegie-Mellon University, the four component model suggested by Hartley and Sleeman and the five ring model presented by O'Shea et al.



Spectrum to show the three Intelligent Tutorial System architectures studied





exploratory learning environments

 Anderson's ACT


 Hartley & Sleeman's proposal


et al






John Anderson's Advance Computer Tutoring (ACT) Principle (Anderson, 1985)


John Anderson, a psychologist from Carnegie-Mellon University, researched in adaptive control of thought. His work has resulted in an ITS architecture called the (ACT) principle. The principle can support ITSs in such diverse applications as Geometry tutoring and LISP tutoring. It was the ACT principle which was behind a tutoring system for the LISP program language.


The ACT's ITS architecture consists of four main components:


Domain expert: this solves problems in the domain which the student is currently trying to learn. These solutions can then be compared with the student's solutions. The domain expert is also referred to as the 'ideal student' model.

Bug catalogue: this is a library of common mistakes and misconceptions that a student is likely to make.

Tutorial knowledge: this contains the strategies used to teach domain knowledge.

User interface: this controls the interactions between the student and the tutor.

The ACT theory is intended to be used in systems where it can take a monitoring role. For example, in the LISP Tutor a student is required to write a piece of LISP code using a normal editor. Whilst the student is doing this the ACT ITS is in the background monitoring his progress and will only come into action when it recognises a coding or planning error.


In comparison to other ITSs, the ACT tutor incorporates a dogmatic and authoritarian approach. Its main concern is deviation of the student from the ideal student model. With a secondary concern of looking out for common known mistakes that a novice is likely to make. After this it will then try to provide corrective information to steer the student back on to the ideal path.


Anderson's ACT principle puts little emphasis on the student model. There is no representation of the student model as a separate component in the architecture but rather the student modelling is embodied within the overall tutoring philosophy. That is the strategy to assess the student via the ideal student model and the bug catalogue of common misconceptions.


The Hartley-Sleeman Four Component Model


Hartley and Sleeman suggested that an ITS should contain four distinct knowledge bases:


Expert rule-bases - Knowledge of the task domain.

A student model/history of the student's behaviour.

Teaching data - possible teaching operations.

Means-ends-guidance-rules (MEGRs) which uses knowledge held in the student model to provide the student with the next appropriate teaching step.

This method differs from Anderson's ACT principle in that it does not give misconceptions in the domain (the bug catalogue) primary importance, but instead it uses the student model as the primary component. This student model is a model of the student's performance throughout the tutorial and possibly in other tutorials.


Another difference with this proposal and Anderson's is the style of tuition delivered. Whilst the ACT principle stays in the background analysing the student's behaviour and correcting it when needed, Hartley and Sleeman's proposal is more tutoring oriented. The interaction provided by the user interface is controlled by the ITS. The MEGRs use information held about the student in the student model to choose which of the possible teaching operations the ITS should present the student with next.


The Hartley-Sleeman model has been around for over a decade and a half and has stood the test of time. It has been adopted as the general structure for many ITSs and is suitable for many diverse applications, with perhaps some design variations depending on the functional requirements for the system.


This method is perhaps unsuitable when there is a requirement to support the handling of a range of strategies within a given tutorial system.


O'Shea et al Five Ring Model (O'Shea, 1985)


This method is called the five ring model as it has five components in its architecture. These are:


Student history

Student model

Teaching strategy

Teaching generator

Teaching administrator

This method has some similarity to the proposed method by Hartley and Sleeman. Like Hartley and Sleeman's it gives primary importance to the student model. It also gives much importance to the teaching strategy and in that way it is radically different to Anderson's ACT principle. The uses of an ideal student model and bug catalogue, which the ACT principle makes primary use of, are undermined in favour of emphasis on teaching strategy.


This method has more in common with traditional Computer Aided Instruction (CAI) methods in the way it places more emphasis on the teaching strategy. It is also possible to buy a tool kit for building ITSs that use the five ring model in much the same way that you can use authoring systems to develop traditional CAI.


These methods can be viewed as being points on a spectrum (shown above). At one end of the spectrum is exploratory learning environments. These require the student to move around knowledge bases of the knowledge domain and are best suited for teaching abstract and general concepts such as use of analogies and model building.


At the other end of the spectrum is traditional CAI which puts emphasis on teaching strategy rather than representation of the knowledge domain. Between the two extremes are the three methods discussed.


The ACT principle is suited for teaching problem solving in a specific domain where the ITS can just sit in the background and detect and correct the student's understanding when required.


In the middle of the spectrum the Hartley and Sleeman method is more suitable for tuition that has to be tutor driven rather than student driven. But with that aside, it still proves suitable for most ITSs as this is the most desired form of interaction between tutor system and student. Also within its general architecture are all the components required for a truly intelligent tutorial system.


The O'Shea et al five ring model is better for teaching concepts which are more concrete and specific. This also makes it easy to construct a domain independent tool kit for an ITS using this principle and, as I have already mentioned, one is already commercially available.


It was concluded that the Hartley-Sleeman four component model provides suitable general guide lines for the preparation of legal ITS materials and indeed previous work on other related projects (LEX) had already followed these guidelines.


Components required were therefore; expert rule-bases on the area of law to be taught and on the pedagogical expertise on how to teach such areas of law, components to store, assess and to give decisions on progression of a particular student - collectively referred to as the student model.




The project began by developing a number of expert legal rule- bases. Three main legal domains were chosen, the United Kingdom Data Protection Act 1984, Unfair Contract Terms 1977 and the Housing Act 1985. These domains were chosen because the School already had traditional CAI programs in these domains.


The development of the rule-bases followed work on formalisation of statute law conducted at Imperial College (Sergot et al., 1986) and at Leicester Polytechnic (Routon, 1989). They represent an attempt to build what are referred to as "Shallow" models of the law.


More recently effort has gone into the hierarchical formalisation of statutory material. Traditional representations treat the rules contained in the statute as if they stand on par with each other, but typically a statute is organised hierarchically. There may be parts, within parts will be sections and within sections sub-sections. Further sections may not contain rules about the legal domain but rules about rules. The proposal is to raise these rules to a higher (level) from whence they may operate more realistically. For example in the Data Protection Act 1984;


A data user shall be treated as being registered under Section 5 if Section 7 (6) (a) and (b) but subject to the exception in Section 7 (8) a or Section 7 (8) b.


A traditional representation of Section 7(6) would be in logical form;


registered if a and b and (not S7(8)a or S7(8)b).


How much clearer to raise the exception in Section 7 (8) to a meta level and have a representation


registered if a and b and ss8.

ss8 if S7(8)a or S7(8)b.


The rule in ss8 now represents the exception and now is given the status of a meta level rule. It reflects more accurately the hierarchical nature of the Act and allows easier updating should amendment be made to the exception (Routon, 1989).


Apart from faithful representations of the Act, attempts have been made to add additional rule-bases that contain the lawyer's heuristic knowledge of the Act. For example in the Data Protection Act 1984 much of the Act revolves around the definition of Data, (Section 1). In PROLOG the code of the section would be;


"Data" means information recorded in a form in which it can be processed by equipment operating automatically in response to instructions given for that purpose.







The rule has been enhanced with heuristic knowledge by the use of so called 'statutory predictions' (Susskind, 1987) to allow for expected interpretation of the above section. The heuristic knowledge related to the requirement that data must be recorded in a form capable of being processed could be that data held on a disk is so held. This is represented in an additional knowledge base as





It was then possible to further add probability factors to give


recorded_in_processable_form( Data),

[held_on_floppy_disk( Data),prob(0.9)


It is essential to maintain these predictions separate from the main rule-base as they do not represent the formal representation of the statute. These additional rule-bases do provide an interesting experimental area for students to work in.


Within the tutorial system a static rule-base would find little use, it being much too limited and restrictive to be useful. Ideally the rule-base should have an adaptive representation of the law and should be able to return to the assessment component not only the goal (decision) but also its reasoning path. The technique of using meta predicates is again being used here. Meta interpreters are interpreters for the language written in the language itself and in consequence can modify the reasoning strategy of the PROLOG system thereby providing an adaptive representation of the knowledge. Further they can look down and maintain a trail of the reasoning process in the rule-base. This trail can then be returned to the assessment component and compared with the student's reasoning trail.


Development of the Pedagogical Rule-base


This is an attempt to represent how to teach the particular subject. Unfortunately there is no correct teaching strategy so the system merely attempts to represent an overall hierarchy of teachable legal skills/topics and their inter-related dependencies based upon the formalisation by Ashley and Rissland (Ashley and Rissland, 1985). At present the rule-base is in linear form with alternate pathways for student progress. In an individual teaching module an and/or graph is devised using a flow pattern through the subject matter. For example, if the student is presented with a new set of case facts and is requested to isolate the main legal issues, the student error may be either a failure to isolate the relevant facts or a misunderstanding of the legal concept. At present the pedagogical rule-base can deal with the first type of error and provides a mechanism for the student to be directed to remedial facilities.


Student Model


The model comprises comparison, storage and assessment elements. Firstly the student response is compared with the response from the rule-base. This information then forms part of the information stored in the user profile. The information may be on both explicit, i.e. use of function keys, teaching modules completed and specific responses given, or implicit, such as the degree of congruence between the students response and that of the rule-base. This information is used to assess the student understanding. There are two main assessment techniques used within the project. The first involves a comparison of the user with a perfect model devised by the tutor; deviations from the perfect model are "bugs" that need to be eliminated (Brown and Burton, 1978). The second is an attempt to build comprehensive profiles of student's understanding (Sleeman, 1982).


In the comparison technique the program makes comparisons at an individual question level. These are then extended to compare a group of responses. The group of responses and subsequent routing represent a pathway through the pedagogical rule-base which are then mapped on to the ideal tutor's path. For example, a tutor may devise a series of questions on the particular topic. For each question there may be several variants, hard, default easy, etc., as well as remedials and help facilities. The system will route the user though the system depending upon responses given so that a student showing an error on question 1 could be routed to the easier variant of question 2 following a remedial on question 1. The student easily answers the question 2 and so is routed to question 7. The path from question 1 to remedial to question 2 easy and question 7 so on represents that student's path.


To provide additional assessment information the system attempts to build a comprehensive model of the individual student. The system reviews the information retained on the user, essentially the results of mapping the student's progress on the pedagogical rule-base. Tutors are then encouraged to develop rules about understanding and misunderstanding based upon the particular student path, domain knowledge, difficulty levels and student's assumed knowledge. Therefore a tutor is encouraged to articulate his view of the understanding of our student struggling with question 1 but answering the easy variant of question 2 without difficulty.


The results of the assessment techniques are then passed to a set of rules referred to as means-ends-guidance-rules (MEGR). These rules make decisions on routing i.e. the content and form of the next teaching module. The rules are an attempt to embody the tutor's heuristic knowledge of how to judge the user's progress and how to decide what instructional material the system should present and how and when it should be presented. A simple algorithm is then used to take this information and search the pedagogical rule-base for the next appropriate topic. It is possible to enhance the algorithm by adding values to the links in the pathways specifying important and unimportant links. The algorithm could then choose only the important links for a student who is finding the subject difficult, but making the more able student consider all the links.


Implementations and Further Work


The project team concentrated upon the development of a comprehensive architecture and development of a number of usable rule-bases. Simplified student models were formulated allowing routes through a small pedagogical rule-base. The rule-bases have already found important teaching uses with students being encouraged to use the rule-bases as an exploratory learning environment and as additions to the existing traditional CAI programs. Originally the rule-bases were developed using IBMPC compatibles but now development is being concentrated onto work stations (Apollo) that have been found to have far more useful implementations of PROLOG.




Anderson, J.R., Boyle, C.F. and Yost, G. (1985) "Intelligent Tutoring Systems", Science, 228, 1985, pp.456-4562.


Ashley, K. and Rissland, E. (1985) "Towards Modelling Legal Argument". In Martino, A. (ed). Logica, Informatica, Diritto: Proceedings of the Second Conference on Legal Informatics, Florence, Italy (September 1985), pp.381-397.


Burris, R. (1985) "Critical Features of Microcomputer-Based Exercises for Effective Teaching and Instruction of Law". IBM ACIS Proceedings.


Burris, R., Keeton, R.E., Landis, C.P. and Park, R. (1979) "Teaching Law with Computers". EDUCOM.


Clancey, W., (1983) "The Epistemology of a Rule-Bases Expert System - A Framework for Explanation", Artificial Intelligence 18, 1983, page 215.


Haft F., Jones R.P. and Wetter T. (1987) "A Natural Language Based Legal Expert System for Consultation and Tutoring - The LEX project". 1st International Conference on A.I. and Law, ACM Press.


Hartley, J.R. and Sleeman, D.H. (1973) "Towards Intelligent Teaching Systems", Int. J. of Man-Machine Studies. 1973.


Heines, J. and O'Shea, T. (1985) "The design of a rule-based CAI tutorial''. Int. J. Man-Machine Studies, 23, pp.1-25.


Jones R.P. (1988) "Artificial Intelligence and the LEX project". Internal IBM report.


Kearsley, G.P. (ed.) (1987) "Artificial Intelligence and Instruction, Application and Methods". Addison-Wesley, Massachusetts.


Leith P. (1988) "Methodologies in Legal CAL - Designing LEXICAL". 4th International Congress, Informatica Et Regolamentazioni Giuridche.


Lesgold, A. (1986) "Information Technologies and Basic Learning". General Report. Centre for Educational Research and Innovation. OECD.


Routon, T. (1989) "Hierarchically Organised Formalisations", 2nd International Conference on A.I. and Law, ACM Press.


Sergot, M., Cory, T., Hammond, P., Kowalski, R., Kriwaczek, F. and Sadri, F. (1986) "Formalisation of the British Nationality Act". 2nd Yearbook of Law Computers and Technology.


Sleeman, D. and Brown, J.S. (1982) "Intelligent Tutoring Systems". Academic Press.


Susskind, R. (1987) "Expert Systems in Law". Oxford University Press.


Uttal, W.R., Rogers, M. Hieronymous, R. and Paisch, T. (1969) "Generative Computer-assisted instruction in Analytic Geometry". Newburyport, MA:Entelek, Inc.


Published in the Law Technology Journal: Vol 2, No 1