File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/98/w98-1409_metho.xml
Size: 26,020 bytes
Last Modified: 2025-10-06 14:15:12
<?xml version="1.0" standalone="yes"?> <Paper uid="W98-1409"> <Title>A NEW APPROACH TO EXPERT SYSTEM EXPLANATIONS</Title> <Section position="1" start_page="0" end_page="0" type="metho"> <SectionTitle> A NEW APPROACH TO EXPERT SYSTEM EXPLANATIONS </SectionTitle> <Paragraph position="0"> * CoGenTex. Inc.</Paragraph> <Paragraph position="1"> t Department*of Computer Science, * Columbia University</Paragraph> </Section> <Section position="2" start_page="0" end_page="78" type="metho"> <SectionTitle> 1 Expert System Explanation </SectionTitle> <Paragraph position="0"> Expert systems were one of the first applications to emerge from initial research in artificial intelligence, and the explanation of expert system reasoning was one of the first applications of natural language generation3 This is because the need for explanations is obvious, and generation from a knowledge-based application such as reasoning should be relatively straightforward. However, while explanation has been universally acknowledged as a desirable functionality in expert systems, natural language generation has not taken a central place in contemporary expert system development. For example, a popular.text book about expert systems such as (Giarratano and Riley, 1994) stresses twice in the introduction the importance of explanation, but provides no further mention of explanation in the remaining 600 pages. (The book is based on the popular CLIPS framework.) In this paper, we present a new approach to enhancing an expert system with an explanation facility. The approach comprises both software components and a methodology for assembling the components. The methodology is minimally intrusive into existing expert system development practice.</Paragraph> <Paragraph position="1"> This paper is structured as follows. In Section* 2, we discuss previous work and identify shortcomings. We present our analysis of knowledge * types in Section 3. Section 4 presents the *Security Assistant and its explanation facility. Finally, we sketch a general methodology for explainable expert system engineering in Section 5.</Paragraph> </Section> <Section position="3" start_page="78" end_page="79" type="metho"> <SectionTitle> 2 Previous Work </SectionTitle> <Paragraph position="0"> A very important early result (based on experiences with explanation 2 in systems such as MYCIN (Shortliffe, 1976)) was the finding that &quot;reasoning strategies employed by programs do not form a good basis for understandable explanations&quot; (Moore, 1994, p.31). Specifically, simply paraphrasing the chain of reasoning of the expertsystem doesnot let a human user easily understand that reasoning.</Paragraph> <Paragraph position="1"> Two separate approaches have been proposed to address this problem: * In the Explainable Expert System (EES) approach (Swartout et al., 1991; Swartout and Moore, 1993), the knowledge representation used by the expert system is enriched to include explicit &quot;strategic&quot; knowledge, i.e., knowledge about how to reason, and domain-specific knowledge. From this knowledge, the rules used by the expert system are compiled, and this knowledge is also used to provide more abstract explanations of the system's reasoning.</Paragraph> <Paragraph position="2"> * In the Reconstructive Explainer (Rex) approach (Wick, 1993), the expert system is unchanged, but after it has performed its reasoning, a causal chain for explanation is constructed from the input data to the conclusion reached previously by the expert system as a separate process. The work of (Tanner et al., 1993) can also be seen as falling in this paradigm, since a separate representation of knowledge (the &quot;functional representation&quot;) is used only for explanation, and the explanation must be specially derived from this.</Paragraph> <Paragraph position="3"> These approaches have in common a preoccupation with a categorization of knowledge used in the system into different types. EES concentrates on an abstract representation of strategic knowledge (how does a particular action of the system relate to the overall goal?) and on the representation of design rationale (why are actions reasonable in view of domain goals?). In addition, there is terminological domain knowledge (definitions of terms). Rex and related approaches have a representation of domain knowledge, along with domain rule knowledge (mainly causality), which is completely separate from that used by the expert system itself. This knowledge is used to derive an &quot;explanation path&quot; through the domain knowledge representation.</Paragraph> <Paragraph position="4"> There are problems with both approaches. EES has not proven to be a fully satisfactory solution to the problem of expert system explanation. The problem is that the writers of expert systems have not been too quick or too eager to adopt frameworks such as EES. The requirement for a more abstract representation of knowledge (from which the actual expert system rules are compiled) that EES imposes may be considered onerous by the expert system developer, appearing unmotivated from the point of view of the core functionality of the system, namely reasoning (as opposed to explanation). Presumably, it is difficult for one and the same person to be a domain expert and a expert on communication in the domain.</Paragraph> <Paragraph position="5"> In the Rex approach, the obvious problem is that in order to generate an explanation, additional reasoning must be performed which in some sense is very similar to that done by the expert 2We do not consider explanation generation from data bases (for example, (McKeown, i985; Paris, 1988; Lester and Porter, 1997)) to be the same problem as expert system reasoning explanation (even though we may use some similar techniques). In data base explanations, the knowledge to be communicated is static and its representation is given a priori as part of the statement of the generation problem. In expert system explanations, the knowledge to be explained is generated dynamically, and the proper representation for this knowledge is part of the solution to the problem of expert system exp:anation, not its statement.</Paragraph> <Paragraph position="6"> system itself (e.g., finding causal chains). This is redundant, and does not result in a clear separation between reasoning and explanation. While Wick (1993) argues against such a separation on philosophical grounds, practical constraints suggest, as indicated above, that the domain expert responsible for implementing the reasoning system shouldnot also be responsibl e for implementing the explanation capability, and that the communication engineer (responsible for implementing the explanation facility) should not need to replicate domain reasoning.</Paragraph> <Paragraph position="7"> In this paper, we present a new approach (architecture and methodology) to expert system explanation which does not require the expert system writer to take into account the needs of the explanation while writing the rules. At the same time, we avoid the necessity of having a separate domain reasoning component for the explanation generation. Instead, the expert system is largely considered a stand-alone application, onto which explanation is added. However, this is done by having a communication enginee r design a second knowledge representation (separate from the expert System's domain * knowledge representation) specifically for the purpose of communicating explanation s. This representation is instantiated by the expert system as it reasons, not by a separate module after reasoning has occurred. Thus, no separat e reasoning facility is needed.</Paragraph> </Section> <Section position="4" start_page="79" end_page="80" type="metho"> <SectionTitle> 3 Types of Knowledge in Explanation* </SectionTitle> <Paragraph position="0"> We follow previous work in distinguishing different types of knowledge. However, use operational criteria: we classify knowledge by what it is used for and who is responsible for its engineering, not by its structure or contents. We briefly present our classification here and illustrate it on an :example in the following section.</Paragraph> <Paragraph position="1"> * Reasoning domain knowledge (RDK). This is knowledge about the domain needed to perform the reasoning. Typically, it includes rules, terminological knowledge, and *instance knowledge.: It is encoded by the domain expert in the expert system proper.</Paragraph> <Paragraph position="2"> * Communication domain knowledge (CDK). This is knowledge about the domain which is needed for communication about the domain. It typically is a different &quot;view&quot; on the * domain knowledge than RDK, and may include additional information not needed for the reasoning itself. It is encoded by the communication engineer in the explanation facility. * Domain communication knowledge (DCK). This is knowledge about how to communi null cate in the domain. DCK typically includes strategies for explanation in the given domain, and knowledge how to describe the entities of the domain. It is encoded by the communication engineer in the explanation facility.</Paragraph> <Paragraph position="3"> The distinctions may at first seem overly fine-grained. However, each type of knowledge is distinguished from the other types. CDK is domain knowledge, but it is'domain knowledge that is needed only for communication, not for reasoning (as is RDK). RDK and CDK of course overlap, but they are not identical. This is in fact the lesson from much previous work in expert system explanation, for example the work of Paris et al. (1988)contrasting &quot;the line of reasoning&quot; and &quot;the line of explanation&quot;, and the claim of Swartout et al. (1991) that the domain representation must be augmented with additional knowledge about the domain and about reasoning in the domain. Many researchers have identified the need for packaging domain knowledge differently for</Paragraph> <Paragraph position="5"> communication. For example, the &quot;views&quot; of Lester and Porter *(1997) can be seen as a form of CDK, though they are not a declarative representation. What is new in our work, however, is the proposal that CDK should be represented explicitly in a distinct representation from the domain knowledge. 3 CDK is different from DCK in that CDK is knowledge about the domain as it is needed for communication, but DCK is knowledge about how to communicate in that domain (and in a specific communicative setting characterized by factors as diverse as communication type or genre,* hearer needs, communication medium, or cultural context). Therefore, for expert system explanation applications, CDK is conceptual knowledge (what conceptual content must be conveyed to the user to explain system reasoning effectively?), while DCK is knowledge about language use (how do we use linguistic acts to explain system reasoning effectively?). 4 DCK may be expressed in communicative plan operators which achieve goals related to the hearer's cognitive state, while CDK would never include plan operators related to the hearer's cognitive state because the hearer is not part of the domain of the expert system.</Paragraph> </Section> <Section position="5" start_page="80" end_page="84" type="metho"> <SectionTitle> 4 The Security Assistant </SectionTitle> <Paragraph position="0"> The Security Assistant or SA (Webber et al:, 1998) is part of the DesignExpert tool (Ehrhart et al., 1998), which helps software engineers analyze system-wide(or &quot;non-functional&quot;) requirements such as security, fault-tolerance, and human-computer interaction. The SA aids a software engineer in * Choosing security measures to protect valuable system assets (e.g. important data) against likely threats (e.g. disclosure or corruption). In the following three subsections~ we discuss how the three types of knowledge discussed in the previous section - RDK, CDK, and DCK, are represented and used in the SA.</Paragraph> <Section position="1" start_page="80" end_page="81" type="sub_section"> <SectionTitle> 4.1 The Expert System: Reasoning Domain Knowledge </SectionTitle> <Paragraph position="0"> The SA first queries the user for information about entities of the system to be analyzed, such as system assets, system components, and system sites, and the damage types that are of concern for these entities. Additional damage types are inferred for each important asset *of a system (e.g. data can suffer from disclosure or corruption). Th e system then reasons about possible defenses that SWhile CDK is closely related to content selection, it should not be equated with content selection, which is often seen as the first task in text planning (followed by content ordering). Content selection is entirely oriented towards the anticipated act of communication, and hence defined by its parameters: what the communicative goal is, What the medium is, who the hearer is, and other constraints (length of communication, and so on). CDK is knowledge needed for content selection, but excludes all choices that depend on knowledge of the intended act of communication.</Paragraph> <Paragraph position="1"> For example, CDK might *include relative salience between domain objects, but does not include information about how salient an object needs to be in order to interest the hearer. However, we admit that the distinction may be blurred, especially in implementations.</Paragraph> <Paragraph position="2"> 4While DCK is domain- and genre-specific knowledge about how to communicate, we do not claim that the same type of reasoner with different domains (say, an expert system for car repair and an expert system for helicopter repair) would necessarily require different DCK. However, the type of expert system in the two cases might be very similar, and it is this fact that would allow us to re-use the same DCK. Thus, from the point of view Of the explanation system, the &quot;domain&quot; is not the domain of the expert system, but the type of the expert system. For a discussion of the distinction between domain communication knowledge and domain-independent communication knowledge, and for an argument in favor of the need for DCK, see (Kittredge et al., 1991).</Paragraph> <Paragraph position="3"> directly prevent these damage types. If no single complete defenses can be found, the SA determines all attack methods which can cause the damage, and then deduces all enabling conditions for such attacks. It subsequently determines defenses that prevent such enabling situations. This reasoning can then be iterated. The result of the SA's reasoning is a list undefended assets and, for each such asset, a a list of recommended defenses.</Paragraph> <Paragraph position="4"> Fo r example, suppose direct modification by a malicious user has been identified as a possible * damage to a system asset (say, a database), and that the SA can determine no immediate defense against direct modification (for example, it is impossible to disable all editors). Modification is 0n!y: possible after the malicious user has gained illegal access to the system. In this case, we would say that illegal access enables modification. A defense against illegal access is therefore also a defense * against modification. .. .... &quot; * The knowledge needed fo r reasoning is expressed in the usual manner as production *rules which, if the conditions are met, assert the existence of new *damages, defenses, enabling conditions, and so On.</Paragraph> <Paragraph position="6"/> </Section> <Section position="2" start_page="81" end_page="83" type="sub_section"> <SectionTitle> 4.2 ' The *Content Representation Graph: Communication Domain Knowledge </SectionTitle> <Paragraph position="0"> In SA, the starting point for expressing CDK is a domain model of the type that is used in object-oriented design and analysis. Our domain model (Figure 1) represents security domain concepts, various attributes and Concept relationships, as they are used in explanation. The domain model was created by analyzing how a domain expert would explain the reasoning of the SA to nonexperts, using a small corpus of explanations. Each of the boxes in the model stands for a concept in th e security domain, and inside these *boxes are attributes associated with the concept. Arrow- null :ii tipped edges represent relations between concepts in the domain model Database, triangle-tipped edges represent is-a relations and diamond-tipped edges are has-a relations. Some: examples: * Defense objects have id (name) and cost attributes; * Damage objects have id, severity and type attributes; * prevent is a relation that holds between a Defense instance and a Damage instance; * Site, Asset and System component are different sub-classes of ProtectedObject; * A System consists of one or more system components. : ......</Paragraph> <Paragraph position="1"> The CDK expressed in this domain model has no role in the expert syste m reasoning. In *fact, during the reasoning process, the expert system models the relations as primary objects, and the concepts of our domain model are merely slots of the relations in the expert system. As a result, the relations typically are not binary, but n-ary. In contrast, the domain model contains only binary relations. This reflects, we claim, the difference between the optimal way of representing knowledge for machine reasoning, and the way in which humans model the world (which is what the CDK domain model captures). As an example of the difference * in relations, the relation that corresponds to the CDK domain model's prevent relation between Defense and Damage corresponds to, in the reasoning component, a.quintary relation between the defense, the location of the defense, the damage it prevents, the locations at which it prevents the damage, and the damages that negate the defense. Another example i s the likely_attack_method relation used in RDK (and its structural. clone, the possible_attack_method relation) of the reasoning component, which is a ternary relation between an asset, a location, and an attack method. As can be seen from the domain model diagram, this relation is not modeled in CDK at all.</Paragraph> <Paragraph position="2"> Knowledge about domain Concepts and relationships is not sufficient for generating an expressive explanation. Additional CDK is required in order to select and organize content according to ex-Planation type and length limitation. The domain model is therefore augmented with the following information.</Paragraph> <Paragraph position="3"> Importance level, which is defined for every relation and attribute. This information about relative importance of attributes and *relations enables us to produce explanations of different length. For example, the relation prevent between Defense and Damage has higher importance level than the relation have between SystemComponent and Mission. * In our domain model, we use a two-valued scale.</Paragraph> <Paragraph position="4"> A key attribute for each concept which is required in instances of the concept and which identifies an instance of the concept. For example, id is a key attributes for Site but HostilityCharacteristiC/ is not a key attribute.</Paragraph> <Paragraph position="5"> Mutual dependencies among concept relations and attributes. This information covers cases in which a particular relation or attribute can be presented only if some other relations or attributes are presented. For example, the relation prevent between Defense and Attack should be included only if the relation cause between Attack and Damage is included as well. * Order among relations and order among attributes of the same concept, namely in what order should relations of the concept be presented, e.g. for concept damage arc goal is ordered before arc enable.</Paragraph> <Paragraph position="6"> * Meta-relations between relations of the same concept. For example, there is a meta-relation purpose between (Defense prevent damage) and (Defense is associated with ProtectedObject). To derive the CDK needed for a specific explanation task, the augmented domain model is instantiated. While the reasoning component performs the reasoning proper, it also populates the concepts of the augmented domain model*with instances. The result is an instantiated model that contains concept instances, and attributes bound to their values. We called this instantiated model the &quot;conten~ representation graph&quot; (CRG) of the explanation. The CRG contains all the information that is needed for the generation of the explanation text. An example of a CRG is shown in</Paragraph> </Section> <Section position="3" start_page="83" end_page="84" type="sub_section"> <SectionTitle> 4.3 Text Planning: Domain Communication Planning </SectionTitle> <Paragraph position="0"> As already mentioned , the CRG does not determine the form of the text, but only restricts its * content. We implemented two different text types that build different text plans (and hence different texts) from the same CRG. The first type is intended to be used in an interactive setting, where the user can request more information if he or she is interested in it, by clicking on hyperlinks. An example i s shown in Figure 3, where hyperlinks are shown by underlining.</Paragraph> <Paragraph position="2"> However, for the DesignExpert application it is also necessary to generate explanations that are free of hypertext for inclusion in printed documents. * These texts must include the entire explanation at a level of detail appropriate for the kind of expected reader. An example is shown in Figure 4. In order to create hyperlink-free explanation text, the CRG must be traversed according to constraints at *every nodes: which attributes to use to describe the object, which relations of this object with other *object must be presented in explanation, in what order to present the relations and what are -meta-relationship between them. The planner processes every graph edg e according to specified order, and structures resulting phrases with resPect to meta-relations.</Paragraph> <Paragraph position="3"> For both text types, we used a text planner with a declarative formalism for text plan specification ' which directly expresses the DCK (Lavoie *and Rambow, i998). Other representations of domain-specific text planning knowledge could also have been used, and we omit details of the formalism here.</Paragraph> </Section> </Section> <Section position="6" start_page="84" end_page="84" type="metho"> <SectionTitle> 5 Methodology </SectionTitle> <Paragraph position="0"> We propose the following methodology for developing an explainable expertsystem. We assume three roles, that of the domain expert (where &quot;domain&quot; refers to the domain of the expert system, such as computer security or infectious diseases), knowledge engineer (a specialist in eliciting and representing domain models, specifically in the form of expert systems), and a communication engineer (a specialist in analyzing and representing the knowledge needed for efficient communication).</Paragraph> <Paragraph position="1"> The domain expert writes several instances of (textual) explanations of the type needed for * the application in question, based on scenarios that the expert system can handle.</Paragraph> <Paragraph position="2"> The communication engineer analyzes the corpus of hand-written explanations along * two lines: * The domain concepts that are reported in the text are analyzed and *recorded using an object-oriented modeling technique, perhaps augmented by more expressive* constructs, such as meta-relations (relations between relations). This Structure is the content representation graph, represents the CDK (both* the augmented domain model and its &quot; instances).</Paragraph> <Paragraph position="3"> * . The structure of the text is recorded using some notation for discourse structure suitable for the text planner being used in the text generator (say, RST (Mann and Thompson, 1987)).</Paragraph> <Paragraph position="4"> 4. Using the CDK representation, the communication engineer consults With the domain expert and the knowledge* engineer to define a mapping from the domain representation used by the.expert system to the CDK domain model devised by the communication engineer. The communication domain*knowledge representation may be modified as a result.</Paragraph> <Paragraph position="5"> 5: The knowledge engineer adds rules to the expert System that instantiate the communication domain knowledge representation with instances generated during the reasoning process. 6. The communication engineer designs a text planner that draws on the knowledge in the CDK representation and produces text. This task involves the creation of an explicit representation of DCK for the domain and task (and genre)at hand.</Paragraph> <Paragraph position="6"> * The resulting system is modular in terms of software modules. The expert system is preserved as a Stand-alone module (though its rule base has been somewhat extended to identify communication domain knowledge), as is the text planner. Both modules can be off-the-shelf components. Only the CDK representation is designed in a task-specific manner, but of course standard *knowledge representation tools, object-oriented data bases, or the like can be used.</Paragraph> <Paragraph position="7"> In addition, the methodology is modular in terms of tasks and expertise. The domain expert and * knowledge engineer do not need to learn about communication , and the communication engineer does not: need to understand the workings of the expert system (though she does need to understand the domain well enough in order to design communication strategies for it, of course).</Paragraph> </Section> class="xml-element"></Paper>