File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/96/c96-1059_metho.xml
Size: 16,596 bytes
Last Modified: 2025-10-06 14:14:13
<?xml version="1.0" standalone="yes"?> <Paper uid="C96-1059"> <Title>A Corpus Study of Negative Imperatives in Natural Language Instructions*</Title> <Section position="4" start_page="346" end_page="346" type="metho"> <SectionTitle> 3 A Priori Hypotheses </SectionTitle> <Paragraph position="0"> Di Eugenio (1993) lint forward the following hypothesis concerning the realization of preventative expressions. In this discussion, S refers to the instructor (speaker / writer) who is referred to with feminine pronouns, and H to the agent (hearer / reader), referred to with masculine t)ronouns: * DONT imperatives. A DONT imperative is used when S expects H to be aware of a certain choice point, but to be likely to choose the wrong alternative among many possibly infinite ones, as in: (4) Dust-mop or vacuum your parquet floor as you would carpeting. Do not scrub or wet-mop the parquet.</Paragraph> <Paragraph position="1"> Here, H is aware of the choice of various cleaning methods, but m W choose an inappropriate one (i.e., scrul)bing or wet-mopping). * Neg-TC imperatives. In general, neg-TC imperatives are used when S expects H to overlook a certain choice point; such choice point may be identified through a possible side effect that the wrong choice will cause.</Paragraph> <Paragraph position="2"> It may, for example, be used when H might execute an action in an undesirable way. Consider: null (5) To make a piercing cut, first drill a hole in the waste stock on the interior of the pattern. If you want to save the waste stock for later use, drill the hole near a corner in the pattern. Be careful not to drill through the pattern line.</Paragraph> <Paragraph position="3"> Here, H has some choices as regards the exact position where to drill, so S constrains him by saying Be careful not to drill through the pattern line.</Paragraph> <Paragraph position="4"> So tile ilypothesis is that H's awareness of the presence of a certain choice point in executing a set of instructions affects the choice of one preventative expression over another. This hypothesis, however, was based on a small corpus and on intuitions. In this paper we present a more systematic analysis.</Paragraph> </Section> <Section position="5" start_page="346" end_page="348" type="metho"> <SectionTitle> 4 Corpus and coding </SectionTitle> <Paragraph position="0"> Our interest is in finding correlations between features related to the function of a preventative expression, and those related to the form of that expression. Functional features are the semantic features of the message being expressed and the pragmatic features of the context of communication.</Paragraph> <Paragraph position="1"> The h)rm feature is the grammatical structure of the expression. In this section we will start with a discussion of our corpus, and then detail the function and form features that we have coded. We will conclude with a discussion of the inter-coder reliability of our coding.</Paragraph> <Section position="1" start_page="346" end_page="347" type="sub_section"> <SectionTitle> 4.1 Corpus </SectionTitle> <Paragraph position="0"> The raw instructional corpus t}'om which we take all the examples we have coded has been collected opportunistically off the internet and from other sources. It is at)l)roximately 4 MB in size and is made entirely of written English instructional texts. The corpus includes a collection of recipes (1.7 MB), two comt)lete do-it-yourself nmnuals (RD, 1991; McGowan and R. DuBern, 1991) (1.2 MB) l , a set of comt)utcr games instructions, the Sun Open-windows on-line instructions, and a collection of administrative application forms. As a collection, these texts are the result of a variety of authors working in a variety of instructional contexts. null We broke the corpus texts into expressions using a simple sentence breaking algorithm and then collected the negative imperatives by probing for expressions that contain the grammatical forms we were interested in (e.g., expressions containing phrases such as &quot;don't&quot; and &quot;take care&quot;). The first row in Table 1 shows the frequency of occurrence for each of the grammatical forms we probed for. These grammatical forms, 1175 occurrences in all, constitute 2.5% of the expressions in the full corpus. We then filtered the results of this probe in two ways: 1. When the probe returned more than 100 examples for a grammatical form, we randomly selected around 100 of those returned. We took all the examples for those forms that returned fewer than 100 examples. The number of examples that resulted is shown in row 2 of Table 1 (labelled &quot;raw sample&quot;).</Paragraph> <Paragraph position="1"> 2. We removed those examples that, although they contained the desired lexical string, did not constitute negative imperatives. This pruning was done when the example was not an imperative (e.g., &quot;If you don't see the Mail Tool window ... &quot;) and when the example was not negative (e.g., &quot;Make sure to lock the bit tightly in the collar.&quot;). The number of examples which resulted is shown in row 3 of Table 1 (labelled &quot;final coding&quot;). Note that the majority of the &quot;make sure&quot; examples were removed here because they were ensurative. null As shown in Table 1, the final corpus sample is made up of 239 examples, all of which have been coded for the features to be discussed in the next two sections.</Paragraph> </Section> <Section position="2" start_page="347" end_page="347" type="sub_section"> <SectionTitle> 4.2 Form </SectionTitle> <Paragraph position="0"> Because of its syntactic nature, the form feature coding was very robust. The possible feature values were: DONT -- for the do not and don't forms discussed above; and neg-TC -- for take care, make sure, ensure, be careful, be sure, be certain expressions with negative arguments.</Paragraph> </Section> <Section position="3" start_page="347" end_page="348" type="sub_section"> <SectionTitle> 4.3 Function Features </SectionTitle> <Paragraph position="0"> The design of semantic/pragmatic features usually requires a series of iterations and modifications. We will discuss our schema, explaining the reasons behind our choices when necessary. We coded for two function features: INTENTIONAL-ITY and AWARENESS, which we will illustrate in turn using ~ to refer to the negated action. The conception of these features was inspired by the hypothesis put forward in Section 3, as we will briefly discuss below.</Paragraph> <Paragraph position="1"> This feature encodes whether the agent consciously adopts the intention of performing a. We settled on two values, CON(scious) and UNC (onscious). As the names of these values may be slightly misleading, we discuss them in detail here: CON is used to code situations where S expects H to intend to perform ~. This often happens when S expects H to be aware that ~ is an alternative to the ~ H should perform, and to consider them equivalent, while S knows that this is not the case. Consider Ex. (4) above.</Paragraph> <Paragraph position="2"> If the negative imperative Do not scrub or wet-mop the parquet were not included, the agent might have chosen to scrub or wet-mop because these actions may result in deeper cleaning, and because he was unaware of the bad consequences.</Paragraph> <Paragraph position="3"> UNC is perhaps a less felicitous name because we certainly don't mean that the agent may perform actions while being unconscious! Rather, we mean that the agent doesn't realise that there is a choice point It is used in two situations: when c~ is totally accidental, as in: (6) Be careful not to burn the garlic.</Paragraph> <Paragraph position="4"> In the domain of cooking, no agent would consciously burn the garlic. Alternatively, an example is coded as UNC when a has to be intentionally planned for, but the agent may not take into account a crucial feature of a, as in: (7) Don't charge - or store a tool where the temperature is below 40 degrees F or above 105 degrees.</Paragraph> <Paragraph position="5"> While clearly the agent will have to intend to perform charging or storing a tool, he is likely to overlook, at least in S's conception, that temperature could have a negative impact on the results of such actions.</Paragraph> <Paragraph position="6"> This binary feature captures whether the agent is AWare or UNAWare that the consequences of are bad. These features are detailed now: UNAW is used when H is perceived to be unaware that a is bad. For example, Exampie (7) (&quot;Don't charge or store a tool where the temt)erature is below 40 degrees F oz' above 105 degrees&quot;) is coded as UNAW because it is unlikely that tile reader will know about this restriction; AW is used when It is aware that a is bad. Example (6) (&quot;Be careful not to burn the garlic&quot;) is coded as AW t)e(:ause the reader is well aware that burning things when cooking them is bad.</Paragraph> </Section> <Section position="4" start_page="348" end_page="348" type="sub_section"> <SectionTitle> 4.4 Inter-coder reliability </SectionTitle> <Paragraph position="0"> Each author independently coded each of the features for all tile examples in tile sample. The percentage agreement is 76.1% for intentionality and 92.5% for awareness. Until very recently, these values would most; likely have been accepted as a basis for fllrther analysis. To support a more rigorous analysis, however, wc have followed Carletta's suggestion (1996) of using the K coettMcnt (Siegel and Castellan, 1988) as a measure of coder agreement. This statistic not only measures agreement, but also factors out chance agreement, and is used for nominal (or categorical) scales. In nominal scales, tiler(; is no relation between the different categories, and classification induces equivalence classes on the set of classified objects. In our coding schema, each feature determines a nominal scale on its own. Thus, we report the values of the K statistics for each feature we coded for.</Paragraph> <Paragraph position="1"> if P(A) is the prot)ortion of times the coders agree, and P(E) is the t)rot)ortion of times that coders are expected to agree by chance, K is computed as follows:</Paragraph> <Paragraph position="3"> Thus, if there is total agreement among the coders, K will be 1; if there is no agreement other than chance agreement, K will be 0. There are various ways of computing P(E); according to Siegel and Castellan (1988), most researchers where m is the nulnber of categories, and pj is the proportiorL of t)bjccts assigned to category j. The mere fact that K may have a vahw.</Paragraph> <Paragraph position="4"> k greater than zero is not sufficient to draw any conclusion, though, as it inust be estabfished whether k is significantly different fl'om zero. While Siegel and Castellan (1988, p.289) point out that it is possible to check tile significance of K when tile ,lumber of objects is large, Rietveh! and van Hout (1993) suggest a much simpler correlation between K values and inter-coder reliability, shown in Figure 2.</Paragraph> <Paragraph position="5"> For the form feature, the Kappa wfiue is 1.0, which is not surprising given its syntactic nature. The flmction features, which are more subjective in nature, engender more disagreenmnt ainong coders, as shown by the K vahms in Table 3. According to Rietveld and van Hout, tile awareness feature shows &quot;substantial&quot; agreement and the intentioimlity feature shows &quot;mo(lerate&quot; agreement.</Paragraph> </Section> </Section> <Section position="6" start_page="348" end_page="349" type="metho"> <SectionTitle> 5 Analysis </SectionTitle> <Paragraph position="0"> In our analysis, we have attempted to discover and to empirically verify correlations between tile function features and the form feature. We did this by computing X 2 statistics for the various functional features as they compared with form distinction between DONT and neg-TC imperatives. Given that the features were all two-valued we were able to use the following definition of the statistic, taken from (Siegel and Castellan, 1988): = N(IAD - BCI (A + B)(C 4- D)(A 4- O)(B 4- D) Here N is the total number of examples and A-D are the values of the elements of the 2x2 contingency table (see Figure 5). The X 2 statistic is appropriate for the correlation of two independent samples of nominally coded data, and this particular definition of it is in line with Siegel's recommendations for 2x2 contingency tables in which N > 40 (Siegel and Castellan, 1988, page 123). Concerning the assumption of independence, while it is, in fact, possible that some of the examples may have been written by a single author, the corpus was written by a considerable number of authors. Even the larger works (e.g., the cookbooks and the do-it-yourself manuals) are collections of the work of multiple authors. We felt it acceptable, therefore, to view the examples as independent and use the X 2 statistic.</Paragraph> <Paragraph position="1"> To compute X 2 for the coded examples in our corpus, we collected all the examples for which we agreed on both of the functional features (i.e., intentionality and awareness). Of the 239 total examples, 165 met this criteria. Table 4 lists the X 2 statistic and its related level of significance for each of the features. The significance levels for intentionality and awareness indicate that the features do correlate with the forms. We will focus on these features in the remainder of this section.</Paragraph> <Paragraph position="2"> The 2x 2 contingency table from which the intentionality value was derived is shown in Table 5. This table shows the frequencies of examples marked as conscious or unconscious in relation to those marked as DONT and neg-TC. A strong tendency is indicated to prevent actions the reader is likely to consciously execute using the DONT form. Note that the table entry for conscious/neg-TC is 0, indicating that there were no examples marked as both CON and neg-TC.</Paragraph> <Paragraph position="3"> Similarly, the neg-TC form is more likely to be used to prevent actions the reader is likely to execute unconsciously.</Paragraph> <Paragraph position="4"> In Section 3 we speculated that the hearer's awareness of the choice point, or more accurately, the writer's view of the bearer's awareness, would affect the appropriate form of expression of the preventative expression. In our coding, awareness was then shifted to awareness of bad consequences rather than of choices per se. However, the basic intuition that awareness plays a role in the choice of surface form is supported, as the contingency table for this feature in Table 6 shows. It indicates a strong preference ibr the use of the DONT form when the reader is presumed to be unaware of the negative consequences of the action to be prevented, the reverse being true for the use of the neg-TC form.</Paragraph> <Paragraph position="5"> The results of this analysis, therefore, demonstrate that the intentionality and awareness features do co-vary with grammatical form, and in particular, support a form of the hypothesis put forward in Section 3.</Paragraph> </Section> <Section position="7" start_page="349" end_page="350" type="metho"> <SectionTitle> 6 Application </SectionTitle> <Paragraph position="0"> We have successfully used the correlations discussed here to support the generation of warning messages in the DRAFTER project (Paris and Vander Linden, 1996). DRAFTER is a technical author= ing support tool which generates instructions for graphical interfaces. It allows its users to specify a procedure to be expressed in instructional form, and in particular, allows them to specify actions which must be prevented at the appropriate points in the procedure. At generation time, then, DRAFTER must be able to select the appropriate grammatical form for the preventative expression.</Paragraph> <Paragraph position="1"> We have used the correlations discussed in this paper to build the text planning rules required to generate negative imperatives. This is discussed in more detail elsewhere (Vander Linden and Di Eugenio, 1996), but in short, we input our coded examples to Quinlan's C4.5 learning algorithm (Quinlan, 1993), which induces a decision tree mapping from the functional features to the appropriate form. Currently, these features are set mammlly I)y the user as they are too ditticult t,o derive automatically.</Paragraph> </Section> class="xml-element"></Paper>