File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/00/c00-2126_metho.xml
Size: 10,671 bytes
Last Modified: 2025-10-06 14:07:16
<?xml version="1.0" standalone="yes"?> <Paper uid="C00-2126"> <Title>Word Order Acquisition from Corpora</Title> <Section position="3" start_page="871" end_page="873" type="metho"> <SectionTitle> 2 Word Order Acquisition and </SectionTitle> <Paragraph position="0"/> <Section position="1" start_page="871" end_page="872" type="sub_section"> <SectionTitle> Estimation 2.1 Word Order Model </SectionTitle> <Paragraph position="0"> This section describes a model which estimates the likelihood of the appropriate word order. We call this model a word order model, and we implemented it within an M.E. framework.</Paragraph> <Paragraph position="1"> Given tokenization of a test corpus, the problem of word order estimation in Japanese can be reduced to the problem of assigning one of two tags to each relationship between two modifiers. A relationship could be tagged with &quot;1&quot; to indicate that the order of the two modifiers is appropriate, or with &quot;0&quot; to indicate that it is not. Ordering all modifiers so as to assign the tag &quot;1&quot; to all relationshit)s indicates that all modifiers art in the appropriate word order. The two tags form the space of &quot;futures&quot; in the M.E. formulation of our estimation problem of word order between two modifiers. The M.E. model, as well as other similar models allows the computation of P(flh) for any f in the space of possible futures, F, and for every h in the space of possible histories, H.</Paragraph> <Paragraph position="2"> A &quot;history&quot; in maximum entropy is all of the conditioning data that enable us to make a decision in the space of futures. In the estimation problem of word order, we could reformulate this in terms of finding the probability of f associated with the relationship at index t in the test cortms as:</Paragraph> <Paragraph position="4"> from the test corpus related to relationship t) The computation of P(flh) in any M.E. models is dependent on a set of &quot;features&quot; which should be hdpful in making a prediction about the flmlre. Like most current M.E. models in computational linguistics, our model is restricted to features which are binary functions of the history and future. For instance, one of our features is</Paragraph> <Paragraph position="6"> Here &quot;has(h,x)&quot; is a binary flmction which returns true if the history h has feature z. We focus on the attributes of a bunsetsu itself and on the features occurring between bunsetsus.</Paragraph> <Paragraph position="7"> Given a set of features and some training data, the maximum entropy estimation process produces a model ill which every feature .qi has associated with it a parameter ai. This allows us to compute the conditional probability as follows (Berger et al., 1996):</Paragraph> <Paragraph position="9"> The maximum entropy estimation technique guarantees that for every feature gi, the expected value of gi according to the M.E. model will equal the empirical expectation of gi in the training corpus. In other words:</Paragraph> <Paragraph position="11"> Here /5 is an empirical probability and l~Ie is the</Paragraph> <Paragraph position="13"> prol)ability assigned by the M.E. model.</Paragraph> <Paragraph position="14"> We detine a word order model as a model which learns the at)l)ropriate order of each pair of nlodifiers which depend on the same modifiee. 'l'his model is derived from Eq. (2) as followsdeg Assmne that there are two bunsetsus 231 and 23~ which depend on the buusetsu B and that It is the information derivable from the test corpus. \]?lie probability that &quot;B\] B2&quot; is the at)propriate order is given by the following equation: null \]iik: :1 ffi(l ,b) (~'i ,i where .qi(1 < i < k) is a fl,atm'e and &quot;1&quot; indicates that the order is at)propriate. The terms cq,i and (~0,i are estimated fl'oln a eorl)us which is nlorphologically and syntactically analyzed. When there are three or more b\]msetsus that det)end on tit('. S}tlne ? moditiee, the probability is estimated as follows: I or ~'t bunsetsus 231, 232, ..., 23n which depend on the bmtsetsu B and for the information h derivaMe from the test corpus, the prot)ability t;hat &quot;23\] 23~ ... 23,&quot; is the at)propriate order, or P(llh), is represented ass the probability that every two bunsetsus &quot;Bi ~i-Fj (1 _< i < n - 1,1 < j < 'n - i)&quot; are the appropri~ ate. order, or P({14~i,i_l.j = l\[l< i <. n-- 1, l :< j < n--i}lh), ilere &quot;I4Li+j -- l&quot; represents that &quot;.l~i 23i-Fj&quot; is the appropriate order. Let us assume that every 14Q,~:+j is independent each other. Then 1)(1 Ih,) is derived as follows:</Paragraph> <Paragraph position="16"> where \[Si,i+ j is the information derivable when fbcusing on the bunsetsu 13 m~d its modifiers 13i and Bi+j.</Paragraph> <Paragraph position="17"> For example, in the sentence &quot;I~ U (kinou, yesterday) / :kflll ~ (Taro_wa, Taro) / -P ~ x ~ (tcnnis_wo, tennis) / b t:o (sita., l)layed.),&quot; where a &quot;/&quot; represents a bunsetsu boundary, there are three bunsetsus that depend on the verb &quot;b ~: (sita).&quot; We train a word order inodel under the assmnl)tion that the orders of three t)airs of modifiers -&quot;I~l U&quot; and &quot;~ f$1.~,&quot; &quot;Net\] &quot; and &quot;C/ 7-:7, ~ ,&quot; and &quot;:kl~l*l~&quot; and &quot;5: m :7, ~&quot; .... are al)ttropriate. We use various ldnds of intormation in and around the target bunsetsus as features. For example, the information or the feature that a noun of time i)recedes a t)rot)er noun is derivable fl'om the order &quot;IP} H (yesterday) / Y;fll~ I~ (Taro) / b 1=o (pl~\yed.),&quot; and the feature that a case followed by a case marker &quot;wPS' precedes a case followed by a caqe marker &quot;wo&quot; is derivable from the order &quot;:~ fll~ It. ( Taro_wa,</Paragraph> <Paragraph position="19"/> </Section> <Section position="2" start_page="872" end_page="872" type="sub_section"> <SectionTitle> 2.2 Word Order Estimation </SectionTitle> <Paragraph position="0"> This section describes the algorithm of estimating the word order by using a trained word order model.</Paragraph> <Paragraph position="1"> The word order estimation is defined as deciding the order of ntoditiers or bunsetsus which depend on the same modifiee. The input of this task consists of modifiers and informat, ion necessary to know whether or not features are found. The output is the order of the inodifiers. We assume that lexical selection in each bunsetsu is already done and all del)endencies in a sentence are found. The information necessary to know whether or not features are found is morphological, syntactic, semmltic, and ('ontextual information, and the locations of bunsetsu bonndaries. The features used in our ext)eriments are described in Section 3.</Paragraph> <Paragraph position="2"> Word order is estimated in the following steps.</Paragraph> <Paragraph position="3"> 1. All possible orders of modifiers are found.</Paragraph> <Paragraph position="4"> 2. For each, the probability that it is apt)ropriate is estimated by a word order model, or Eq. (6).</Paragraph> <Paragraph position="5"> 3. The order with the highest probability of 1)eing approl)riate is selected.</Paragraph> <Paragraph position="6"> l)br example, given the sentence &quot;1~ U (kinou, yesterday) /:kfilIl~ (Taro_wa, Taro) /C/::x ~ (tcn- nis_wo, temfis) / b t:o (sita., played.),&quot; tim moditiers of a verb &quot;b ?:_ (played)&quot; are three tmnsetsus, &quot;l~ U (yesterday),&quot; &quot;:k~/i ~ (Taro),&quot; &quot;C/ = x ~ (tennis).&quot; Their apt)ropriate order is estimated in the following steps.</Paragraph> <Paragraph position="7"> 1. The probabilities that the orders of the three pairs of modifiers &quot;N- LI &quot; and &quot;:is: BII l~ ,&quot; &quot;I~ U&quot; and &quot;C/~:7,~,&quot; and &quot;~fllIl+-&quot; and &quot;C/ c.x ~&quot; are appropriate are estimated. Assume, for example, ~-H ,;k~l~ta, PrI~ It ,C/ :-~. ~, and</Paragraph> <Paragraph position="9"> 2. As shown in Table 1, probabilities are estimated for all six possible orders. The order &quot;I~ U / :k fill IS / -7- ~- y. ~ / b \]Co ,&quot; which has the highest probability, is selected as the most apt)ropriate order.</Paragraph> </Section> <Section position="3" start_page="872" end_page="873" type="sub_section"> <SectionTitle> 2.3 Performance Evaluation </SectionTitle> <Paragraph position="0"> The pcrformancc of a word order model can be evaluated in the following way. First, extract from a test corpus bunsetsus having two or more modifiers.</Paragraph> <Paragraph position="1"> Then, using those 1)unsetsus and their modifiers as Moditiers whose modiliee is the bunsetsu in the left column.</Paragraph> <Paragraph position="3"> input, estimate the orders of the modifiers as described in Section 2.2. The percentage of the modiflees whose modifiers' word order agrees with that in the original text then gives what we call the agreement rate. It is a measure of how close the word order estimated by the model is to the actual word order in the training corpus.</Paragraph> <Paragraph position="4"> We use the following two measurements to calculate the agreement rate.</Paragraph> <Paragraph position="5"> Pair of modifiers The first measurement is the percentage of the pairs of modifiers whose word order agrees with that in the test corpus. For exmnple, given the sentence in a test corpus &quot;N</Paragraph> <Paragraph position="7"> played.),&quot; if the word order estimated by the model is &quot;~ H (yesterday) / -7&quot; -- 2. ~ (tennis) / ~1~ ~:~ (Taro) / b too (played.),&quot; then the orders of the pairs of modifiers in the original sentence are &quot;N H / ;k~l~ ~ ,&quot; &quot;15 H / -7- =- :7, ~ ,&quot; and &quot;~lit:~ / ~--2` ~ ,&quot; and those in the estimated word order are &quot;~H / -~---2`~,&quot; &quot;~H / 1~1~ la~ ,&quot; and &quot;Y- = 2` ~ / %:t~ll lak .&quot; The agreement rate is 67% (2/3) because two of the three orders are the same as those in the original sentence.</Paragraph> <Paragraph position="8"> Complete agreement The second measurement is the percentage of the modifiees whose modifiers' word order agrees with that in the test corpus.</Paragraph> </Section> </Section> class="xml-element"></Paper>