File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/98/p98-2198_metho.xml
Size: 16,873 bytes
Last Modified: 2025-10-06 14:15:01
<?xml version="1.0" standalone="yes"?> <Paper uid="P98-2198"> <Title>Locating noun phrases with finite state transducers.</Title> <Section position="3" start_page="1212" end_page="1212" type="metho"> <SectionTitle> 2 Formal Description </SectionTitle> <Paragraph position="0"> We deal with noun phrases containing a description of an occupation, a proper noun, or both combined. For example, 'a senior RAF o\]flcer', 'Peter Lilley, the Shadow Chancellor', 'Sir Terence Burns, the Treasury Permanent Secretary' or 'a former Haitian prime minister, Rosny Smarth'. For our purpose, we must have a formal way of describing and identifying such sequences.</Paragraph> <Section position="1" start_page="1212" end_page="1212" type="sub_section"> <SectionTitle> 2.1 Description of occupations </SectionTitle> <Paragraph position="0"> We describe occupations by means of local grammars, which are directly written in the form of FS graphs. These graphs are equivalent to FSTs with inverted representation (FST) (Roche and Schabes, 1997) as in figure 1, where each box represents a transition of the automaton (input of the transducer), and the label under a box is an output of the transducer. The initial state is the left arrow, the final state is the double square. The optional grey boxes, (cf figure 2), represent sub-transducers: in other words, by 'zooming' on all sub-transducers, we view a given FST as a simple graph, with no parts singled out. However, we insist on keeping sub-FST automata, as they will be computed independently, and as they allow us to keep a simple representation of complex constructions. The output of a grey box, is the output of the sub-transducer. The symbol labeled <E> represents the void transition, and the different lines inside are parallel transitions. Such a representation is convenient to formulate linguistic constraints. A graph editor (Silberztein, 1993) is available to directly construct FSTs. In theory, such FSTs are more powerful than traditional FSTQ.</Paragraph> <Paragraph position="1"> In figure 1, the transducer recognizes the sequences a, b, ca, cb. To each of these input sequences, it associates an output noted val(input). Here, val(a) = {ab}, val(b) = {b}, 2 If a sub-automaton refers to a parent automaton, we will be able to express context dependent words such as a'*b n . val(c) is not defined as c is not recognized by the automaton, val(ca) = {d}, and val(cb) = {b}.</Paragraph> <Paragraph position="2"> We define an ordering relation on the set of recognized sequences by a transducer T, that is: x <_T Y C/:~ Veeval(x), eEval(y). In our example, b --<T a and b =7- cb with derived equality relation.</Paragraph> <Paragraph position="3"> We construct our transducer describing occupations in such a way that with this ordering 3 relation: - Two sequences x, y are synonyms if and only if x =7- Y - The sequence y is an hyponym of x (i.e. y is a x) if and only if x --<T Y.</Paragraph> <Paragraph position="4"> The transducer in figure 2 describes 4 different sequences referring to the word minister.</Paragraph> <Paragraph position="5"> Sub-parts of the transducers Country and Nationality are given in figure 3 and 4.</Paragraph> <Paragraph position="6"> By construction, all the sequences recognized are grammatically correct. For example, the variant of minister of European affairs: minister for European affairs is recognized, but not 3 The equality relation --r az~d the strict comparison are directly deduced from _<T definition.</Paragraph> </Section> </Section> <Section position="4" start_page="1212" end_page="1216" type="metho"> <SectionTitle> 4 For the sake of clarity, it is not complete, for exam- </SectionTitle> <Paragraph position="0"> ple it doesn't take into account regional ministries as in USA or in India. It doesn't represent either the sequence deputy prime minister. Moreover, a large part could be factorized in a sub-automaton.</Paragraph> <Paragraph position="1"> French minister for agriculture. The output of the transducer is compatible with our definition prime minister is a minister but 'deputy minister~7-'minister' i.e. a deputy minister is not a minister.</Paragraph> <Paragraph position="2"> Reciprocally, given an output, it easy to find all paths corresponding to this output (by inverting the inputs and the outputs in the transducer). This will be very useful to fornmlate answers to requests, or to translate noun phrases: the ':natural language&quot; sequences corresponding to the set {minister, French} are : &quot;French minister&quot; or &quot;minister of France&quot;. We will note val-i({minister, French}) = {'french minister', 'minister of France'}.</Paragraph> <Section position="1" start_page="1212" end_page="1212" type="sub_section"> <SectionTitle> 2.2 Full Name description </SectionTitle> <Paragraph position="0"> The full name description is based oll the same methodology (cf. figure 5), except that the boxes containing <PN : F+-rstName> and <PN:SurName> represent words of the proper nouns dictionaries. The output of this transducer is computed in a different way: the output is the surname, the firstname if available, and the gender implied either by the firstname, or by the short title: Mr., Sir, princess, etc ....</Paragraph> <Paragraph position="1"> 3 Handling requests: a dynamic dictionary In order to instantly obtain answers for all requests, we build an incremental index of all matches described by the FST library. At this stage, the program proposes new possible proper nouns not yet listed, they complete the dictionary. Our index has the following property: when an FST is modified, it is not the whole library which is recompiled, but only the FSTs affected by the modification. We now describe this stage and show how the program consults the index and the FST library to construct the answer.</Paragraph> </Section> <Section position="2" start_page="1212" end_page="1216" type="sub_section"> <SectionTitle> 3.1 Constructing the database </SectionTitle> <Paragraph position="0"> In (Senellart, 1998), a fast algorithm that parses regular expressions on full inverted text is presented. We use such an algorithm for locating occurrences of the FSTs in the text.</Paragraph> <Paragraph position="1"> For each FST, and for each of its occurrences in the text, we compute the position, the length, and the FST associated output of the occurrence.</Paragraph> <Paragraph position="2"> This type of index is compressed in tile same way entries of the full inverted text are. This choice of structure has the following features: 1. There is no difference of parsing between a 'grey (autonomous) box' and a 'norreal one'. Once sub-transducers have been compiled, they behave like normal words.</Paragraph> <Paragraph position="3"> Thus, the parsing algorithms are exactly the same.</Paragraph> <Paragraph position="4"> 2. A makefile including dependencies between the different graphs is built, and modifications of one graph triggers the re-compilation of the graphs directly or indirectly dependent.</Paragraph> <Paragraph position="5"> . This structure is incrementah adding new texts to the database is easy, we only need to index them and to merge the previous index with the new one by. a trivial pointer operation.</Paragraph> <Paragraph position="6"> A description of a whole noun phrase is given made by the graph of figure 6.</Paragraph> <Paragraph position="7"> (Information of the general purpose dictionary) We use a second structure: a dynamic proper noun dictionary ~ that relies on the indexes of Occupation.graph and FullName.graph. T) is called 'dynamic' dictionary, because the information associated to the entries depend on the locations in the text we are looking for. The algorithm that constructs T) is the following: 1. For each recognized occurrence we associate O1 which is the output of Full-Name.graph and the output 02 of the Occupation.graph (see section 4 for examples). null 2. If O1 is not empty., find O1 in :D: that is, find the last e in T) such that O1 <__7- e. If there is none, create one : i.e. associate this FullName with the occupation 02 and with the current location in the text.</Paragraph> <Paragraph position="8"> - If there exists one, and its occupation is compatible with 02 then add the current location to this entry. Or else, create a new entry for O1 (eventually completed by the information from e) with its new occupation 02, and pointing to the current loca-tion in the text.</Paragraph> <Paragraph position="9"> 3. If O1 is empty: the noun phrase is limited to the occupation part. Find the last entry in :D compatible with 02, and then add the</Paragraph> </Section> <Section position="3" start_page="1216" end_page="1216" type="sub_section"> <SectionTitle> 3.2 Consulting the database </SectionTitle> <Paragraph position="0"> Given a request of type 1: Who is P. We first apply tile NounPhrases.graph to P. If P is not recognized, the research fails. It it is recognized, we obtain two outputs O1 and 02 as previously mentioned. For this type of request O1 cannot be empty. So we look in T) for the entries that match O1 (there can be several, often when the first name is not given, or given by its initial). Then, we print the different occupations associated to these entries.</Paragraph> <Paragraph position="1"> Given a request of type 2: the result is just an extension of the previous case: once we have found the entries in T~, we print all positions associated in the text.</Paragraph> <Paragraph position="2"> Given a request of type 3, the method is different: we begin by applying the Noun-Phrases.graph to P. In this case, O1 is empty. Then we look up the entries of 2), and check if at some location of the text, its occupation is compatible with the occupation of the request.</Paragraph> </Section> </Section> <Section position="5" start_page="1216" end_page="1216" type="metho"> <SectionTitle> 4 Examples of use </SectionTitle> <Paragraph position="0"> Consider the following chronological extract of French newspaper : I- M. Jack Lan K, minlstre de i'dducation nationale et de la culture, 2- ChafE& le 7 avril 1992 par M. Lan K de rdfldchlr aux conditions de 3- M. Jack Lank a lanc4 dimanche soir ~I la t&Idvision l'idde d'impliquer 4- Commentant Faction du mlnlstre de la culture, le premier adjolnt 5- En d4finltive l'idde de M. Lan K apparaTt comme un r~ve ! 6- Le directeur de l'American Ballet Theater, Kevin McKenzle : 7- M. Lan K pr~sente son pro jet de r~forme des lycdes prdvoyant 8- Tous, soutenez la |oi Lan K, par distraction, de temps en temps, ici 9- M. Jack Lan K, maire de Blols, a omclellement d~posd sa I0- Sortants : Michel Fromet, suppldant de Jack LanK, se repr~sente 11- De son cotd. Carl Lan K, secr~talre gdn@ra\] du Front national, a 12- et Jack Lan K, anclen mlnlstre de \['dducatlon natlonale et de la culture, 13- l'ancien ministre, Jack LanK, et son successeur, Jacques Toubon, 14- Jack Lang, malre de Blois et anclen minlstre, 15- ..., le nouveau ministre de l'4ducation nationale, Jacques Woubo., - At the beginning 7) is empty.</Paragraph> <Paragraph position="1"> - We read the sentence h</Paragraph> <Paragraph position="3"> no entry in 7) corresponding to 01, thus we create in 7) the following entry : SurName=LanE, FirstName=Jack, Gender=m, (Line 1 Occupation=minis%or,education,culture) - We read the sentence 2:O1 --- {m, Lang}. 01 matches the only entry in 7), and moreover as 02 is empty: it also matches the entry. Thus we add the line 2, as a reference to this first entry.</Paragraph> <Paragraph position="4"> tioning the minister of culture, we apply NounPhrases.graph to this request and we find O1 = {}, O2 = {minister, culture}. The only entries in 7) matching 02 correspond to the lines 1,2,3,4,5,7,13,15. This was expected, lines referring to the homonym of Jack Lang have not be considered, nor line referring Jack Lang designated as the mayor of Blois.</Paragraph> </Section> <Section position="6" start_page="1216" end_page="1217" type="metho"> <SectionTitle> 5 Remaining errors </SectionTitle> <Paragraph position="0"> Some cases are difficult to solve, as we can see in the sentence: In China, the first minister has .... The first phrase of the sentence: In China is an adverbial, and could be located everywhere in the sentence. It could even be implicit, that is, implied by the rest of the text. In such a case, a human reader will need the context, to identify the person designated. We are not able, to extract the information we need, thus the result is not false, but imprecise.</Paragraph> <Paragraph position="1"> Another situation leads to wrong results: when one same person has several occupations, and is designated sometimes by one, sometimes by another. To resolve such a case, we must represent the set of occupations that axe compatible. This is a rather large project ell the 'semantics' of occupation.</Paragraph> <Paragraph position="2"> Finally, as we can see if figure 6, a determiner and an adjective can be found between the Fullname part, and the Occupation part. In most case, it is something 'this', or 'tile', or 'the wellknown', or 'our great', and can be easily described by a FST. But in very exceptional case, we can find also complex sequences between the Fullname part, and the Occupation part. For example: 'M. X, who is since 1976, the prime minister of ...'. In this case, it is not possible, in tile current state of the developpment of out FST library, to provide a complete description.</Paragraph> <Paragraph position="3"> 6 Building the dictionaries and the database The results of our approach is in proportion tile size of the database we use. We show that using variables in FSTs, and the bootstrapping method, this constraint is not as huge it seems.</Paragraph> <Paragraph position="4"> One can start with a minimal database and improve tile database, when testing it on a new corpus. Suppose for example, that the database is empty (we only have general purpose dictionaries). We ask the system to find all occurrences of the word 'minister', the result has the following form of concordance.&quot; The Israeli foreign minister. $himon Peres. said the intern the Russian foreign minister. Indrei V. gozyrev, was likely Berlusconi as prime minister, but ty issue ought to be the C/oturi, as the Creek minister of culture, thought up the ide fir:~ deputy prime minister, Oleg Soskove~s; Moscow has pl On this small sample, we see that it is interesting to search the different occurrences of &quot;(<A>+<N>) <minister>&quot; and we obtain the list: prime, foreign, Greek, finance, trade, interior, Cambodian, ...</Paragraph> <Paragraph position="5"> We separate automatically in this list, words with uppercase first letter and lowercase words.</Paragraph> <Paragraph position="6"> This provide a first draft for a Nationality dictionary (on a 1Mo corpus, we obtain 234 entries (only with this simple rule). The list is then manually checked to extract noise as 'Trade minister of ...'. We then sort the lowercase adjective and begin to construct the minister graph. We find directly 23 words in the sub-graph &quot;SpecialityMinisterLeft&quot;, plus the special compounds &quot;prime minister&quot; and &quot;chief minister&quot;. We then apply this graph to the corpus and attempt to extend occurrences to the left and to the right. We notice that we can find a name of country with an &quot;'s&quot;just to the left of the occupation, and thus we catch potential names of country with the following request: &quot;\[A-Z\]\[a-z\]*'s :MinisterOccupation&quot;, where \[A-Z\] \[a-z\] * is any word beginning with an uppercase letter. This is an example of variable in the automaton. Pursuing this text-based method and starting from scratch, in roughly 10 minutes, we build a first version of the dictionaries: Country (87 entries) and Nationality (255 entries), Firstname (50 entries), Surname (47 entries), plus a first version of the MinisterOccupation and the FullName FSTs... The graphical tools and the real-time parsing algorithms we use are crucial in this construction.</Paragraph> <Paragraph position="7"> Remark that the strict domain of proper noun cannot be bounded: when we describe occupations in companies, we must catch the company names. When we describe the medical occupation, we are lead to catch the hospital names... Very quickly the coverage of the database enlarges, and dictionaries of names of companies, of towns must be constructed. Concerning the French, in a newspaper corpus, one word out of twenty is included in a occupation sequence: i.e.</Paragraph> <Paragraph position="8"> one sentence out of two in our corpus contained such noun phrase.</Paragraph> </Section> class="xml-element"></Paper>