File Information
File: 05-lr/acl_arc_1_sum/cleansed_text/xml_by_section/metho/05/p05-1009_metho.xml
Size: 28,072 bytes
Last Modified: 2025-10-06 14:09:40
<?xml version="1.0" standalone="yes"?> <Paper uid="P05-1009"> <Title>Towards Developing Generation Algorithms for Text-to-Text Applications</Title> <Section position="3" start_page="0" end_page="68" type="metho"> <SectionTitle> 2 The IDL Language for NLG </SectionTitle> <Paragraph position="0"/> <Section position="1" start_page="0" end_page="67" type="sub_section"> <SectionTitle> 2.1 IDL-expressions </SectionTitle> <Paragraph position="0"> IDL-expressions have been proposed by Nederhof & Satta (2004) (henceforth N&S) as a representation for finite languages, and are created from strings using four operators: concatenation ( a2 ), interleave ( a3 ), disjunction (a4 ), and lock ( a5 ). The semantics of IDL-expressions is given in terms of sets of strings.</Paragraph> <Paragraph position="1"> The concatenation ( a2 ) operator takes two arguments, and uses the strings encoded by its argument expressions to obtain concatenated strings that respect the order of the arguments; e.g., a5 a2a7a6 encodes the singleton set a8a9a5a10a6a12a11 . The a13 nterleave ( a3 ) operator interleaves the strings encoded by its argument expressions; e.g., a3a15a14a16a5 a2a17a6a19a18a21a20a9a22 encodes the set a8a9a20a23a5a10a6a24a18a21a5a25a20a26a6a27a18a21a5a10a6a12a20a28a11 . The a29 isjunction (a4 ) operator allows a choice among the strings encoded by its argument expressions; e.g., a4a30a14a16a5a7a18a4a6a31a22 encodes the set a8a9a5a7a18a4a6a12a11 . The a32 ock ( a5 ) operator takes only one argument, and &quot;locks-in&quot; the strings encoded by its argument expression, such that no additional material can be interleaved; e.g., a3a15a14 a5a33a14a16a5 a2a25a6a31a22a34a18a21a20a9a22 encodes the set a8a9a20a23a5a10a6a19a18a21a5a10a6a12a20a10a11 .</Paragraph> <Paragraph position="2"> Consider the following IDL-expression:</Paragraph> <Paragraph position="4"> The concatenation ( a2 ) operator captures precedence constraints, such as the fact that a determiner like the appears before the noun it determines. The lock</Paragraph> <Paragraph position="6"> such as the fact that the captives is a phrase which should be used as a whole. The disjunction (a4 ) operator allows for multiple word/phrase choice (e.g., the prisoners versus the captives), and the interleave ( a3 ) operator allows for word-order freedom, i.e., word order underspecification at meaning representation level. Among the strings encoded by IDL-expression 1 are the following: finally the prisoners were released the captives finally were released the prisoners were finally released The following strings, however, are not part of the language defined by IDL-expression 1: the finally captives were released the prisoners were released finally the captives released were The first string is disallowed because the a5 operator locks the phrase the captives. The second string is not allowed because the a3 operator requires all its arguments to be represented. The last string violates the order imposed by the precedence operator between were and released.</Paragraph> </Section> <Section position="2" start_page="67" end_page="67" type="sub_section"> <SectionTitle> 2.2 IDL-graphs </SectionTitle> <Paragraph position="0"> IDL-expressions are a convenient way to compactly represent finite languages. However, IDL-expressions do not directly allow formulations of algorithms to process them. For this purpose, an equivalent representation is introduced by N&S, called IDL-graphs. We refer the interested reader to the formal definition provided by N&S, and provide here only an intuitive description of IDL-graphs.</Paragraph> <Paragraph position="1"> We illustrate in Figure 1 the IDL-graph corresponding to IDL-expression 1. In this graph, vertices a0a2a1 and a0a4a3 are called initial and final, respectively. Vertices a0a6a5 , a0 a2 with in-going a7 -labeled edges, and a0a9a8 , a0 a2 a5 with out-going a10 -labeled edges, for example, result from the expansion of the a3 operator, while vertices a0a2a11 , a0a13a12 with in-going a14 -labeled edges, and a0a4a15 , a0a9a8 a12 with out-going a14 -labeled edges result from the expansion of the a4 operator. Vertices a0 a3 to a0a4a16 and a0a9a8a17a5 to a0a9a8 a11 result from the expansion of the two a5 operators, respectively. These latter vertices are also shown to have rank 1, as opposed to rank 0 (not shown) assigned to all other vertices.</Paragraph> <Paragraph position="2"> The ranking of vertices in an IDL-graph is needed to enforce a higher priority on the processing of the higher-ranked vertices, such that the desired semantics for the lock operator is preserved.</Paragraph> <Paragraph position="3"> With each IDL-graph a18 a14a20a19a24a22 we can associate a finite language: the set of strings that can be generated by an IDL-specific traversal of a18 a14a20a19a24a22 , starting from a0a2a1 and ending in a0a2a3 . An IDL-expression a19 and its corresponding IDL-graph a18 a14a20a19a24a22 are said to be equivalent because they generate the same finite language, denoted a21 a14a20a19a24a22 .</Paragraph> </Section> <Section position="3" start_page="67" end_page="68" type="sub_section"> <SectionTitle> 2.3 IDL-graphs and Finite-State Acceptors </SectionTitle> <Paragraph position="0"> To make the connection with the formulation of our algorithms, in this section we link the IDL formalism with the more classical formalism of finite-state acceptors (FSA) (Hopcroft and Ullman, 1979). The FSA representation can naturally encode precedence and multiple choice, but it lacks primitives corresponding to the interleave ( a3 ) and lock ( a5 ) operators. As such, an FSA representation must explicitly enumerate all possible interleavings, which are implicitly captured in an IDL representation. This correspondence between implicit and explicit interleavings is naturally handled by the notion of a cut of an IDL-graph a18 a14a20a19a24a22 .</Paragraph> <Paragraph position="1"> Intuitively, a cut through a18 a14a20a19a24a22 is a set of vertices that can be reached simultaneously when traversing a18 a14a20a19a24a22 from the initial node to the final node, following the branches as prescribed by the encoded a13 , a29 , and a32 operators, in an attempt to produce a string in a21a57a14a20a19a24a22 . More precisely, the initial vertex a0a9a1 is considered a cut (Figure 2 (a)). For each vertex in a given cut, we create a new cut by replacing the start vertex of some edge with the end vertex of that edge, observing the following rules: a22 the vertex that is the start of several edges labeled using the special symbol a7 is replaced by a sequence of all the end vertices of these edges (for example, a0a6a5a23a0 a2 is a cut derived from a0a2a1 (Figure 2 (b))); a mirror rule handles the special symbol a10 ; a22 the vertex that is the start of an edge labeled using vocabulary items or a14 is replaced by the end vertex of that edge (for example, a0a24a8a25a0 a2 , a0a4a5a23a0 a11 ,</Paragraph> <Paragraph position="3"> non-cut is presented in (e).</Paragraph> <Paragraph position="4"> a0 a5 a0a4a11 , and a0 a5 a0 a3 , respectively, see Figure 2 (cd)), only if the end vertex is not lower ranked than any of the vertices already present in the cut (for example, a0 a8a25a0a4a27 is not a cut that can be derived from a0a2a5a26a0a4a27 , see Figure 2 (e)). Note the last part of the second rule, which restricts the set of cuts by using the ranking mechanism. If one would allow a0 a8 a0 a27 to be a cut, one would imply that finally may appear inserted between the words of the locked phrase the prisoners.</Paragraph> <Paragraph position="5"> We now link the IDL formalism with the FSA formalism by providing a mapping from an IDL-graph a18 a14a20a19a24a22 to an acyclic finite-state acceptor a1 a14a20a19a24a22 . Because both formalisms are used for representing finite languages, they have equivalent representational power. The IDL representation is much more compact, however, as one can observe by comparing the IDL-graph in Figure 1 with the equivalent finite-state acceptor a1 a14a20a19a24a22 in Figure 3. The set of states of a14a20a19a24a22 is the set of cuts of a18 a14a20a19a24a22 . The initial state of the finite-state acceptor is the state corresponding to cut a0a2a1 , and the final states of the finite-state acceptor are the state corresponding to cuts that contain a0a9a3 . In what follows, we denote a state of a1 a14a20a19a24a22 by the name of the cut to which it corresponds. A transi- null the IDL-graph in Figure 1.</Paragraph> <Paragraph position="6"> tion labeled a2 in a1 a14a20a19a24a22 between state a3 a0a5a4a6a5a7a8a7a8a7 a0a9a4a10 a7a8a7a8a7 a0a11a4a12a14a13 and state a3 a0 a4 a4a6 a7a8a7a8a7 a0 a4 a4a10 a7a8a7a8a7 a0 a4 a4a12 a13 occurs if there is an edge</Paragraph> <Paragraph position="8"> a22 in a18 a14a20a19a24a22 . For the example in Figure 3, the transition labeled were between states a3 a0a9a5a23a0a9a8a17a27 a13 and a3 a0a4a5a26a0 a8a17a16 a13 occurs because of the edge labeled were between nodes a0 a8a17a27 and a0a9a8a17a16 (Figure 1), whereas the transition labeled finally between states a3 a0 a5a23a0a9a8a17a27 a13 and a3 a0 a8 a0 a8a17a27 a13 occurs because of the edge labeled finally between nodes a0a4a5 and a0a9a8 (Figure 1). The two representations a18 a14a20a19a24a22 and a1 a14a20a19a24a22 are equivalent in the sense that the language generated by IDL-graph a18 a14a20a19a24a22 is the same as the language accepted by FSA a1 a14a20a19a24a22 .</Paragraph> <Paragraph position="9"> It is not hard to see that the conversion from the IDL representation to the FSA representation destroys the compactness property of the IDL formalism, because of the explicit enumeration of all possible interleavings, which causes certain labels to appear repeatedly in transitions. For example, a transition labeled finally appears 11 times in the finite-state acceptor in Figure 3, whereas an edge labeled finally appears only once in the IDL-graph in Figure 1.</Paragraph> </Section> </Section> <Section position="4" start_page="68" end_page="71" type="metho"> <SectionTitle> 3 Computational Properties of </SectionTitle> <Paragraph position="0"/> <Section position="1" start_page="68" end_page="69" type="sub_section"> <SectionTitle> IDL-expressions 3.1 IDL-graphs and Weighted Finite-State Acceptors </SectionTitle> <Paragraph position="0"> As mentioned in Section 1, the generation mechanism we propose performs an intersection of IDL-expressions with n-gram language models. Following (Mohri et al., 2002; Knight and Graehl, 1998), we implement language models using weighted finite-state acceptors (wFSA). In Section 2.3, we presented a mapping from an IDL-graph a18 a14a20a19a24a22 to a finite-state acceptor a1 a14a20a19a24a22 . From such a finite-state acceptor a1 a14a20a19a24a22 , we arrive at a weighted finite-state acceptor a18 a14a20a19a24a22 , by splitting the states of a1 a14a20a19a24a22 ac- null cording to the information needed by the language model to assign weights to transitions. For example, under a bigram language model a21a1a0 , state Note that, at this point, we already have a na&quot;ive algorithm for intersecting IDL-expressions with n-gram language models. From an IDL-expression a19 , following the mapping a19a11a10 a18 a14a20a19a24a22a12a10 a1 a14a20a19a24a22a12a10 a18 a14a20a19a24a22 , we arrive at a weighted finite-state acceptor, on which we can use a single-source shortest-path algorithm for directed acyclic graphs (Cormen et al., 2001) to extract the realization corresponding to the most probable path. The problem with this algorithm, however, is that the premature unfolding of the IDL-graph into a finite-state acceptor destroys the representation compactness of the IDL representation. For this reason, we devise algorithms that, although similar in spirit with the single-source shortest-path algorithm for directed acyclic graphs, perform on-the-fly unfolding of the IDL-graph, with a mechanism to control the unfolding based on the scores of the paths already unfolded. Such an approach has the advantage that prefixes that are extremely unlikely under the language model may be regarded as not so promising, and parts of the IDL-expression that contain them may not be unfolded, leading to significant savings.</Paragraph> </Section> <Section position="2" start_page="69" end_page="70" type="sub_section"> <SectionTitle> 3.2 Generation via Intersection of </SectionTitle> <Paragraph position="0"> state acceptor a18 corresponding to an IDL-graph a18 incrementally, by keeping track of a set of active states, called a39 a58a9a45a16a52a42a59a25a49 . The incrementality comes from creating new transitions and states in a18 originating in these active states, by unfolding the IDL-graph a18 ; the set of newly unfolded states is called cording to the language model. If a final state of a18 is not yet reached, the while loop is closed by making the a13 a37a15a14a26a56a25a41 a64 set of states to be the next set of a39 a58a9a45a16a52a42a59a25a49 states. Note that this is actually a breadth-first search (BFS) with incremental unfolding. This algorithm still unfolds the IDL-graph completely, and therefore suffers from the same drawback as the na&quot;ive algorithm.</Paragraph> <Paragraph position="1"> The interesting contribution of algorithm IDL-NGLM-BFS, however, is the incremental unfolding. If, instead of line 8 in Figure 4, we introduce mechanisms to control which a13 a37a15a14a26a56a25a41 a64 states become part of the a39 a58 a45a16a52a48a59a10a49 state set for the next unfolding iteration, we obtain a series of more effective algorithms.</Paragraph> <Paragraph position="2"> Algorithm IDL-NGLM-Aa30 We arrive at algorithm IDL-NGLM-Aa30 by modifying line 8 in Figure 4, thus obtaining the algorithm in Figure 5. We use as control mechanism a priority queue, a39a28a54a26a45 a39a10a51a22a31 , in which the states from a13 a37a32a14a26a56a10a41a62a64 are PUSH-ed, sorted according to an admissible heuristic function (Russell and Norvig, 1995). In the next iteration, a39 a58 a45a16a52a48a59a10a49 is a singleton set containing the state POP-ed out from the top of the priority queue.</Paragraph> <Paragraph position="3"> Algorithm IDL-NGLM-BEAM We arrive at algorithm IDL-NGLM-BEAM by again modifying line 8 in Figure 4, thus obtaining the algorithm in Figure 6. We control the unfolding using a probabilistic beam a33a34a49a21a39a35a34 , which, via the BEAMSTATES function, selects as a39 a58 a45a16a52a48a59a10a49 states only the states in a37a15a14a23a56a10a41 a64 reachable with a probability higher or equal to the current maximum probability times the probability beam a33a63a49a63a39a35a34 .</Paragraph> </Section> <Section position="3" start_page="70" end_page="70" type="sub_section"> <SectionTitle> 3.3 Computing Admissible Heuristics for IDL-expressions </SectionTitle> <Paragraph position="0"> The IDL representation is ideally suited for computing accurate admissible heuristics under language models. These heuristics are needed by the IDL-NGLM-Aa30 algorithm, and are also employed for pruning by the IDL-NGLM-BEAM algorithm.</Paragraph> <Paragraph position="1"> For each state a1 in a weighted finite-state acceptor a18 corresponding to an IDL-graph a18 , one can efficiently extract from a18 - without further unfolding - the set1 of all edge labels that can be used to reach the final states of a18 . This set of labels, denoted a2a4a3a6a5a8a7a9a7a10 , is an overestimation of the set of future events reachable from a1 , because the labels under the a4 operators are all considered. From a2a4a3 a5a8a7a11a7a10 and the a12 -1 labels (when using an a12 -gram language model) recorded in state a1 we obtain the set of label sequences of length a12 -1. This set, denoted a2a14a13a15a3 a10 , is an (over)estimated set of possible future conditioning events for state a1 , guaranteed to contain the most cost-efficient future conditioning events for state a1 .</Paragraph> <Paragraph position="2"> Using a2a16a13a15a3 a10 , one needs to extract from a2a17a3 a5a18a7a11a7a10 the set of most cost-efficient future events from under each a4 operator. We use this set, denoted a2a4a3 a10 , to arrive at an admissible heuristic for state a1 under a language model a21a1a0 , using Equation 2:</Paragraph> <Paragraph position="4"> If a19 a30 a14 a1 a22 is the true future cost for state a1 , we guarantee that a19 a14 a1 a22a52a51 a19 a30a25a14 a1 a22 from the way a2a4a3 a10 and a2a16a13a15a3 a10 are constructed. Note that, as it usually happens with admissible heuristics, we can make a19 a14 a1 a22 come arbitrarily close to a19 a30a25a14 a1 a22 , by computing increasingly better approximations a2a16a13a15a3 a10 of a2a14a13a15a3 a30a10 . Such approximations, however, require increasingly advanced unfoldings of the IDL-graph a18 (a complete unfolding of a18 for state a1 gives a2a14a13a15a3 a10 a20 lows that arbitrarily accurate admissible heuristics exist for IDL-expressions, but computing them on-the-fly requires finding a balance between the time and space requirements for computing better heuristics and the speed-up obtained by using them in the search algorithms.</Paragraph> </Section> <Section position="4" start_page="70" end_page="71" type="sub_section"> <SectionTitle> 3.4 Formal Properties of IDL-NGLM algorithms </SectionTitle> <Paragraph position="0"> The following theorem states the correctness of our algorithms, in the sense that they find the maximum probability path encoded by an IDL-graph under an n-gram language model.</Paragraph> <Paragraph position="1"> Theorem 1 Let a19 be an IDL-expression, G(a19 ) its IDL-graph, and W(a19 ) its wFSA under an n-gram language model LM. Algorithms IDL-NGLM-BFS and IDL-NGLM-Aa30 find the 1Actually, these are multisets, as we treat multiply-occurring labels as separate items.</Paragraph> <Paragraph position="2"> path of maximum probability under LM. Algorithm IDL-NGLM-BEAM finds the path of maximum probability under LM, if all states in W(a19 ) along this path are selected by its BEAMSTATES function. The proof of the theorem follows directly from the correctness of the BFS and Aa30 search, and from the condition imposed on the beam search.</Paragraph> <Paragraph position="3"> The next theorem characterizes the run-time complexity of these algorithms, in terms of an input IDL-expression a19 and its corresponding IDL-graph a18 a14a20a19a24a22 complexity. There are three factors that linearly influence the run-time complexity of our algorithms: a0 is the maximum number of nodes in a18 a14a20a19a24a22 needed to represent a state in a1 a14a20a19a24a22 -a0 depends solely on a19 ; a1 is the maximum number of nodes in a18 a14a20a19a24a22 needed to represent a state in a18 a14a20a19a24a22 -a1 depends on a19 and a12 , the length of the context used by the a12 -gram language model; anda2 is the number of states of a18 a14a20a19a24a22 -a2 also depends on a19 and a12 . Of these three factors, a2 is by far the predominant one, and we simply call a2 the complexity of an IDL-expression.</Paragraph> <Paragraph position="4"> Theorem 2 Let a19 be an IDL-expression, a18 a14a20a19a24a22 its IDL-graph, a1 a14a20a19a24a22 its FSA, and a18 a14a20a19a24a22 its wFSA under an n-gram language model. Let a3 a3 We omit the proof here due to space constraints. The fact that the run-time behavior of our algorithms is linear in the complexity of the input IDL-expression (with an additional log factor in the case of Aa30 search due to priority queue management) allows us to say that our algorithms are efficient with respect to the task they accomplish.</Paragraph> <Paragraph position="5"> We note here, however, that depending on the input IDL-expression, the task addressed can vary in complexity from linear to exponential. That is, for the intersection of an IDL-expression a19 a20 plexity. This exponential complexity comes as no surprise given that the problem of intersecting an n-gram language model with a bag of words is known to be NP-complete (Knight, 1999). On the other hand, for intersecting an IDL-expression a19 a20 a1 a8 a2 a14 a12a27a22 generation algorithm.</Paragraph> <Paragraph position="6"> In general, for IDL-expressions for which a0 is bounded, which we expect to be the case for most practical problems, our algorithms perform in polynomial time in the number of words available for generation.</Paragraph> </Section> </Section> <Section position="5" start_page="71" end_page="72" type="metho"> <SectionTitle> 4 Evaluation of IDL-NGLM Algorithms </SectionTitle> <Paragraph position="0"> In this section, we present results concerning the performance of our algorithms on a word-ordering task. This task can be easily defined as follows: from a bag of words originating from some sentence, reconstruct the original sentence as faithfully as possible. In our case, from an original sentence such as &quot;the gifts are donated by american companies&quot;, we create the IDL-expressiona33a19a5a34 a2 a3a15a14a46a45a48a47a7a49 a18 a24a25a52a14a26a45a48a54a10a18a9a64 a56a10a37a40a39a25a45 a49a63a64a31a18a9a58a63a56a35a34a57a50a17a39 a37 a52 a49a53a54a25a18a22a33a26a43a40a18a9a39a10a51a63a49 a18a9a39 a34 a49a26a51a53a52a16a58a34a39a25a37 a22 a2 a33a36a35a35a19a5a34, from which some algorithm realizes a sentence such as &quot;donated by the american companies are gifts&quot;. Note the natural way we represent in an IDL-expression beginning and end of sentence constraints, using the a2 operator. Since this is generation from bag-of-words, the task is known to be at the high-complexity extreme of the run-time behavior of our algorithms. As such, we consider it a good test for the ability of our algorithms to scale up to increasingly complex inputs.</Paragraph> <Paragraph position="1"> We use a state-of-the-art, publicly available toolkit2 to train a trigram language model using Kneser-Ney smoothing, on 10 million sentences (170 million words) from the Wall Street Journal (WSJ), lower case and no final punctuation. The test data is also lower case (such that upper-case words cannot be hypothesized as first words), with final punctuation removed (such that periods cannot be hypothesized as final words), and consists of 2000 unseen WSJ sentences of length 3-7, and 2000 unseen WSJ sentences of length 10-25.</Paragraph> <Paragraph position="2"> The algorithms we tested in this experiments were the ones presented in Section 3.2, plus two baseline algorithms. The first baseline algorithm, L, uses an inverse-lexicographic order for the bag items as its output, in order to get the word the on sentence initial position. The second baseline algorithm, G, is a greedy algorithm that realizes sentences by maximizing the probability of joining any two word sequences until only one sequence is left.</Paragraph> <Paragraph position="3"> For the Aa30 algorithm, an admissible cost is computed for each state a1 in a weighted finite-state automaton, as the sum (over all unused words) of the minimum language model cost (i.e., maximum probability) of each unused word when conditioning over all sequences of two words available at that particular state for future conditioning (see Equation 2, with the beam algorithm for deciding which IDL-graph nodes are not unfolded. We also test a greedy version of the Aa30 algorithm, denoted Aa30a10 , which considers for unfolding only the nodes extracted from the priority queue which already unfolded a path of length greater than or equal to the maximum length already unfolded minus a0 (in this notation, the Aa30 algorithm would be denoted Aa30a1 ). For the beam algorithms, we use the notation Ba2 to specify a probabilistic beam of size a2 , i.e., an algorithm that beams out the states reachable with probability less than the current maximum probability times a2 .</Paragraph> <Paragraph position="4"> Our first batch of experiments concerns bags-of-words of size 3-7, for which exhaustive search is possible. In Table 2, we present the results on the word-ordering task achieved by various algorithms.</Paragraph> <Paragraph position="5"> We evaluate accuracy performance using two automatic metrics: an identity metric, ID, which measures the percent of sentences recreated exactly, and BLEU (Papineni et al., 2002), which gives the geometric average of the number of uni-, bi-, tri-, and four-grams recreated exactly. We evaluate the search performance by the percent of Search Errors made by our algorithms, as well as a percent figure of Estimated Search Errors, computed as the percent of searches that result in a string with a lower probability than the probability of the original sentence. To measure the impact of using IDL-expressions for this task, we also measure the percent of unfolding of an IDL graph with respect to a full unfolding. We report speed results as the average number of seconds per bag-of-words, when using a 3.0GHz CPU machine under a Linux OS.</Paragraph> <Paragraph position="6"> The first notable result in Table 2 is the savings savings (Unfold), and speed results.</Paragraph> <Paragraph position="7"> achieved by the Aa30 algorithm under the IDL representation. At no cost in accuracy, it unfolds only 12% of the edges, and achieves a 7 times speedup, compared to the BFS algorithm. The savings achieved by not unfolding are especially important, since the exponential complexity of the problem is hidden by the IDL representation via the folding mechanism of the a3 operator. The algorithms that find sub-optimal solutions also perform well. While maintaining high accuracy, the Aa30</Paragraph> <Paragraph position="9"> rithms unfold only about 5-7% of the edges, at 12-14 times speed-up.</Paragraph> <Paragraph position="10"> Our second batch of experiments concerns bag-of-words of size 10-25, for which exhaustive search is no longer possible (Table 3). Not only exhaustive search, but also full Aa30 search is too expensive in terms of memory (we were limited to 2GiB of RAM for our experiments) and speed. Only the greedy</Paragraph> </Section> <Section position="6" start_page="72" end_page="72" type="metho"> <SectionTitle> versions Aa30 </SectionTitle> <Paragraph position="0"> and Aa30a2 , and the beam search using tight probability beams (0.2-0.1) scale up to these bag sizes. Because we no longer have access to the string of maximum probability, we report only the percent of Estimated Search Errors. Note that, in terms of accuracy, we get around 20% Estimated Search Errors for the best performing algorithms (Aa30a2 and Ba5a4a3 a8 ), which means that 80% of the time the algorithms are able to find sentences of equal or better probability than the original sentences.</Paragraph> </Section> <Section position="7" start_page="72" end_page="73" type="metho"> <SectionTitle> 5 Conclusions </SectionTitle> <Paragraph position="0"> In this paper, we advocate that IDL expressions can provide an adequate framework for develop- null BLEU), Estimated Search Errors, and speed results. ing text-to-text generation capabilities. Our contribution concerns a new generation mechanism that implements intersection between an IDL expression and a probabilistic language model. The IDL formalism is ideally suited for our approach, due to its efficient representation and, as we show in this paper, efficient algorithms for intersecting, scoring, and ranking sentence realizations using probabilistic language models.</Paragraph> <Paragraph position="1"> We present theoretical results concerning the correctness and efficiency of the proposed algorithms, and also present empirical results that show that our algorithms scale up to handling IDL-expressions of high complexity. Real-world text-to-text generation tasks, such as headline generation and machine translation, are likely to be handled graciously in this framework, as the complexity of IDL-expressions for these tasks tends to be lower than the complexity of the IDL-expressions we worked with in our experiments.</Paragraph> </Section> class="xml-element"></Paper>