<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-2019">
  <Title>Sydney, July 2006. c(c)2006 Association for Computational Linguistics Constraint-based Sentence Compression An Integer Programming Approach</Title>
  <Section position="7" start_page="149" end_page="150" type="evalu">
    <SectionTitle>
5 Results
</SectionTitle>
    <Paragraph position="0"> Our results are summarised in Table 3 which details the compression rates6 and average human  term compression rate to refer to the percentage of words O: Apparently Fergie very much wants to have a career in television.</Paragraph>
    <Paragraph position="1"> G: Fergie wants a career in television.</Paragraph>
    <Paragraph position="2"> D: A career in television.</Paragraph>
    <Paragraph position="3"> LM: Fergie wants to have a career.</Paragraph>
    <Paragraph position="4"> Sig: Fergie wants to have a career in television. O: The SCAMP module, designed and built by Unisys and based on an Intel process, contains the entire 48-bit A-series processor.</Paragraph>
    <Paragraph position="5"> G: The SCAMP module contains the entire 48-bit A-series processor.</Paragraph>
    <Paragraph position="6"> D: The SCAMP module designed Unisys and based on an Intel process.</Paragraph>
    <Paragraph position="7"> LM: The SCAMP module, contains the 48-bit A-series processor.</Paragraph>
    <Paragraph position="8"> Sig: The SCAMP module, designed and built by Unisys and based on process, contains the A-series processor.</Paragraph>
    <Paragraph position="9">  (CompR) and average human judgements (Rating); /: sig. diff. from gold standard; : sig. diff. from LangModel+Signi cance ratings (Rating) for the three systems and the gold standard. As can be seen, the IP language model (LangModel) is most aggressive in terms of compression rate as it reduces the original sentences on average by half (49%). Recall that we enforce a minimum compression rate of 40% (see (22)). The fact that the resulting compressions are longer, indicates that our constraints instill some linguistic knowledge into the language model, thus enabling it to prefer longer sentences over extremely short ones. The decision-tree model compresses slightly less than our IP language model at 56.1% but still below the gold standard rate. We see a large compression rate increase from 49% to 73.6% when we introduce the signi cance score into the objective function. This is around 10% higher than the gold standard compression rate.</Paragraph>
    <Paragraph position="10"> We now turn to the results of our elicitation study. We performed an Analysis of Variance (ANOVA) to examine the effect of different system compressions. Statistical tests were carried out on the mean of the ratings shown in Table 3. We observe a reliable effect of compression type by subretained in the compression.</Paragraph>
    <Paragraph position="12"> tests revealed that gold standard compressions are perceived as signi cantly better than those generated by all automatic systems (a &lt; 0.05). There is no signi cant difference between the IP language model and decision-tree systems. However, the IP model with the signi cance score delivers a signi cant increase in performance over the language model and the decision tree (a &lt; 0.05).</Paragraph>
    <Paragraph position="13"> These results indicate that reasonable compressions can be obtained with very little supervision. Our constraint-based language model does not make use of a parallel corpus, whereas our second variant uses only 50 parallel sentences for tuning the weights of the objective function. The models described in this paper could be easily adapted to other domains or languages provided that syntactic analysis tools are to some extent available.</Paragraph>
  </Section>
class="xml-element"></Paper>