<?xml version="1.0" standalone="yes"?>
<Paper uid="P06-2116">
  <Title>Sydney, July 2006. c(c)2006 Association for Computational Linguistics A Grammatical Approach to Understanding Textual Tables using Two-Dimensional SCFGs</Title>
  <Section position="7" start_page="911" end_page="911" type="evalu">
    <SectionTitle>
5 Results and Discussion
</SectionTitle>
    <Paragraph position="0"> The experimental results are summarized in Table 10. Alltablescouldbeparsed; ingeneral,itisvery rareforanytabletoberejectedbytheparser, since the grammar permits so many different configurations that can be recursively composed.</Paragraph>
    <Paragraph position="1"> Unfortunately it is impossible to compare results directly against previous models, since neither those models nor the data they evaluated on are available.</Paragraph>
    <Paragraph position="2"> Moreover, it is difficult to compare with previous models as our evaluation criteria are more stringent than in earlier work. Most previous work evaluated the performance in terms of the (vaguer and less demanding) criteria of number of correct attribute-value pairings. Such an evaluation approachgivesundulyhighweighttolargerepetitive null tables,andneglectsstructuralerrorsintheanalysis of the table. In contrast, our approach gives equal weight to all tables regardless of how many entries they contain, requires semantically valid structural analyses, and yet still accepts any parse that yields the correct attribute-value pairings (since the tagging of the test set includes all legitimate types when there are multiple valid alternatives).</Paragraph>
    <Paragraph position="3"> The fact that precision was lower than recall is due to the fact that many tables were wrongly interpretedastableswithoutschemaorinwrongori- null entations. The current grammar has difficulty distinguishing attributes from values. Significant improvement can be obtained by using constraints to limit the number of incorrect parses, a strategy we are currently implementing.</Paragraph>
  </Section>
class="xml-element"></Paper>