<?xml version="1.0" standalone="yes"?>
<Paper uid="W06-2607">
  <Title>Tree Kernel Engineering in Semantic Role Labeling Systems</Title>
  <Section position="6" start_page="54" end_page="55" type="concl">
    <SectionTitle>
5 Discussions and Conclusions
</SectionTitle>
    <Paragraph position="0"> The design of automatic systems for the labeling of semantic roles requires the solution of complex problems. Among others, feature engineering is made difficult by the structural nature of the data, i.e. features should represent information contained in automatic parse trees. This raises two problems: (1) the modeling of effective features, partially solved in the literature work and (2) the implementation of the software for the extraction of a large number of such features.</Paragraph>
    <Paragraph position="1"> A system completely based on tree kernels alleviate both problems as (1) kernel functions automatically generate features and (2) only a procedure for subtree extraction is needed. Although some of the manual designed features seem to be superior to those derived with tree kernels, their combination seems still worth applying.</Paragraph>
    <Paragraph position="2"> In this paper, we have improved tree kernels by studying different strategies: MPAF and the combined classifier (for internal and pre-terminal nodes) highly improve efficiency and accuracy in both the boundary detection and argument classification tasks. In particular, MPAF improves the old PAF-based tree kernel of about 8 absolute percent points in the boundary classification task, and when used along the combined classifier approach the speed of the model increases of 3.5 times. In case of argument classification the improvement is less evident but still consistent, about 2%.</Paragraph>
    <Paragraph position="3"> We have also studied tree representations based on complete argument structures (MSTs). Our preliminary results seem to suggest that additional information extracted from other arguments is not effective. However, such findings are affected by two main problems: (1) We used adjuncts in the tree representation. They are likely to add more noise than useful information for the recognition of the argument type. (2) The traditional PAF contains subtrees that cannot be derived by the  MMSTs, thus we should combine these structures rather than substituting one with the other.</Paragraph>
    <Paragraph position="4"> In the future, we plan to extend this study as follows: First, our results are computed individually for boundary and classification tasks. Moreover, in our experiments, we removed arguments whose PAF or MST could not be extracted due to errors in parse trees. Thus, we provided only indicative accuracy to compare the different tree kernels. A final evaluation of the most promising structures using the CoNLL 2005 evaluator should be carried out to obtain a sound evaluation.</Paragraph>
    <Paragraph position="5"> Second, as PAFs and MSTs should be combined to generate more information, we are going to carry out a set of experiments that combine different kernels associated with different subtrees. Moreover, as shown in (Basili and Moschitti, 2005; Moschitti, 2006), there are other tree kernel functions that generate different fragment types. The combination of such functions with the marking strategies may provide more general and effective kernels.</Paragraph>
    <Paragraph position="6"> Third, once the final set of the most promising kernels is established, we would like to use all the available CoNLL 2005 data. This would allow us tostudythepotentialityofourapproachbyexactly comparing with literature work.</Paragraph>
    <Paragraph position="7"> Next, our fast tree kernel function along with the combined classification approach and the improved tree representation make the learning and classification much faster so that the overall running time is comparable with polynomial kernels. However, when these latter are used with SVMs the running time is prohibitive when very large datasets (e.g. millions of instances) are targeted. Exploiting tree kernel derived features in a more efficient way is thus an interesting line of future research.</Paragraph>
    <Paragraph position="8"> Finally, as CoNLL 2005 has shown that the most important contribution relates on re-ranking predicate argument structures based on one single tree (Toutanova et al., 2005) or several trees (Punyakanok et al., 2005), we would like to use tree kernels for the re-ranking task.</Paragraph>
  </Section>
class="xml-element"></Paper>