<?xml version="1.0" standalone="yes"?>
<Paper uid="W00-0701">
  <Title>Learning in Natural Language: Theory and Algorithmic Approaches*</Title>
  <Section position="6" start_page="4" end_page="4" type="concl">
    <SectionTitle>
5 Future Research Issues
</SectionTitle>
    <Paragraph position="0"> Research on learning in NLP needs to be integrated with work on knowledge representation and inference to enable studying higher level NL tasks. We mention two important directions the implications on the learning issues.</Paragraph>
    <Paragraph position="1"> The unified view presented reveals that all methods blow up the dimensionality of the original space in essentially the same way; they generate conjunctive features over the linear structure of the sentence (i.e., n-gram like features in the word and/or pos space).</Paragraph>
    <Paragraph position="2"> This does not seem to be expressive enough.</Paragraph>
    <Paragraph position="3"> Expressing complex concepts and relations necessary for higher level inferences will require more involved intermediate representations (&amp;quot;features&amp;quot;) over the input; higher order structural and semantic properties, long term dependencies and relational predicates need to be represented. Learning will stay manageable if done in terms of these intermediate representations as done today, using functionally simple representations (perhaps cascaded).</Paragraph>
    <Paragraph position="4"> Inductive logic programming (MDR94; Coh95) is a natural paradigm for this. However, computational limitations that include both learnability and subsumption render this approach inadequate for large scale knowledge intensive problems (KRV99; CR00).</Paragraph>
    <Paragraph position="5"> In (CR00) we suggest an approach that addresses the generation of complex and relational intermediate representations and supports efficient learning on top of those. It allows the generation and use of structured examples which could encode relational information and long term functional dependencies. This is done using a construct that defines &amp;quot;types&amp;quot; of (potentially, relational) features the learning process might use. These represent infinitely many features, and are not generated explicitly; only those present in the data are generated, on the fly, as part of the learning process. Thus it yields hypotheses that are as expressive as relational learners in a scalable fashion. This approach, however, makes some requirements on the learning process. Most importantly, the learning approach needs to be able to process variable size examples. And, it has to be feature efficient in that its complexity depends mostly on the number of relevant features. This seems to favor the SNoW approach over other algorithms that learn the same representation.</Paragraph>
    <Paragraph position="6"> Eventually, we would like to perform inferences that depend on the outcomes of several different classifiers; together these might need to coherently satisfy some constrains arising from the sequential nature of the data or task and domain specific issues. There is a need to study, along with learning and knowledge representation, inference methods that suit this framework (KR97). Work in this direction requires a consistent semantics of the learners (Val99) and will have implications on the knowledge representations and learning methods used. Preliminary work in (PRO0) suggests several ways to formalize this problem and is evaluated in the context of identifying phrase structure.</Paragraph>
  </Section>
class="xml-element"></Paper>