<?xml version="1.0" standalone="yes"?>
<Paper uid="W03-0602">
  <Title>Words and Pictures in the News</Title>
  <Section position="2" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 Introduction
</SectionTitle>
    <Paragraph position="0"> For the past year we have been building a collection of captioned news photos and illustrated news articles. We believe that, for many applications, words and pictures together provide very rich information on document content. Photographs can link articles in ways that pure textual analysis may overlook or underestimate, and text provides high level descriptions of image contents that current vision techniques cannot obtain.</Paragraph>
    <Paragraph position="1"> Our analysis of image captions has revealed various journalistic conventions that we believe make this dataset particularly appealing for a number of applications. Captions act as concise summaries of events, and we believe we can use them to isolate the most pertinent words for different topics. We have implemented various clustering methods to explore this idea.</Paragraph>
    <Paragraph position="2"> Captions are also tightly tied to the actual content of the image. The difficulty here on the text side, is in identifying those portions of a caption that refer to tangible objects, physically present in the image. On the image side, we need to isolate objects of interest and solve the correspondence problem between caption extracts and image regions. We are exploring these issues in the context of trying to build an automated celebrity directory and face classifier.</Paragraph>
  </Section>
class="xml-element"></Paper>