<?xml version="1.0" standalone="yes"?>
<Paper uid="E91-1003">
  <Title>DESIGNING ILLUSTRATED TEXTS: HOW LANGUAGE PRODUCTION IS INFLUENCED BY GRAPHICS GENERATION</Title>
  <Section position="3" start_page="0" end_page="0" type="intro">
    <SectionTitle>
1 INTRODUCTION
</SectionTitle>
    <Paragraph position="0"> With increases in the amount and sophistication of information that must be communicated to the users of complex technical systems comes a corresponding need to find new ways to present that information flexibly and efficiently. Intelligent presentation systems are important building blocks of the next generation of user interfaces, as they translate from the narrow output channels provided by most of the current application systems into high-bandwidth communications tailored to the individual user. Since in many situations information is only presented efficiently through a *The WlP project is supported by the German Ministry of Research and Technology under grant ITW8901 8. We would like to thank Doug Appelt, Steven Feiner and Ed Hovy for stimulating discussions about multimodal information presentation.</Paragraph>
    <Paragraph position="1"> ,,, , i ,,,,,  In the project WIP, we try to generate on the fly illustrated texts that are customized for the intended target audience and situation, flexibly presenting information whose content, in contrast to hypermedia systems, cannot be fully anticipated. The current testbed for WIP is the generation of instructions for the use of an espresso-machine. It is a rare instruction manual that does not contain  -8illustrations. WIP's 2D display of 3D graphics of machine parts help the addressee of the synthesized multimodal presentation to develop a 3D mental model of the object that he can constantly match with his visual perceptions of the real machine in front of him. Fig. 1 shows a typical text-picture sequence which may be used to instruct a user in filling the watercontainer of an espresso-machine. Currently, the technical knowledge to be presented by WIP is encoded in a hybrid knowledge representation language of the KL-ONE family including a terminological and assertional component (see Nebel 90). In addition to this propositional representation, which includes the relevant information about the structure, function, behavior, and use of the espresso-machine, WIP has access to an analogical representation of the geometry of the machine in the form of a wireframe model.</Paragraph>
    <Paragraph position="2"> The automatic design of multimodal presentations has only recently received significant attention in artificial intelligence research (cf. the projects SAGE (Roth et al. 89), COMET (Feiner &amp; McKeown 89), FN/ANDD (Marks &amp; Reiter 90) and WlP (Wahlster et al. 89)). The WIP and COMET projects share a strong research interest in the coordination of text and graphics. They differ from systems such as SAGE and FN/ANDD in that they deal with physical objects (espresso-machine, radio vs. charts, diagrams) that the user can access directly. For example, in the WIP project we assume that the user is looking at a real espresso-machine and uses the presentations generated by WlP to understand the operation of the machine. In spite of many similarities, there are major differences between COMET and WIP, e.g., in the systems' architecture. While during one of the final processing steps of COMET the layout component combines text and graphics fragments produced by mode-specific generators, in WIP a layou\[ manager can interact with a presentation planner before text and graphics are generated, so that layout considerations may influence the early stages of the planning process and constrain the mode-specific generators.</Paragraph>
  </Section>
class="xml-element"></Paper>