<?xml version="1.0" standalone="yes"?>
<Paper uid="P98-2209">
  <Title>Reactive Content Selection in the Generation of Real-time Soccer Commentary</Title>
  <Section position="4" start_page="1282" end_page="1284" type="metho">
    <SectionTitle>
3 Brief Description of MIKE's Design
</SectionTitle>
    <Paragraph position="0"> A detailed description, of MIKE, especially its soccer game analysis capabilities can be found in (Tanaka-Ishii et al., 1998). Here we simply give a brief overview.</Paragraph>
    <Section position="1" start_page="1282" end_page="1284" type="sub_section">
      <SectionTitle>
3.1 MIKE's Structure
</SectionTitle>
      <Paragraph position="0"> MIKE, 'Multi-agent Interactions Knowledgeably Explained', is designed to produce simultaneous commentary for the Soccer Server, originally proposed as a standard evaluation method for multi-agent systems(Noda and Matsubara, 1996). The Soccer Server provides a real-time game log 1 of a very high quality, sending information on the positions of the players and the ball to a monitoring program every 100msec. Specifically, this information consists of:  * player location and orientation, * ball location, * game score and play modes (such as throw ins, goal kicks, etc ).</Paragraph>
      <Paragraph position="1">  From this low-level input, the current implementation of MIKE can generate the range of comments shown in Figure 1.</Paragraph>
      <Paragraph position="2">  system 2 __ is shown in Figure 2. Here, the ovals represent concurrently running modules and the rectangles represent data.</Paragraph>
      <Paragraph position="3"> All communication among modules is mediated by the internal symbolic representation of commentary 2In natural language processing, the multi-agent approach dates back to Hearsay-II (Erraan et al., 1980), which was the first to use the blackboard architecture. The core organization of MIKE, however, is more akin to a subsumption architecture (Brooks, 1991), because the agents are regarded as behavior modules which are both directly connected to the external environment (through sensor readings from the shared memory) and can directly produce system behavior (by suggesting commentary). However, MIKE does not exactly fit the subsumption architecture model because the agents can also communicate with each other: there are some portions of the shared memory that are global and some that are exported to only a limited number of agents. This division of shared memory leads to more possibilities for inter-agent communication.  fragments, which we call propositions. A proposition is represented with a tag and some attributes. For example, a kick by player No.5 is represented as (Kick 5), where Kick is the tag and 5 is the attribute. So far, MIKE has around 80 sorts of tags, categorized in two ways: as being local or global and as being state-based or event-based. Table 1 shows some examples of categorized proposition tags.</Paragraph>
      <Paragraph position="4"> Some of the important modules in MIKE'S architecture can be summarized as follows.</Paragraph>
      <Paragraph position="5"> There are six Soccer Analyzers that try to interpret the game. Three of these analyze events (shown in the figure as the 'kick analysis', 'pass work', and 'shoot' modules). The other three carry out state-based analysis (shown as the 'basic strategy', 'formation', and 'play area' modules). The modules analyze the data from the Soccer Server, communicate with each other via the shared memory, and then post the results as propositions into the Pool.</Paragraph>
      <Paragraph position="6"> The Real Time Inference Engine processes the propositions. Prpositions deposited in the Pool are bare facts and are often too detailed to be used as comments. MIKE therefore uses forward chaining rules of the form precedents---, antecedents to draw further inferences. The types of rules used for this process are shown in Figure 3. Currently, MIKE has about 110 such rules.</Paragraph>
      <Paragraph position="7"> The Natural Language Generator selects the proposition from the Pool that best fits the current state of the game (considering both the situation on the field and the comment currently being made).</Paragraph>
      <Paragraph position="8"> It then translates the proposition into NL. So far, MIKE just carries out this final step with the simple mechanism of template-matching. Several templates are prepared for each proposition tag, and the out- null of a proposition put can be is in English, French or Japanese.</Paragraph>
      <Paragraph position="9"> To produce speech, MIKE uses off the shelf text-to-speech software. English is produced by</Paragraph>
    </Section>
    <Section position="2" start_page="1284" end_page="1284" type="sub_section">
      <SectionTitle>
4.1 Importance of a Proposition
</SectionTitle>
      <Paragraph position="0"> The Soccer Analyzers attach an importance score to a proposition, which intuitively captures the amount of information that the proposition would transmit to an audience.</Paragraph>
      <Paragraph position="1"> The importance score of a proposition is planned to change over time as follows (Figure 4). After being posted to the Pool, the score decreases over time while it remains in the Pool waiting to be uttered.</Paragraph>
      <Paragraph position="2"> When the importance score of a proposition reaches zero, it is deleted. This decrease in importance models the way that an event's relevance decreases as the game progresses.</Paragraph>
      <Paragraph position="3"> The rate at which importance scores decrease can be modeled by any monotonic function. For simplicity, MIKE'S function is currently linear. Since it seems sensible that local propositions should lose their score more quickly than global ones, several functions with different slopes are used, depending on the degree to which a proposition can be considered local or global. When a proposition is used for utterance or inference, the score is reduced in order to avoid the redundant use of the same proposition, but not set to zero, thus leaving a small chance for other inferences.</Paragraph>
      <Paragraph position="4"> There is also an initialization process for the importance scores as follows. First, to reflect the situation of the game, the local propositions are modified by a multiplicative factor depending on the state of the game. This factor is designed so that local propositions are more important when the ball is near the goal. Global propositions are always initialized with the default value.</Paragraph>
      <Paragraph position="5"> Secondly, to reflect the topic of the discourse, MIKE has a feedback control which enables each Soccer Analyzer module to take into account MIKE's past and present utterances. The NL generator broadcasts the current subject to the agents and they assign greater initial importance scores to propositions with related subjects. For example, when MIKE is talking about player No.5, the Analyzers assign a higher importance to propositions relating to this player No.5.</Paragraph>
    </Section>
    <Section position="3" start_page="1284" end_page="1284" type="sub_section">
      <SectionTitle>
4.2 Maximization of the Importance Score
</SectionTitle>
      <Paragraph position="0"> As the importance score is designed to intuitively reflect the information transmitted to the audience, the natural application of our content selection principles described in SS2 is simply to attempt to maximize the total importance of all the propositions that are selected for utterance.</Paragraph>
      <Paragraph position="1"> MIKE has the very basic function of uttering the most important content at any given time. That is, MIKE repeatedly selects the proposition with the largest importance score in the Pool.</Paragraph>
      <Paragraph position="2"> The NL Generator translates the selected proposition into a natural language expression and sends it to the TTS-administrator module. Then the NL Generator has to wait until the Text-to-Speech software finishes the utterance before sending out the next expression. During this time lag, however, the game situation might rapidly unfold and numerous further propositions may be posted to the Pool. It is to cope with this time lag that MIKE implements a alternative function, that allows a more flexible selection of propositions by modeling the processes of interruption, abbreviation, and repetition, Interruption If a proposition with a much larger importance score than the one currently being uttered is inserted into the Pool, the total importance score may become larger by immediately interrupting the current utterance and switching to the new one. For example, the left of Figure 5 shows (solid line) the change of the importance score with time when an interruption takes place (the dotted line. represents the importance score without interruption). The left part of the solid line is lower than the dotted, because the first utterance conveys less of its importance score (information) when it is not completely uttered. The right part of the dotted line is lower than that of the solid, because the importance of the second utterance decreases over time when waiting  to be selected.</Paragraph>
      <Paragraph position="3"> Thus, the sum of the importance of the uttered propositions can no longer be used to access the system's performance. Instead, the area between the lines and the horizontal axis indicates the total importance score over time. Whether or not to make interruption should be decided by comparing two areas made by the solid and dotted, and the larger area size is the total importance score gain. Further, this selection decides what to be said and how at the same time.</Paragraph>
      <Paragraph position="4"> Note that interruptions raise the importance score gain by reacting sharply to the sudden increase of the importance score.</Paragraph>
      <Paragraph position="5"> Abbreviation If the two most important propositions in the Pool are of similar importance, it is possible that the amount of communicated information could be maximized by quickly uttering the most important proposition and then moving on to the second before loses importance due to some development of the game situation. In the Figure 5, we have illustrated this in the same way we did for the case of interruption. The left hand side of the solid line is lower than that of the dotted because an abbreviated utterance (which might not be grammatically correct, or whose context might not be fully given) transmits less information than a more complete utterance. As the second proposition can be uttered before losing its importance score, however, the right hand part of the solid line is higher than that of the dotted. As before, the benefits or otherwise of this modification should be decided by comparing with Red3 collects the ball from Red$, Red3, Red-Team, wonderful goal! P to ~! Red3's great center shot! Equal! The Red-Team's formation is now breaking through enemy line from center, The Red-Team's counter attack (Red4 near at the center line made a long pass towards Red3 near the goal and he made a shot very swiftly.), Red3's goal! Kick o~, Yellow-Team, Red1 is very active because, Red1 always takes good positions, Second hall o\] RoboCup'9? quaterfinal(Some background is described while the ball is in the mid field.) Left is Ohta Team, Japan, Right is Humboldt, Germany, Red1 takes the ball, bad pass, (Yellow team's play after kick off was interrupted by Read team) Interception by the Yellow-Team, Wonderful dribble, YellowP, YellowP (Yellow6 approaches Yellow2 for guard), Yellow6's pass, A pass through the opponents' defense, Red6 can take the ball, because, Yellow6 is being marked by Red6, .... The Red- Team's counter attack, The Red- Team's \]ormation is (system's interruption), Yellow5, Back pass of YellowlO, Wonderful pass,  the two areas made by the solid and the dotted line with the horizontal axis. Again, this selection decides how and what-to-say at the same point.</Paragraph>
      <Paragraph position="6"> In this case we would hope that abbreviations raise the importance score by smoothing sudden decreases of the importance scores posted to the Pool. Repetition Whenever a proposition is selected to be uttered, its importance value is decreased. It is also marked as having been uttered, to prevent its re-use. However, sometimes it can happen that the remaining unuttered propositions in the Pool have much smaller values than any of those that have already been selected. In this case, we investigate the effects of allowing previously uttered propositions to be repeated. null</Paragraph>
    </Section>
  </Section>
  <Section position="5" start_page="1284" end_page="1284" type="metho">
    <SectionTitle>
5 Evaluation
</SectionTitle>
    <Paragraph position="0"/>
    <Section position="1" start_page="1284" end_page="1284" type="sub_section">
      <SectionTitle>
5.1 Output Example
</SectionTitle>
      <Paragraph position="0"> An example of MIKE's commentary (when employing interruption, abbreviation and repetition) is shown in Figure 6. In practice, this output is designed to accompany a visual game, but it is impractical to reproduce here enough screen-shots to describe the course of the play. We have therefore instead included some context and further explanations in parentheses. This particular commentary  Rouge3, 2e but. Score de 2~ P. Tit du centre par Rouge3 ! Egalite ! Rouge3, but / Attaque rapide de l'dquipe rouge, JaunelO, La formation de l'gquipe jaune est basde sur l'attaque par le centre. L 'dquipe japonaise a gagng dans le Groupe C du deuxidme Tour, tandis que l'dquipe allemande a gagng dans le Groupe D. Rouge1 prend la baUe, mauvaise passe C'est l'gquipe jaune qui relance le jeu, Magnifique dribble du JauneP, Passe pour JauneS. Est-ce que  covers a roughly 20 second period of a quater-final from RoboCup'97.</Paragraph>
      <Paragraph position="1"> For comparison, we have included MIKE'S French and Japanese descriptions of the same game period in Figure 8 and Figure 7. In general, the generated commentary differs because of the timing issues resulting from two factors: agent concurrency and the length of the NL-templates. One NL template is randomly chosen from several candidates at translation time and it is the length of this template that decides the timing of the next content selection.</Paragraph>
    </Section>
    <Section position="2" start_page="1284" end_page="1284" type="sub_section">
      <SectionTitle>
5.2 Effect of Rearrangements
Importance Score Increase
</SectionTitle>
      <Paragraph position="0"> Figure 9 plots the importance score of the Propositions in MIKE'S commentary for the some RoboCup'97 quater-final we used in the previous section. The horizontal axis indicates time unit of 100msec and the vertical axis the importance score of the comment being uttered (taking into account reductions due to interruption, abbreviation, or repeated use of a proposition). The solid line describes the importance score change with interruption, abbreviation and repetition, whereas the dotted shows that without such rearrangements. As we described in SS4, the area between the plotted lines and the</Paragraph>
    </Section>
  </Section>
  <Section position="6" start_page="1284" end_page="1284" type="metho">
    <SectionTitle>
RoboCup'97 quater-final game
</SectionTitle>
    <Paragraph position="0"> horizontal axis indicates the total importance score.</Paragraph>
    <Paragraph position="1"> Two observations: * The graph peaks generally occur earlier for the solid line than for the dotted. This indicates that the commentary with rearrangements is more timely than the commentary that repeatedly selects the most important proposition. For instance, the peaks caused by a goal around time 2200 spread out for the dotted line, which is not the case for the solid line. Also, the peaks are higher for the solid line than dotted.</Paragraph>
    <Paragraph position="2"> * The area covered by the solid line is larger than that by the dotted, meaning that the total importance score is greater with rearrangements.</Paragraph>
    <Paragraph position="3"> During this whole game, the total importance score with rearrangements amounted 9.90% more than that without.</Paragraph>
    <Paragraph position="4"> Decrease of Delayed Utterances As a further experiments, we manually annotated each statement in the Japanese output for the RoboCup'9? quater-final with it optimal time for utterance. We then calculated the average delay in the appearance of these statements in MIKE'S commentary both with and without rearrangements. We found that adding the rearrangements decreased this delay from 2.51sec to 2.16sec , a improvement at about 14%.</Paragraph>
  </Section>
  <Section position="7" start_page="1284" end_page="1284" type="metho">
    <SectionTitle>
6 Related Works
</SectionTitle>
    <Paragraph position="0"> (Suzuki et al., 1997) have proposed new interaction styles to replace conventional goal-oriented dialogues. Their multi-agent dialogue system that chats with a human considers topics and goals as being situated within the context of interactions among participants. Their model of context handling is an adaptation of a subsumption architecture. One important common aspect between their system and MIKE is that the system itself creates topics.</Paragraph>
    <Paragraph position="1"> The SOCCER system described in (Andr~ et al., 1994), combines a vision system with an intelligent multimedia generation system to provide commentary on 'short sections of video recordings of real soccer games'. The system is built on VITRA, which uses generalized simultaneous scene description to produce concurrent image sequence evaluation and natural language processing. The vision system translates TV images into information and the intelligent multimedia generation module then takes this information and presents it by combining media such as text, graphics and video.</Paragraph>
  </Section>
class="xml-element"></Paper>