<?xml version="1.0" standalone="yes"?>
<Paper uid="J99-3003">
  <Title>Lucent Technologies Bell Laboratories</Title>
  <Section position="3" start_page="0" end_page="364" type="relat">
    <SectionTitle>
2. Related Work
</SectionTitle>
    <Paragraph position="0"> Call routing is similar to text categorization in identifying which one of n topics (or in the case of call routing, destinations) most closely matches a caller's request. Call routing is distinguished from text categorization by requiring a single destination to be selected, but allowing a request to be refined in an interactive dialogue. The closest previous work to ours is Ittner, Lewis, and Ahn (1995), in which noisy documents produced by optical character recognition are classified against multiple categories. We are further interested in carrying out the routing process using natural, conversational language.</Paragraph>
    <Paragraph position="1"> The only work on natural language call routing to date that we are aware of is that by Gorin and his colleagues (Gorin, Riccardi, and Wright 1997; Abella and Gorin 1997; Riccardi and Gorin 1998), who designed an automated system to route calls to AT&amp;T operators. They select salient phrase fragments from caller requests (in response to the system's prompt of How may I help you ?), such as made a long distance and the area code for, and sometimes including phrases that are not meaningful syntactic or semantic units, such as it on my credit. These salient phrase fragments, which are incorporated into their finite-state language model for their speech recognizer, are then used to compute likely destinations, which they refer to as call types. This is done by either computing a posteriori probabilities for all possible call types (Gorin 1996) or by passing the weighted fragments through a neural network classifier (Wright, Gorin, and Riccardi 1997). Abella and Gorin (1997) utilized the Boolean formula minimization algorithm for combining the resulting set of call types based on a hand-coded hierarchy of call types. This algorithm provides the basis for determining whether or not the goal of the request can be uniquely identified, in order to select from a set of dialogue strategies for response generation.</Paragraph>
    <Paragraph position="2">  (b) Content words 3. Corpus Analysis  To examine human-human dialogue behavior, we analyzed a set of 4,497 transcribed telephone calls involving actual customers interacting with human call operators at a large call center. In the vast majority of these calls, the first customer utterance contained between 1 and 20 words, while the longest first utterance had 131 words. However, these utterances included only a few content words, 2 with almost all calls containing fewer than 10 content words in the initial user utterance. Figures l(a) and l(b) show histograms of call lengths based on total words and content words in the initial user utterance in each call, respectively.</Paragraph>
    <Paragraph position="3"> Figure 2 shows the distribution of calls to the top 23 destinations on a log scale in our corpus. 3 The perplexity of a probability distribution provides a measure of the difficulty of classification of samples drawn from that distribution. Using the estimate of call distribution based on Figure 2, our task perplexity is 6.836. 4 We further analyzed our corpus of calls along two dimensions: the semantics of caller requests and the dialogue actions for operator responses. The analysis of the semantics of caller requests is intended to examine the ways in which users typically express their goal when prompted, and is used to focus on an appropriate subset of the classes of user utterances that the call router should handle automatically (as opposed to transferring to a human operator). The analysis of the dialogue actions for operator responses, on the other hand, is intended to determine the types of responses the call router should be able to provide in response to user utterances in order to help design the response generation component of the call router. The analysis of the corpus along both dimensions was performed by the first author.</Paragraph>
    <Section position="1" start_page="362" end_page="364" type="sub_section">
      <SectionTitle>
3.1 Semantics of Caller Requests
</SectionTitle>
      <Paragraph position="0"> In our corpus, all callers respond to an initial open-ended prompt of /ABC/ banking services call director; how may I direct your call? Their responses varied greatly in their 2 Content words are keywords automatically extracted from the training corpus that are considered relevant for routing purposes. For details on how the list of content words is selected, see Section 4.1.2. 3 These are destinations that received more than 10 calls in the corpus we analyzed. 4 Recall that the entropy of a distribution p is the expected value of the log probability, given by H(p) = - Y'~x p(x) log 2 p(x). The perplexity is given by 2 H(p) and can be thought of roughly as the number of equiprobable categories that would lead to the same classification difficulty.</Paragraph>
      <Paragraph position="2"> Destinations Figure 2 Distribution of calls.</Paragraph>
      <Paragraph position="3"> degree of specificity. We roughly classified the calls into the following three broad classes: Destination Name, in which the caller explicitly specifies the name of the department to which he wishes to be transferred. The requested destination can form an answer to the operator's prompt by itself, as in deposit services, or be part of a complete sentence, as in I would like to speak to someone in auto leasing please.</Paragraph>
      <Paragraph position="4"> Activity, in which the caller provides a description of the activity he wishes to perform, and expects the operator to transfer his call to the appropriate department that handles the given activity. Such descriptions may be ambiguous or unambiguous, depending on the level of detail the caller provides, which in turn depends on the caller's understanding of the organization of the call center. Because all transactions related to savings accounts are handled by the deposit services department in the call center we studied, the request I want to talk to someone about savings accounts will be routed to Deposit Services. On the other hand, the similar request I want to talk to someone about car loans is ambiguous between Consumer Lending, which handles new car loans, and Loan Services, which handles existing car loans. Queries can also be ambiguous due to the caller's providing more than one activity, as in I need to get my checking account balance and then pay a car loan.</Paragraph>
      <Paragraph position="5"> Indirect Request, in which the caller describes his goal in a roundabout way, often including irrelevant information. This typically occurs with callers who are unfamiliar with the call center organization, or those who have difficulty concisely describing their goals. An example of an actual indirect request is ah I'm calling &amp;quot;cuz ah a friend gave me this number and ah she told me ah with this number I can buy some cars or whatever but she didn't know how to explain it to me so I just called you you know to get that information. Table I shows the distribution of caller requests in our corpus with respect to these semantic types. Our analysis shows that in the vast majority of calls, the request was based on either destination name or activity. Since in our corpus there are only 23 dis- null tinct destinations, 5 and each destination only handles a fairly small number (dozens to hundreds) of activities, requests based on destination names and activities are expected to be more predictable and thus more suitable for handling by an automatic call router. However, our system does not directly classify calls in terms of specificity; this classification was only intended to provide a sense of the distribution of calls received.</Paragraph>
    </Section>
    <Section position="2" start_page="364" end_page="364" type="sub_section">
      <SectionTitle>
3.2 Dialogue Actions for Operator Responses
</SectionTitle>
      <Paragraph position="0"> In addition to analyzing how the callers phrased their requests in response to the operator's initial prompt, we also analyzed how the operators responded to the callers' requests. 6 We found that in our corpus, the human operator either notifies the caller of a destination to which the call will be transferred, or queries the caller for further information, most frequently when the original request was ambiguous and, much less often, when the original request was not heard or understood.</Paragraph>
      <Paragraph position="1"> Table 2 shows the frequency with which each dialogue action was employed by human operators in our corpus. It shows that nearly 20% of all caller requests require further disambiguation. We further analyzed these calls that were not immediately routed and noted that 75% of them involve underspecified noun phrases, such as requesting car loans without specifying whether it is an existing car loan or a new car loan. The remaining 25% mostly involve underspecified verb phrases, such as asking to transfer funds without specifying the accounts to and from which the transfer will take place, or missing verb phrases, such as asking for direct deposit without specifying whether the caller wants to set up a direct deposit or change an existing direct deposit.</Paragraph>
      <Paragraph position="2"> Based on our analysis of operator responses, we decided to first focus our router responses on notifying the caller of a selected destination in cases where the caller request is unambiguous, and on formulating a query for noun phrase disambiguation in the case of noun phrase underspecification in the caller request. For calls that</Paragraph>
    </Section>
  </Section>
class="xml-element"></Paper>