<?xml version="1.0" standalone="yes"?>
<Paper uid="P05-3024">
  <Title>The Wild Thing!</Title>
  <Section position="3" start_page="0" end_page="0" type="metho">
    <SectionTitle>
1 Motivation: Phone App
</SectionTitle>
    <Paragraph position="0"> Cell phones and PDAs are everywhere. Users love mobility. What are people doing with their phone? You'd think they would be talking on their phones, but a lot of people are typing. It is considered rude to talk on a cell in certain public places, especially in Europe and Asia. SMS text messaging enables people to communicate, even when they can't talk.</Paragraph>
    <Paragraph position="1"> It is bizarre that people are typing on their phones given how painful it is. &amp;quot;Talking on the phone&amp;quot; is a collocation, but &amp;quot;typing on the phone&amp;quot; is not. Slate (slate.msn.com/id/2111773) recently ran a story titled: &amp;quot;A Phone You Can Actually Type On&amp;quot; with the lead: &amp;quot;If you've tried to zap someone a text message recently, you've probably discovered the huge drawback of typing on your cell phone. Unless you're one of those cyborg Scandinavian teenagers who was born with a Nokia in his hand, pecking out even a simple message is a thumb-twisting chore.&amp;quot; There are great hopes that speech recognition will someday make it unnecessary to type on your phone (for SMS or any other app), but speech recognition won't help with the rudeness issue. If people are typing because they can't talk, then speech recognition is not an option. Fortunately, the speech community has developed powerful language modeling techniques that can help even when speech is not an option.</Paragraph>
  </Section>
  <Section position="4" start_page="0" end_page="93" type="metho">
    <SectionTitle>
2 K-Best String Matching
</SectionTitle>
    <Paragraph position="0"> Suppose we want to search for MSN using a cell phone. A standard approach would be to type 6 &lt;pause&gt; 777 &lt;pause&gt; 66, where 6 barb2right M, 777 barb2right S and 66 barb2right N. (The pauses are necessary for disambiguation.) Kids these days are pretty good at typing this way, but there has to be a better solution.</Paragraph>
    <Paragraph position="1"> T9 (www.t9.com) is an interesting alternative.</Paragraph>
    <Paragraph position="2"> The user types 676 (for MSN). The system uses a (unigram) language model to find the k-best matches. The user selects MSN from this list.</Paragraph>
    <Paragraph position="3"> Some users love T9, and some don't.</Paragraph>
    <Paragraph position="4"> The input, 676, can be thought of as short hand for the regular expression: /^[6MNOmno][7PRSprs][6MNOmno]$/ using standard Unix notation. Regular expressions become much more interesting when we consider wildcards. So-called &amp;quot;word wheeling&amp;quot; can be thought of as the special case where we add a wildcard to the end of whatever the user types.</Paragraph>
    <Paragraph position="5"> Thus, if the user types 676 (for MSN), we would find the k-best matches for:</Paragraph>
    <Paragraph position="7"> See Google Suggests1 for a nice example of word wheeling. Google Suggests makes it easy to find popular web queries (in the standard nonmobile desktop context). The user types a prefix. After each character, the system produces a list of the k most popular web queries that start with the specified prefix.</Paragraph>
    <Paragraph position="8"> Word wheeling not only helps when you know what you want to say, but it also helps when you don't. Users can't spell. And things get stuck on the tip of their tongue. Some users are just browsing. They aren't looking for anything in particular, but they'd like to know what others are looking at.</Paragraph>
    <Paragraph position="9"> The popular query application is relatively easy in terms of entropy. About 19 bits are needed to specify one of the 7 million most popular web queries. That is, if we assign each web query a probability based on query logs collected at msn.com, then we can estimate entropy, H, and discover that H[?]19. (About 23 bits would be needed if these pages were equally likely, but they aren't.) It is often said that the average query is between two and three words long, but H is more meaningful than query length.</Paragraph>
    <Paragraph position="10"> General purpose trigram language models are effective for the general case (unrestricted text), but there are important special cases like popular web queries, where more restricted language models are even more effective than trigram models.</Paragraph>
    <Paragraph position="11"> Our language model for web queries is simply a list of queries and their probabilities. We consider queries to be a finite language, unlike unrestricted text where the trigram language model allows sentences to be arbitrarily long.</Paragraph>
    <Paragraph position="12"> Let's consider another example. The MSN query was too easy. Suppose we want to find Condoleezza Rice, but we can't spell her name.</Paragraph>
    <Paragraph position="13"> And even if we could, we wouldn't want to. Typing on a phone isn't fun.</Paragraph>
    <Paragraph position="14"> We suggest spelling Condoleezza as 2*, where 2 barb2right [ABCabc2] and * is the wildcard. We then type '#' for space. Rice is easy to spell: 7423.</Paragraph>
    <Paragraph position="15"> Thus, the user types, 2*#7423, and the system searches over the MSN query log to produce a list  of k-best (most popular) matches (k defaults to 10): 1. Anne Rice 2. Book of Shadows 3. Chris Rice 4. Condoleezza Rice  The letters matching constants in the regular expression are underlined. The other letters match wildcards. (An implicit wildcard is appended to the end of the input string.) Wildcards are very powerful. Strings with wildcards are more expressive than prefix matching (word wheeling). As mentioned above, it should take just 19 bits on average to specify one of the 7 million most popular queries. The query 2*#7423 contains 7 characters in an 12-character alphabet (2-9 barb2right [A-Za-z2-9] in the obvious way, except that 0 barb2right [QZqz0]; # barb2right space; * is wild). 7 characters in a 12 character alphabet is 7 log212 = 25 bits. If the input notation were optimal (which it isn't), it shouldn't be necessary to type much more than this on average to specify one of the 7 million most popular queries.</Paragraph>
    <Paragraph position="16"> Alphabetic ordering causes bizarre behavior.</Paragraph>
    <Paragraph position="17"> Yellow Pages are full of company names starting with A, AA, AAA, etc.. If prefix matching tools like Google Suggests take off, then it is just a matter of time before companies start to go after valuable prefixes: mail, maps, etc. Wildcards can help society avoid that non-sense. If you want to find a top mail site, you can type, &amp;quot;*mail&amp;quot; and you'll find: Gmail, Hotmail, Yahoo mail, etc..</Paragraph>
  </Section>
  <Section position="5" start_page="93" end_page="94" type="metho">
    <SectionTitle>
3 Collaboration &amp; Personalization
</SectionTitle>
    <Paragraph position="0"> Users quickly learn when they can get away with wildcards. Typing therefore becomes a collaborative exercise, much like Palm's approach to handwriting recognition. Recognition is hard. Rather than trying to solve the general case, Palm encourages users to work with the system to write in a way that is easier to recognize (Graffiti). The system isn't trying to solve the AI problem by itself, but rather there is a man-machine collaboration where both parties work together as a team.</Paragraph>
    <Paragraph position="1"> Collaboration is even more powerful in the web context. Users issue lots of queries, making it clear what's hot (and what's not). The system constructs a language model based on these queries to direct users toward good stuff. More and more users will then go there, causing the hot query to move up in the language model. In this way, collaboration can be viewed as a positive feedback  loop. There is a strong herd instinct; all parties benefit from the follow-the-pack collaboration.</Paragraph>
    <Paragraph position="2"> In addition, users want personalization. When typing names of our friends and family, technical terms, etc., we should be able to get away with more wildcards than other users would. There are obvious opportunities for personalizing the language model by integrating the language model with a desktop search index (Dumais et al, 2003).</Paragraph>
  </Section>
  <Section position="6" start_page="94" end_page="94" type="metho">
    <SectionTitle>
4 Modes, Language Models and Apps
</SectionTitle>
    <Paragraph position="0"> The Wild Thing demo has a switch for turning on and off phone mode to determine whether input comes from a phone keypad or a standard keyboard. Both with and without phone mode, the system uses a language model to find the k-best expansions of the wildcards.</Paragraph>
    <Paragraph position="1"> The demo contains a number of different language models, including a number of standard tri-gram language models. Some of the language models were trained on large quantities (6 Billion words) of English. Others were trained on large samples of Spanish and German. Still others were trained on small sub-domains (such as ATIS, available from www.ldc.upenn.edu). The demo also contains two special purpose language models for searching popular web queries, and popular web domains.</Paragraph>
    <Paragraph position="2"> Different language models are different. With a trigram language model trained on general English (containing large amounts of newswire collected over the last decade), pres* rea* *d y* t* it is v* imp* barb2right President Reagan said yesterday that it is very important null With a Spanish Language Model, pres* rea* barb2right presidente Reagan In the ATIS domain, pres* rea* barb2right &lt;UNK&gt; &lt;UNK&gt; The tool can also be used to debug language models. It turns out that some French slipped into the English training corpus. Consequently, the English language model expanded the * in en * de to some common French words that happen to be English words as well: raison, circulation, oeuvre, place, as well as &lt;OOV&gt;. After discovering this, we discovered quite a few more anomalies in the training corpus such as headers from the AP news. There may also be ESL (English as a Second Language) applications for the tool. Many users have a stronger active vocabulary than passive vocabulary. If the user has a word stuck on the tip of their tongue, they can type a suggestive context with appropriate wildcards and there is a good chance the system will propose the word the user is looking for.</Paragraph>
    <Paragraph position="3"> Similar tricks are useful in monolingual contexts. Suppose you aren't sure how to spell a celebrity's name. If you provide a suggestive context, the language model is likely to get it right:</Paragraph>
  </Section>
class="xml-element"></Paper>