Discovery of Manner Relations and their Applicability to Question
Answering
Roxana Girjua0a2a1 a3 , Manju Putchaa0 and Dan Moldovana0
Human Language Technology Research Institutea0
University of Texas at Dallas
and
Department of Computer Sciencea3
Baylor University
girju@ecs.baylor.edu, moldovan@utdallas.edu
Abstract
The discovery of semantic relations from
text becomes increasingly important for
applications such as Question Answer-
ing, Information Extraction, Summariza-
tion, Text Understanding and others. This
paper presents a method for the auto-
matic discovery of manner relations using
a Naive Bayes learning algorithm. The
method was tested on the UPenn Tree-
bank2 corpus, and the targeted manner re-
lations were detected with a precision of
64.44% and a recall of 68.67%.
1 Introduction
1.1 Problem description
An important semantic relation for several NLP ap-
plications is the manner relation. Consider the sen-
tence (from the Democratic response to the Presi-
dent Bush’ 2003 State of the Union Address):
We want to work together to build our new
economy, creating jobs by investing in technology
so America can continue to lead the world
in growth and opportunity.
There are four manner relations in this text: (1)
together is a manner adverb that modifies the verb
work, (2) creating jobs is an adverbial phrase at-
tached through a manner relation to the verb work,
(3) by investing in technology is a prepositional
phrase that expresses manner and attaches to the
verb create, and (4) in growth and opportunity is a
manner prepositional phrase that modifies the verb
lead.
The discovery of manner relations in open text al-
lows Question Answering systems to identify these
relations and formulate answers to manner questions
that otherwise are not possible even with state-of-
the-art QA systems. For example, by identifying the
manner relations in the example above, the follow-
ing how questions may be answered:
Q: How do Democrats want America to lead the
world ? A: in growth and opportunity
Q: How do Democrats want to work? A: work to-
gether (with Republicans).
Q: How do Democrats want to build the economy ?
A: by creating jobs;
Q: How do Democrats want to create jobs? A: by
investing in technology
This paper provides a method for discovering
manner semantic relations in open text.
1.2 The semantics of manner relation
In WordNet, the manner relation is defined as a way
of acting or behaving. Similar definitions are pro-
vided by psychology researchers (Graesser et al.,
2000).
There are different ways of expressing man-
ner and the difficulty arises that the same lexico-
syntactic patterns that express manner also express
other semantic relations in different contexts. A pos-
sible way to check whether or not a verb expression
conveys manner is to answer correctly the question
“In what manner/how a4 to verb a5 ?” For exam-
ple, for run quickly, we ask how to run? However,
this test holds only when there are no other answers
to questions like “Where a4 verb a5 ?”, or “When a4
verb a5 ?” that make sense. For example, jump over
the fence or jump always are not manner relations
although they may answer correctly a how question.
1.3 Previous work
Although manner relations were studied by philoso-
phers (Aristotle, 350BC), logicians, psychologists
and linguists (Quirk et al., 1985), (Fellbaum, 2002),
not much work has been done to automatically iden-
tify the manner relations in texts. Hearst (Hearst,
1998) developed a method for the automatic acqui-
sition of hypernymy relations by identifying a set of
frequently used and unambiguous lexico-syntactic
patterns. Then, she tried applying the same method
to other semantic relations, such as part-whole, but
without much success, as the patterns detected were
ambiguous.
2 Lexico-syntactic patterns expressing
manner
2.1 Manner as semantic role
The most frequently occurring form of manner is
as a semantic role (Quirk et al., 1985). In this
case, manner is encoded as a relationship between
a verb and one of its arguments which can be repre-
sented by various parts of speech, the most common
ones being adverb, adverbial phrase, prepositional
phrase, noun phrase, and clause.
Verb-adverb patterns
One of the most frequently used patterns expressing
manner is verb-adverb. In English, there are differ-
ent kinds of adverbs (Quirk et al., 1985): adverbs of
time, manner, degree, location, direction, frequency,
transition and hedges.
Based on the classification provided by Quirk et
al. (Quirk et al., 1985) and our statistics of English
texts, we present below the adverbial patterns in or-
der of their frequency of occurrence:
a) Adverbs of manner that end in “-ly”
This manner adverbs are the most frequently used.
Their position is not fixed, as they can be placed
either before or after the verb they modify. These
adverbs can be modified by other adverbs forming
this way adverbial expressions. Examples: slowly,
heavily, angrily, etc.
b) Adverbs of manner that do not end in “-ly”
These adverbs also called Quality description ad-
verbs provide a description of a particular quality.
Example: fast, good, well, etc.
c) Adverbial expressions
These are expressions that modify the underly-
ing verb and refer along with the verb to a man-
ner relation. Examples of such patterns are: a4 as
adv manner as a0a2a1a4a3a6a5a8a7a10a9a11a3a13a12 a5 , a4 NP as adv manner
a5 , a4 as adv manner S a5 .
Examples: several times as fast, as much as 60%
faster, louder than ever, all around, etc.
d) Compound adverbs of manner
These adverbs are usually formed with words linked
by hypens. Examples: radio-style, tax-free, flat-out,
first-hand, etc
e) Foreign adverbial expressions
There are expressions boroughed from other lan-
guages that are in a manner relationship with the
underlying verb. Examples: in flagrante, a la Gor-
bachev, en masse, etc.
2.2 Other forms of manner relations
In addition to the manner roles expressed as verb-
adverb pairs, manner relations are also expressed as
(1) complex nominals (fast car), (2) verbs of im-
plicit manner (for example whisper is a manner of
speaking), (3) verb-PP (I took your coat by mistake),
(4) verb-NP (He breathed a deep breath), (5) verb
clauses (I cook vegetables as Chinese do), and oth-
ers.
All these lexico-syntactic patterns are ambiguous.
Thus we need some syntactic and semantic con-
straints to differentiate the manner relations from the
other possible meanings these patterns may have.
In this paper we focus only on the discovery of
manner semantic roles expressed as verb- adverb
pairs. The method, however, is extendable to many
other manner forms and even to other semantic rela-
tions.
3 Approach
The learning procedure proposed here is supervised,
for the learning algorithm is provided with a set of
inputs along with the corresponding set of correct
outputs. In this paper we use the Naive Bayes Clas-
sifier approach to determine whether or not a verb-
adverb pair indicates a manner relation. This method
is similar with the basic algorithm for Document
Classification (Mitchell, 1997).
Nr. Feature
1 Specific adverb statistics
2 Parent phrase type
3 Present or not in the Adverb Dictionary
4 Distance between verb and adverb
5 Component before adverb
6 Component after the adverb
7 Adverbs ends or not with ’ly
Table 1: Summary of Manner Features.
This approach requires a decision on how to rep-
resent an arbitrary text in terms of attribute (or fea-
tures) values and how to estimate their probabilities
as required by the Naive Bayes Classifier.
4 Selecting features
Many researchers ((Blaheta-Charniak, 2000),
(Gildea-Jurafsky, 2000), (Gildea-Palmer, 2002))
showed that lexical and syntactic information is
very useful for predicate-argument recognition
tasks. Their systems are statistical-based and have
been trained to automatically label semantic roles
only from the output of syntactic parsers.
However, lexical and syntactic information alone
is not sufficient for the detection of the manner se-
mantic roles, semantic information is necessary as
well.
To represent the text for the discovery of manner
relations, seven features which contribute the most
to the classification were chosen. These features
capture the context of the adverb and help in decid-
ing the presence of the manner (MNR) component.
We have developed an Adverb Dictionary that is
a source for some of the features. The Adverb Dic-
tionary is created with adverbs from WordNet and
TreeBank. The adverbs that contain the pattern “in
a —– manner” in their gloss were extracted from
WordNet. The adverbs that are annotated in Tree-
Bank as MNR adverb-verb pairs are also included
in the Dictionary. A total of 2183 adverbs were in-
cluded in the Dictionary.
The features are explained with the help of the
following example:
(S1 (S (NP (DT The) (NN bank)) (VP (AUX is)
(ADVP (RB now))(VP (ADVP (RB aggressively))
(VBG marketing) (NP (JJ retail)(NNS services))
(PP (IN at) (NP (PRP$ its) (JJ domestic) (NNS
branches))))) (. .)))
(1) Specific adverb statistics
Feature 1 checks if a specific adverb is present in the
Dictionary or not. For example, aggressively is part
of the Dictionary, where as now is not. The posi-
tive frequency calculated from this feature is the to-
tal number of times that adverb was encountered in
the training corpus. In the case the adverb of a sen-
tence in the testing corpus is part of the Dictionary,
this feature helps in deciding what are its chances of
being a Positive/Negative Indicator of Manner. This
is a good feature as long as the training corpus is
very rich (i.e it covers all adverbs).
(2) Parent phrase type
The second feature is the phrase type to which the
adverb attaches. Here both now and aggressively at-
tach to “VP”. Most of the MNR indicating adverbs
attach to verbs. This feature helps eliminate adverbs,
which modify nouns or adjectives.
(3) Whether or not Adverb is present in the
Dictionary
Feature 3, like feature 1 checks whether or not an
adverb is present in the Adverb Dictionary. The dif-
ference is that its statistics are not calculated on the
training corpus like in feature 1, but instead it takes
the probability of being a manner adverb in the Ad-
verb Dictionary.
The usefulness of feature 3 is realized when the
test corpus has an adverb which was not encountered
in the training corpus. The estimates from feature 1
fail to be of any use at such a point because it is
a missing value and both positive and negative fre-
quencies are the same. However, feature 3 assigns
the probabilities of that adverb being a manner ad-
verb in the Adverb Dictionary. So, we still have a
good estimate from this feature to decide if it is a
potential MNR indicator or not (which would have
been nullified, had we relied only on feature 1).
For example, let’s say we encounter the adverb
excitedly in the test corpus and it is present in the
Adverb Dictionary but not in the training corpus.
Feature 1 will not contribute to the decision while
feature 3 will help. We can use the lookup table for
feature 3 and it is evident that an adverb present in
the Dictionary has a higher probability of indicating
manner.
(4) Distance between verb and adverb
The fourth feature is the distance between verb and
adverb. This doesn’t take into consideration whether
the adverb precedes or succeeds the verb. Distance
refers to the number of English words that separate
them. For example, there are no words between
aggressively and marketing, thus the distance is 0.
Similarly, the distance between now and marketing
is 1. The rational of this feature is based on the ob-
servation that most frequently a MNR indicating ad-
verb appears immediately next to a VB.
(5) Component before the adverb
The fifth feature concerns the POS of the word pre-
ceding the adverb. This captures the context of the
adverb. This is based on the observation that an ad-
verb that succeeds an AUX is usually not a MNR
indicator. For example now is preceeded by “AUX”
and aggressively is preceded by an “ADVP”.
(6) Component after the adverb
The sixth feature concerns the POS of the word after
the RB. For example now is succeeded by an “AUX”
and aggressively by an “VBG”.
(7) Adverb ends in “ly”
This feature is 1 when the adverb ends in “ly” and 0
otherwise. The rational for this feature is that many
adverbs in manner roles end in “ly”.
Estimating Probabilities
The next step is to calculate the probabilities re-
quired by the Naive Bayes Classifier.
a. Class prior probabilities. This is the ratio between
the number of adverbs of each class over the total
number of adverbs in the training examples. In our
case the classes are positive (or Manner) and nega-
tive (not Manner). This is defined as:
a1a1a0a3a2a5a4a7a6a9a8a11a10a12a4 a3a13a10
where a10a14a4 is the total number of examples for which
the target value is a2a15a4 and a10 is the total number of
examples.
b. Class conditional probability. This is the proba-
bility that any of the seven features drawn from the
parsed text tagged positive or negative will belong
to the domain of the corresponding features. We use
the m-estimate to avoid the cases when a16a18a17a20a19 a21a23a22a25a24a27a26
and a28a29a24a31a30 a21a23a22a20a24a32a26 are very small.
a1a33a22a20a17a20a34a35a0a3a36a37a6a9a8a38a0a39a16a18a17a20a19 a21a23a22a25a24a27a26a9a36a41a40a32a6 a3a15a0a3a2a43a42a45a44a33a46a48a47a41a36a50a49a51a10a37a52a53a49a54a6
a1a33a22a20a17a20a34a35a0a56a55a51a6a9a8a38a0a57a28a29a24a58a30 a21a23a22a25a24a32a26a59a36a60a40a32a6 a3a15a0a3a2a61a42a45a44a33a46a48a47a60a36a62a49a51a10a37a52a53a49a54a6
where a16a23a17a35a19 a21a23a22a25a24a32a26 is the number of times the fea-
ture occurred in the Positive class, a28a29a24a58a30 a21a23a22a25a24a27a26 is the
number of times the feature occurred in the Nega-
tive class, a2a43a42a63a44a33a46a48a47 is the distinct number of posi-
tive and negative instances for a given feature, and
a49a51a10a63a52a62a49 is the total number of all positive and nega-
tive instances in the examples.
4.1 Learning Algorithm
The algorithm learns the probability that a given ad-
verb indicates manner (i.e. how many times the ad-
verb occurred in the positive class and how many
times in the negative class). Similarly, it learns the
probability that it attaches to a VP/NP/... in each of
the positive and negative classes. The same is true
for all features.
At the end of the learning process, the algorithm
creates look-up tables for all the features. These are
used by the classifier. The learning step along with
the output are explained in the next section.
a2a65a64a13a66a67a8 a5a5a22a13a30a69a68 a5a5a70 a1a1a0 a9a27a4a7a6a72a71 a1a1a0a73a21a13a74 a3 a9a27a4a7a6
. a9a27a4a37a75a50a2 a21a13a74a76a75a62a77
where a2a65a64a13a66 is the output of the Naive Bayes Classi-
fier, a9a27a4 is the class in the target set a2 , and a21a25a74 are
the individual features from the set a77 of the seven
features.
5 Experimental Setting
5.1 Building the Training and Test Corpus
In order to learn the constraints, we used the Tree-
bank2 (Marcus, 1994) text collection and LA Times
Corpus. Treebank2 is a corpus featuring one mil-
lion words of 1989 Wall Street Journal material an-
notated with several predicate-argument structures.
It is annotated with the following semantic roles:
BNF (beneficiary), DIR (direction), EXT (spatial ex-
tent), LOC (location), MNR (manner), PRP (purpose
and reason), and TMP (temporal). Treebank2 con-
tains different types of manner annotations: ADVP-
MNR (1683), PP-MNR(952), SBAR-MNR (60),
NP-MNR(54), S-MNR(48), UCP-MNR (8), ADJP-
MNR(1). For the work in this paper we used the
ADVP-MNR annotations from Treebank2.
The input to the program is a parsed text. For
training and testing the Treebank2 corpus is split in
the 3:1 ratio. The algorithm doesn’t work on the
parsed text directly. Instead, the parsed text is con-
verted into the 7-feature format augmented with the
value of the target function as shown in Table 2
Adverb Parent In-Dict Distance Before After ly target
now VP 0 1 AUX VP 0 no
then S 0 3 no before NP 0 no
long VP 0 1 AUX VP 0 no
back VP 0 5 NP SBAR 0 no
aggressively VP 1 0 ADVP VBG 1 yes
magisterially VP 1 2 NP . 1 yes
directly VP 1 0 VBN PP 1 yes
rapidly VP 1 0 AUX VP 1 yes
Table 2: A sample of training data
Creation of the Look-Up table
Given this format as input, the learning algorithm
creates LookUp tables using the Class Conditional
Probability and Reference files. These files contain
the domain of the features. Each feature can take a
set of legal values encountered during training. Ta-
ble 3 exemplifies the lookup entries for some feature
examples.
6 Results for discovering manner relations
Let us define the precision and recall performance
metrics in this context.
a0a2a1a4a3a6a5a8a7a10a9a11a7a13a12a4a14a16a15
a17a16a18a2a19a16a20
a3a6a1a21a12a23a22a24a5a25a12a4a1a26a1a27a3a6a5a29a28a31a30a33a32a34a1a4a3a11a28a35a1a26a7a36a3a38a37a39a3a6a40a41a1a27a3a38a30a33a42a43a28a44a7a36a12a26a14a45a9
a17a16a18a46a19a16a20
a3a6a1a21a12a27a22a24a1a27a3a38a30a33a42a43a28a44a7a36a12a26a14a45a9a47a1a4a3a6a28a44a1a4a7a36a3a6a37a39a3a6a40
a1a4a3a6a5a8a42a48a30a49a30a50a15
a17a16a18a2a19a51a20
a3a6a1a52a12a23a22a53a5a25a12a4a1a26a1a4a3a38a5a29a28a31a30a33a32a54a1a27a3a11a28a35a1a26a7a36a3a6a37a39a3a38a40a41a1a4a3a26a30a55a42a39a28a44a7a13a12a4a14a45a9
a17a16a18a2a19a51a20
a3a6a1a52a12a23a22a56a5a8a12a26a1a4a1a4a3a6a5a25a28a57a1a27a3a38a30a33a42a43a28a44a7a36a12a26a14a45a9
The experiments were conducted with the annota-
tions in UPenn’s Treebank2. The results of the first
experiment are shown in Tables 4.
First experiment
Training = (1176 Positive + 2546 Negative) = 3722
examples
Testing = (507 Positive + 1183 Negative) =1690 ex-
amples.
Output of the program:
Prior Positive Probability = 0.315959162
Prior Negative Probability = 0.684040838
Precision = 191/242 = 78.92%
Recall = 191/507 = 37.62%
Second experiment
Based on the results from the previous set of results
it is observed that considering adverbs like more-
over, then, thus which can never indicate MNR re-
duces both the precision and recall. Therefore they
were removed from the set of negative examples.
Similarly the intensifiers like much, very, so were
also removed from the positive examples.
Training examples = 1103 Pos + 1352 Neg = 2355
Test Corpus = 508 Pos + 1183 Neg = 1690
Prior Positive Probability = 0.4492
Prior Negative Probability = 0.5740
The results are shown in Table 5.
Relations No. of relations
Nr of MNR relations in corpus 507
Number MNR relations retrieved 242
Number of correctly retrieved rel 191
Precision 191/242 = 78.92a58
Recall 191/507 = 37.62a58
Table 4: The precision and recall for experiment 1
Relations No. of relations
Nr of MNR relations in corpus 507
Number MNR relations retrieved 540
Number of correctly retrieved rel 348
Precision 348/540 = 64.44a58
Recall 348/507 = 68.67a58
Table 5: The precision and recall for experiment 2
7 Application to Question Answering
The manner semantic relation occurs with high fre-
quency in open text. Its discovery is paramount for
many applications, such as Information Extraction,
Text Mining, Knowledge Base construction, etc. In
this section we mentioned only Question Answer-
ing.
The concepts and manner relations acquired from
a collection of documents can be useful in answer-
ing difficult questions that normally can not be han-
dled based solely on keywords matching and prox-
imity. As the level of difficulty increases, Question
Answering systems need richer semantic resources,
including the discovery of semantic relations in open
texts. In the case of a manner question, the answer
Feature Feature Example Nr Pos Nr Neg Prob(+) Prob(-)
aggressively 18 2 0.000087 0.000014
adverb magisterially 4 0 000023 000005
directly 34 0 0.000159 0.000005
VP 1510 329 0.011178 0.002441
parent no par 99 243 0.000740 0.001805
S 42 117 0.000318 0.000873
Dictionary yes 1 1175 1107 0.005371 0.005061
no 0 2 1440 0.000014 0.006582
0 881 1036 0.004028 0.004736
distance 2 68 275 0.000315 0.001260
1 142 515 0.000653 0.002356
ADVP 32 55 0.000151 0.000256
POS preceding NP 273 661 0.001251 0.003023
VBN 107 101 0.000493 0.000466
VBG 57 27 0.000265 0.000128
POS after - 93 70 0.000429 0.000324
PP 211 221 0.000968 0.001014
ends with “ly” 1 990 740 0.004526 0.003385
0 185 1805 0.000850 0.008249
Table 3: Example of features look-up table
type of that question may be tagged as MNR. To pro-
vide the correct answer, often it is sufficient to locate
first the paragraph where the potential answer is and
then identify the MNR tag in that paragraph. In case
when several such MNR tags exist, more reason-
ing is necessary. Consider the following examples
which show the MNR tag in the answer sentence.
Q: How did Bob Marley die?
A1: Bob Marley died a0 of Melanomaa1a2a0 MNRa1 .
Q: How was little Johnny dressed last night?
A1: Dressed a0 in a cowboy stylea1a2a0 MNRa1 , Johnny
walked proudly on the street.
Q: How does Marry dance?
A1: Marry danced a0 as well as Billa1a2a0 MNRa1 .
Q: How does Lina Mayors charms her audience?
A1: Countering every unfruitful description, her
work communicates and a0 impresses through the
rhythm of the colorsa1a2a0 MNRa1 .
8 Conclusions
The method presented in this paper for the detection
and validation of manner relations is automatic and
novel. We combined lexical, syntactic and semantic
features for a more accurate learning.
Naive Bayes Classifier assumes feature indepen-
dence. Here, features 1 and 4 are independent, the
rest are dependent on each other. This is the rea-
son for 65-70% precision and recall. By using some
heuristics like removing unambiguous adverbs these
were helped. The improvement made in the second
experiment is significant because if an adverb like
now, or moreover is included in the negative exam-
ples, then other features which contribute to a posi-
tive example are nullified and the decision becomes
less precise. For example, apparently attaches to VP
and VP usually occurs in a positive class, and the
inclusion of this example in the negative example
reduces the estimates of VP to contribute to positive
examples.
The Naive Bayes Classifier, though over-
simplified by the independence assumption, proved
to be a good classifier in the document classification
and also promises to be a useful method for the dis-
covery of semantic relations.

References
Aristotle. On Sophistical Refutations. On Sofistical Refula-
tions, section 3, Translated by W. A. Pickard-Cambridge.
Don Blaheta and Eugene Charniak, 2000. Assigning Function
Tags to Parsed Text. Proceedings of the 1st Annual Meeting
of the North American Chapter of the Association for Com-
putational Linguistics, Seattle, May 2000, pp. 234–240
Martha Palmer, Joseph Rosenzweig, William Schuler 1998
Capturing Motion Verb Generalizations with Synchronous
TAGs Predicative Forms in NLP,pp 250-277, ed by Patrick
St. Dizier, Kluwer Press, December, 1998.
Beth Levin - English Verb Classes and Alternations The Uni-
versity of Chicago Press
Cornelia Maria Verspoor 1997 Contextually Dependent Lexical
Semantics The University of Edinburgh, 1997
Julia B. St.John On the Semantics of Manner Adverbs Carolina
Working Papers in Linguistics Vol 1, Issue 1
2002 Oriented Adverbs Issues in Lexical Semantics of Event
Adverbs, Von Wilhem Gauder, 2002
Christiane Fellbaum 2002 On the Semantics of Troponymy
Cognitive Science Laboratory, Princeton University, Decem-
ber 2002.
Tom Mitchell 1997 Machine Learning McGraw Hill, 1997
Daniel Gildea and Daniel Jurafsky. 2000. Automatic Label-
ing of Semantic Roles. In Proceedings of the 38th Annual
Conference of the Association for Computational Linguis-
tics (ACL-00), pages 512-520, Hong Kong, October 2000.
Daniel Gildea and Martha Palmer. 2002. The Necessity of Syn-
tactic Parsing for Predicate Argument Recognition. In Pro-
ceedings of the 40th Annual Conference of the Association
for Computational Linguistics (ACL-02), Philadelphia, PA,
2002.
Arthur C Grasser, Peter Weimer Hastings and Katiga Waimer
Hastings. 2002. Constructing Inferences and Relations dur-
ing Text Comprehension.
M. Hearst. 1998. Automated Discovery of WordNet Relations,
An Electronic Lexical Database and Some of its Applica-
tions. MIT Press, Cambridge MA, 1998.
Judith Levi. 1978. The Syntax and Semantics of Complex Nom-
inals. NY: Academic Press.
Beth Levin. 1993. English Verb Classes and Alternations. The
University of Chicago Press
M. Marcus. 1994. The Penn treebank: A revised corpus de-
sign for extracting predicate-argument structur. In Proceed-
ings of hte ARPA Human Language Technology Workshop,
Princeton, NJ, 1994.
R. Quirk, S. Greenbaum, G. Leech, and J. Svartvik. 1995. A
comprehensive grammar of English language. Longman,
Harlow, 1985
