Representing semantics of texts - a non-statistical approach
Svetlana Hensman
School of Computing
Dublin Institute of Technology
Kevin Street
Dublin 8, Ireland
Svetlana.Hensman@comp.dit.ie
John Dunnion
Intelligent Information Retrieval Group
Department of Computer Science
University College Dublin
Belfield, Dublin 4, Ireland
John.Dunnion@ucd.ie
Abstract
This paper describes a non-statistical
approach for semantic annotation of
documents by analysing their syntax
and by using semantic/syntactic behav-
iour patterns described in VerbNet. We
use a two-stage approach, firstly iden-
tifying the semantic roles in a sen-
tence, and then using these roles to rep-
resent some of the relations between
the concepts in the sentence and a list
of noun behaviour patterns to resolve
some of the unknown (generic) rela-
tions between concepts. All outlined
algorithms were tested on two corpora
which differs in size, type, style and
genre, and the performance does not
vary significantly.
1 Introduction
This paper describes a system for semi-automatic
conceptual graph acquisition using a combina-
tion of linguistic resources, such as VerbNet and
WordNet, together with semi-automatically com-
piled domain-specific knowledge. Such seman-
tic information has a number of possible applica-
tions, for example in the area of information re-
trieval/extraction for enhancing the search meth-
ods or in question-answering systems, allowing
users to communicate with the system in natural
language (English).
We use conceptual graphs (CGs) (Sowa, 1984),
a knowledge-representation formalism based on
semantic networks and the existential graphs of
C. S. Pierce, to represent the semantics of doc-
uments. There are number of systems for gen-
erating conceptual graphs representation of sen-
tences: Sowa and Way (Sowa and Way, 1986)
use a lexicon of canonical graphs which are com-
bined to build a conceptual graph representation
of a sentence, while Veraldi at al. (Velardi et al.,
1988) describe a prototype of a semantic proces-
sor for Italian sentences, which uses a manually
acquired lexicon of about 850 word-sense defin-
itions, each including 10 – 20 surface semantic
patterns (SSPs) representing both usage informa-
tion and semantic constraints.
There are also systems aimed at extracting par-
tial knowledge from texts, by either filling seman-
tic templates (Hobbs et al., 1996) or by generating
a set of linguistic patterns for information extrac-
tion (Harabagiu and Maiorano, 2000), to name
but a few.
The following sections describe in more detail
the various aspects of our system, the experiments
that we carried out to test the proposed algorithms
and finally draw some conclusions.
2 System overview
We use a two-step approach for constructing con-
ceptual graph representations of texts: firstly,
by using VerbNet and WordNet, we identify
the semantic roles in a sentence, and secondly,
using these semantic roles and a set of syn-
tactic/semantic rules we construct a conceptual
graph.
To evaluate our algorithms we use test docu-
ments from two corpora in different domains –
the Reuters-21578 text categorization test collec-
209
tion (Reuters, 1987) and the collection of aviation
incident reports provided by the Irish Air Acci-
dent Investigation Unit (AAIU) (Air Accident In-
vestigation Unit, 2004). All documents are parsed
using Eugene Charniak’s maximum entropy in-
spired parser (Charniak, 2000).
3 Semantic role identification
There are number of different existing ap-
proaches for identifying semantic roles, varying
from traditional parsing approaches, for exam-
ple using HPSG grammars and Lexical Func-
tional Grammars, that strongly rely on manually-
developed grammars and lexicons, to data-driven
approaches, for example AutoSlog (Riloff and
Schmelzenbach, 1998). In the domain of the Air
Traveler Information System (Miller et al., 1996)
the authors apply statistical methods to compute
the probability that a constituent can fill in a se-
mantic slot within a semantic frame. Gildea and
Jurafsky (Gildea and Jurafsky, 2002) describe a
statistical approach for semantic role labelling us-
ing data collected from FrameNet by analysing a
number of features such as phrase type, grammat-
ical function, position in the sentence, etc.
Shi and Mihalcea (Shi and Mihalcea, 2004)
propose a rule-based approach for semantic pars-
ing using FrameNet and WordNet. They extract
rules from the tagged data provided by FrameNet,
which specify the realisation (order and different
syntactic features) for the present semantic roles.
They also create a feature set representation of
the sentence and match it to each of the extracted
rules. The result is the rule providing the most
feature matches. The authors do not provide any
information on how they select between different
matches with the same score, or if there is any
semantic check on suitability of a phrase to re-
alise a semantic role (FrameNet does not provide
any restrictions on the semantic roles similar to
the selectional restrictions present in VerbNet).
The approach we propose for semantic role
identification uses information about each verb’s
behaviour, provided in VerbNet, and the Word-
Net taxonomy to decide whether a phrase can be
a suitable match for a semantic role.
VerbNet (Kipper et al., 2000) is a computa-
tional verb lexicon, based on Levin’s verb classes,
that contains syntactic and semantic information
for English verbs. Each VerbNet class defines a
list of members, a list of possible thematic roles,
and a list of frames (patterns) of how these se-
mantic roles can be realized in a sentence.
WordNet (Fellbaum, 1998) is an English lex-
ical database containing about 120 000 entries
of nouns, verbs, adjectives and adverbs, hier-
archically organized in synonym groups (called
synsets), and linked with relations such as hyper-
nym, hyponym, holonym and others.
To identify the semantic roles for a clause in a
sentence we identify and match the clause pattern
to each of the possible semantic frames for the
clause verb (from VerbNet). The result is a list
of all possible semantic role assignments, from
which we must identify the correct one.
3.1 Constructing sentence patterns for the
verbs in a sentence
For each sentence clause we construct a syntac-
tical pattern, which is a flat parse representation
that identifies the main verb and the other main
categories of the clause. As a sentence can have
subordinate clauses, we usually have more than
one syntactic pattern per sentence. Each such pat-
tern is processed individually.
Using a constituency parser (such as Char-
niak’s) is suitable in the majority of cases, but
there are some sentences where the correct set of
role fillers cannot be identified by using the parse
tree. For example, for sentences such as
The price of oil will rise by 5 cents by
the end of the year.
the phrase the price of oil will be identified as
a possible role filler by our system, while the cor-
rect result would have the price identified as the
Attribute and oil as the Patient. For such cases the
use of a dependency parser (such as a Link Gram-
mar parser or a Functional Dependency Grammar
parser) would be required.
We also address some simple cases of pronoun
anaphoric reference. For example, for patterns
such as
Iomega Corp said it has laid off over a
quarter of its professional and manage-
ment staff.
210
we identify the pronoun it as referring to the
subject of the verb in the main clause (which here
is Iomega Corp) if they agree by gender and num-
ber. In cases where the type of the concept repre-
sented by the phrase is known, an agreement by
type is also required.
Some cases of intersentential pronoun
anaphoric references are also resolved by
analysing the previous sentence context for
suitable candidates, that agree by gender, number
and type. Agreement by type is present if the
type of the phrase is compatible (or the same) as
the type of the phrase it references. For example,
if the company refers to Iomega Corp, which is
listed as an instance of the type organization, then
the types of the two phrases are compatible, as
company is defined as sub-type of organization.
If agreement by type cannot be assured, the
reference is not resolved. The reference is only
resolved if there is a single possibility for its
resolution.
3.2 Extracting VerbNet semantic role frames
Each verb can be described in VerbNet as a mem-
ber of more than one class, and therefore the list
of its possible semantic frames is a combination
of the semantic frames defined in each of the
classes in which it participates.
We extract all the semantic frames in a class
and consider them to be possible semantic frames
for each of the verbs that are members of this
class. Each verb class also defines a list of se-
lectional constraints for the semantic roles. For
example, for all the verbs that are members of the
VerbNet class get-13.5.1 one of the possible se-
mantic role frames is:
Agent[+animate OR +organization] V Theme
Prep(from) Source[+concrete].
The selectional constraints check is imple-
mented using one or a combination of the fol-
lowing techniques: hypernym relations defined in
WordNet, pattern matching techniques, syntactic
rules and some heuristics.
3.3 Matching algorithm
The matching algorithm matches the sentence
pattern against each of the possible semantic role
frames extracted from VerbNet. We match the
constituents before and after the verb in the sen-
tence pattern to the semantic roles before and after
the verb in the semantic role frame.
If the number of the available constituents in
the sentence pattern is less than the number of
the required slots in the frame, the match fails.
If there is more than one constituent available to
fill a slot in a semantic frame, each of them is
considered a different match. If, for a seman-
tic frame, we find a constituent for each of the
semantic role slots that complies with the selec-
tional constraints, the algorithm considers this a
possible match.
Multiple results are identified when there are
two or more phrases in a sentence that are possi-
ble semantic role realisations, or if there are two
or more semantic frames for which matches were
found. To select the correct role assignment we
use a weighting function that assigns scores to
each result and returns the one with the highest
score. For each identified role the weighting func-
tion adds one point if the role does not have any
selectional restrictions, and two points if there are
restrictions (including prepositional restrictions).
The total score for a solution is the sum of the
scores for each identified roles. The solution with
the highest score is selected.
For example, for the sentence
USAir bought Piedmont for 69 dlrs
cash per share.
the algorithm identifies two possible role as-
signments:
Agent[+animate OR +organization]
matching NP(The company)
Theme[] matching NP(the shares)
Asset[+currency] matching PP(for 69
dlrs cash per share)
with weight
frame1
=2+1+2=5and the
second solution
Agent[+animate OR +organization]
matching NP(The company)
Theme[] matching the NP(the shares)
with weight
frame5
=2+1=3
Therefore, the algorithm returns the first set of
role assignments as a result.
211
4 Building conceptual graphs
The conceptual graph representation of the sen-
tence is built through the following steps: firstly,
for each of the constituents of the sentence we re-
cursively build a conceptual graph representation;
then we link all the conceptual graphs represent-
ing the constituents into a single graph; and fi-
nally, we resolve the unknown (generic) relations.
Each of these steps is described in more detail in
the following sub-sections.
4.1 Building a conceptual graph
representation of a phrase
The first step involves building a conceptual graph
for a phrase. Our general assumption is that
each lexeme in the sentence is represented us-
ing a separate concept, therefore all nouns, adjec-
tives, adverbs and pronouns are represented using
concepts, while the determiners and numbers are
used to specify the referent of the relevant concept
(thus further specifying the concept).
Below we illustrate the procedure for building
a conceptual graph for some of the most common
types of phrases.
 NP -> DT JJ NN
For phrases following this pattern we create
two concepts - one for the NN with a concept
referent corresponding to the type of the de-
terminer DT, and another concept represent-
ing the adjective, and link both of them by
an Attribute relation. If the phrase contains
more than one adjective, each of them is rep-
resented by a separate concept and they are
all linked with Attribute relations to the con-
cept representing the noun.
 NP -> NP , SBAR ,
This pattern represents phrases where the
noun is further specified by the SBAR (for
example, The co-pilot, who was acting as
a main pilot, landed the plane.) For these
patterns a conceptual graph is built for the
SBAR and the head concept, if a WHNP
phrase (e.g. which or who), is replaced by
the concept created for the NP.
 PP -> IN NP
For such prepositional phrases we construct
a conceptual graph representing the noun
phrase. We also keep track of the preposition
heading the prepositional phrase, as it is used
to mark the relation between this phrase and
the other relevant phrases in the sentence.
4.2 Attaching all constituents to the verb
Once the graphs for each of the constituents are
constructed they are linked together in a single
conceptual graph. As each of them describes
some aspect of the concept represented by the
verb, we link them to that concept.
If the constituent already has an identified se-
mantic role during the previous phase, the same
relation is used when constructing the concep-
tual graph between the CG representing the con-
stituent and the verb. If the constituent does not
have any semantic roles identified, a relation with
a generic label is used, which allows us to build
the structure of the CG concentrating on the con-
cepts involved, and to resolve the generic labels
at a later stage. The generic labels used are ei-
ther REL, or in the case of prepositional phrases
headed with a proposition prep, REL prep (e.g.
REL on).
4.3 Resolving unknown relations
Finally we resolve some of the unknown (generic)
relations in the conceptual graph. We keep a data-
base of the most common syntactic realisation of
relations between concepts with specific types.
An example of a relation correction rule is:
Flight REL from City -> Flight Source City
where the left part of the rule represents the two
concepts linked by a generic relation and the right
side represents the graph after the modification.
All generic relations present after this step must
be manually resolved by the user. The system of-
fers help by suggesting possible relations intro-
duced by a preposition. For example, the preposi-
tion for can indicate Beneficiary (e.g. a book for
Mary), Duration (e.g. for three hours), etc.
5 Query representation
Representation of questions differs than represen-
tation of declarative sentences and deserves spe-
cial attention. For sentences representing ques-
tions we try to identify the statement that will
212
correspond to the question and then construct the
conceptual graph in a similar way as for declara-
tive sentences.
5.1 Yes/No questions
We process simple yes/no questions (questions
that require a yes/no answer) that are constructed
by a subject-verb inversion by applying a trans-
formation to reverse the question to a declarative
sentence.
5.2 Wh question
The parse tree of a sentence expressing a
wh question has the following general struc-
ture: SBARQ ->WH phrase SQ ? where the
WH phrase is either WHNP, WHADVP or WHPP
and represents the concept that triggers the query.
The SQ represents the rest of the sentence.
Similarly to yes/no questions, these type of
questions are also transformed to declarative sen-
tences. The wh word (e.g. who, what, where,
when) is represented by a generic concept. The
relation that attaches this concept to the verb de-
pends on the type of the wh phrase and can be one
of the following:
WHNP
These phrases are headed by the wh ques-
tion words who, what or which. The rela-
tion between the wh phrase and the verb is
either identified from applying a suitable se-
mantic frame for this verb, or it is a generic
one, REL.
WHADVP
These phrases represent an adverbial modi-
fier for time, place or location. If the phrase
marked as WHADVP is where the relation is
locative;ifitiswhen, the relation is tempo-
ral; and if it is how, the relation is manner.
WHPP
Such phrases are not processed by our sys-
tem.
6 Experimental results
Each module of the system was evaluated sepa-
rately.
The first experiment we carried out was to es-
timate the accuracy of the sentence frame con-
structed by the role labelling module and it was
performed on randomly selected 2% of the verbs
in Reuters and 7% of the verbs in AAIU corpora.
The parse trees produced by Charniak’s parser
were manually edited to avoid any errors due to
incorrect parses. The results showed that the sys-
tem identified the correct set of possible candi-
dates for semantic roles for 90% and 89% of the
verbs in the Reuters and in the AAIU documents
respectively.
Further experiments were carried out to eval-
uate the performance of the role assigning mod-
ule. As a testbed we randomly selected 2% of
the verbs in Reuters and 15% of the verbs in
the AAIU documents. From these, we analysed
only those cases where the verb is a member of
at least one VerbNet frame and the possible role
candidates were correctly identified. For 60% and
70% of the remaining verbs, respectively, the al-
gorithm identifies a single correct solution. In
3% and 4% of the cases respectively a partially
correct result is found (in the majority of such
cases it is Agent, Patient and Theme roles that are
correctly identified, together with some incorrect
ones).
In 11% and 9% of the cases for Reuters and
AAIU, respectively, the algorithm identifies a set
of possible solutions, containing the correct and
several incorrect ones. For these cases the weight-
ing function identifies the correct solution in 38%
of the of the cases for AAIU documents and 59%
of the cases for the Reuters documents, while in
40% and 21% of the cases, respectively, it identi-
fies the correct and one or more incorrect results.
We also evaluated the percentage of the syntac-
tic patterns that the graph builder recognises: for
AAUI and Reuters documents, respectively, we
can build a graph for 76% and 67% of the noun
phrases, for 95% and 94% of the prepositional
phrases and for 91% and 97% of the subordinate
clauses.
7 Conclusions
In this paper we have described an approach
for constructing conceptual graphs for English
sentences, using VerbNet, WordNet and some
domain-specific knowledge. The achieved accu-
213
racy is strongly influenced by the lack of VerbNet
descriptions of many verbs present in both cor-
pora, as well as the lack of semantic frames for
the present verb sense. Also, as the approach is
not statistical, it does not require large amount of
training data.
There are several other lexical resources cur-
rently available that seem suitable for semantic
role identification, among them FrameNet and
PropBank. Our choice of VerbNet as a lexical re-
source is based on our belief that a set of domain-
independent descriptive role labels (such as those
defined in VerbNet) is more suitable as it allows
for generalisations.
A drawback of both FrameNet and PropBank
is that the roles do not include any selectional
restrictions, which makes it hard for a non-
statistical method to identify the correct filler for
each role. As shown earlier, the selectional re-
strictions defined for the semantic roles prove to
be a valuable asset when deciding if a phrase can
be a role filler. While we can resolve the majority
of them by analysing the syntactic structure or by
using the WordNet hierarchy, some are more dif-
ficult to resolve. For example, the restriction solid
describes an attribute or a state of an object, rela-
tions which cannot be checked by using WordNet.
FrameNet on the other hand defines usages not
only for verbs, but also for nouns. As one of the
causes for the relatively poor performance of the
conceptual graph building module is the lack of a
sufficient number of relation-correction rules, our
current approach to increasing their number is try-
ing to extract the rules from FrameNet.
Work on the system is ongoing and efforts
are continuing to implement a verb sense disam-
biguation component.
References
Air Accident Investigation Unit. 2004. Irish Air Acci-
dent Investigation Unit Reports. Available online:
(http://www.aaiu.ie/).
Eugene Charniak. 2000. A Maximum-Entropy-
Inspired Parser. In Proceedings of NAACL-2000,
pages 132–139.
Christiane Fellbaum, editor. 1998. WordNet: An Elec-
tronic Lexical Database. MIT Press, May.
Daniel Gildea and Daniel Jurafsky. 2002. Automatic
Labeling of Semantic Roles. Computational Lin-
guistics, 28(3):245–288.
Sanda Harabagiu and Steven Maiorano. 2000. Ac-
quisition of linguistic patterns for knowledge-based
information extraction. In Proceedings of LREC-
2000, Athens, June.
Jerry Hobbs, Douglas Appelt, John Bear, David Is-
rael, Megumi Kameyama, Mark Stickel, and Mabry
Tyson. 1996. FASTUS: A cascaded finite-state
transducer for extracting information from natural-
language text. In In Finite State Devices for Natural
Language Processing, Cambridge, MA. MIT Press.
Karin Kipper, Hoa Trang Dang, and Martha Palmer.
2000. Class-Based Construction of a Verb Lexicon.
In Proceedings of Seventeenth National Conference
on Artificial Intelligence (AAAI-2000), pages 691 –
696, Austin, TX, July 30 - August 3.
Scott Miller, David Stallard, Robert Bobrow, and
Richard Schwartz. 1996. A fully statistical ap-
proach to natural language interfaces. In Proceed-
ings of the 34th Annual Meeting of the Association
for Computational Linguistics, pages 55–61, Santa
Cruz, June. Morgan Kaufmann Publishers, Inc.
Reuters. 1987. Reuters-21578 Text Cate-
gorization Collection. Available online:
(http://kdd.ics.uci.edu/databases/reuters21578/-
reuters21578.html).
Ellen Riloff and Mark Schmelzenbach. 1998. An Em-
pirical Approach to Conceptual Case Frame Acqui-
sition. In Proceedings of the Sixth Workshop on
Very Large Corpora.
Lei Shi and Rada Mihalcea. 2004. Open Text Parsing
Using FrameNet and WordNet. In Daniel Marcu
Susan Dumais and Salim Roukos, editors, Proceed-
ings of HLT-NAACL 2004: Demonstration Papers,
pages 247–250, Boston, Massachusetts, USA, May
2 – May 7. Association for Computational Linguis-
tics.
John F. Sowa and Eileen C. Way. 1986. Imple-
menting a semantic interpreter using conceptual
graphs. IBM Journal of Research and Develop-
ment, 30(1):57–69, January.
John F. Sowa. 1984. Conceptual Structures: Infor-
mation Processing in Mind and Machine. Addison-
Wesley, Reading, M.
Paola Velardi, Maria Teresa Pazienza, and Mario De-
Giovanetti. 1988. Conceptual graphs for the analy-
sis and generation of sentences. IBM Journal of Re-
search and Development, 32(2):251–267, March.
214
