Causal Relation Extraction

Títol: Causal Relation Extraction Volum: I Alumne: Eduardo Blanco Villar Director/Ponent: Dan Moldovan / Nuria Castell Ariño Departament: LSI Data: 7 novembre 2007 DADES DEL PROJECTE Títol del Projecte: Causal Relation Extraction Nom de l'estudiant: Eduardo Blanco Villar Titulació: Enginyeria en Informàtica Crèdits: 37.5 Director/Ponent: Dan Moldovan / Nuria Castell Ariño Departament: LSI MEMBRES DEL TRIBUNAL (nom i signatura) President: Vocal: Secretari: QUALIFICACIÓ Qualificació numèrica: Qualificació descriptiva: Data: Causal Relation Extraction Eduardo Blanco November 7, 2007 1 2 Prefacio El presente documento describe el trabajo hecho durante la realizaci´on del Proyecto Final de Carrera (PFC), ´ultimo requisito para obtener los t´ıtulos de Enginyeria en Inform`atica yM`aster Europeu en Llenguatge i Parla por la Universitat Polit`ecnica de Catalunya (UPC). El trabajo ha sido realizado en el Human Language Technology Research Institute (HLTRI), perteneciente a The University of Texas at Dallas (UTD). El director es Dan Moldovan, Co-Director del HLTRI. Nuria Castell es la profesora de la UPC que me puso en contacto con Dan Moldovan y sigui´o mis pasos en el HLTRI. La estancia en el HLTRI se extendi´o desde el 1 de febrero al 31 de agosto de 2007, comprendiendo un total de siete meses. El PFC puede ser calificado como un trabajo de investigaci´on. Se han estudiado las relaciones sem´anticas de causalidad y se propone un m´etodo para su detecci´on y extracci´on. No se presenta un sistema comercial o un prototipo listo para su utilizaci´on directa. Las tareas desarrolladas han sido las t´ıpicas de este tipo de trabajos: lectura de literatura relacionada con el campo y an´alisis del estado del arte (2 meses), definici´on de los objetivos y experimentos a realizar (1 mes), realizaci´on de los experimentos y extracci´on de resultados (3 meses), y por ´ultimo, escritura de la memoria explicando el trabajo realizado (1 mes). 3 Prefaci Aquest document descriu el treball fet durant la realitzaci´o del Projecte Final de Carrera (PFC), ´ultim requisit per obtenir els t´ıtols d’Enginyeria en Inform`atica i M`aster Europeu en Llenguatge i Parla per la Universitat Polit`ecnica de Catalunya (UPC). La feina ha sigut realitzada en el Human Language Technology Research Institute (HLTRI), pertanyent a The University of Texas at Dallas (UTD). El director ´es Dan Moldovan, Co-Director del HLTRI. Nuria Castell ´es la professora de la UPC que em va posar en contacte amb Dan Moldovan i qui ha seguit les meves passes en el HLTRI. L’estada al HLTRI va comen¸car l’1 de febrer de 2007 i va acabar el 31 d’agost del mateix any. El PFC es pot considerar un treball d’investigaci´o. S’han estudiat les relacions sem`antiques de causalitat y es proposa un m`etode per la seva detecci´o i extracci´o. No es presenta un sistema comercial ni un prototipus preparat per ser utilitzat directament. Les tasques desenvolupades s´on les t´ıpiques d’un treball d’aquestes caracter´ıstiques: lectura de la literatura relacionada amb el camp y an`alisi de l’estat de l’art (2 mesos), definifi´o dels objectius i experiments a realitzar (1 mes), realitzaci´o dels experiments i extracci´o de resultats (3 mesos), i per ´ultim, escriptura de la mem`oria explicant la feina feta (1 mes). CONTENTS 4 Contents 1 Introduction 6 2 Semantic Relations 8 2.1 Examples of sets of semantic relations . . . . . . . . . . . . . . . . 8 2.2 The expression of semantic relations . . . . . . . . . . . . . . . . . 9 2.3 The difficulty of deciding the semantic relation . . . . . . . . . . . 11 3 Causations 13 3.1 InPhilosophy ............................. 13 3.2 InPsychology............................. 14 3.3 In Theoretical Linguistics . . . . . . . . . . . . . . . . . . . . . . 15 3.3.1 In Typological Linguistics . . . . . . . . . . . . . . . . . . 17 3.3.2 In the Cognitive Approach . . . . . . . . . . . . . . . . . . 17 3.3.3 Within Discourse Theory . . . . . . . . . . . . . . . . . . . 18 3.4 Causation and other semantic relations . . . . . . . . . . . . . . . 18 3.5 Encoding of Causation . . . . . . . . . . . . . . . . . . . . . . . . 19 4 Previous work 21 4.1 On Semantic relations . . . . . . . . . . . . . . . . . . . . . . . . 21 4.2 Focused on causal relations . . . . . . . . . . . . . . . . . . . . . . 22 5 The method 25 5.1 Syntactic patterns that encode causation . . . . . . . . . . . . . . 25 5.2 Pattern matching . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 5.3 Machine Learning approach . . . . . . . . . . . . . . . . . . . . . 30 5.3.1 Data preparation . . . . . . . . . . . . . . . . . . . . . . . 31 5.3.2 Feature selection . . . . . . . . . . . . . . . . . . . . . . . 31 5.3.3 Feature extraction . . . . . . . . . . . . . . . . . . . . . . 35 5.3.4 Machine Learning algorithm . . . . . . . . . . . . . . . . . 37 5.3.5 Evaluation........................... 39 5.3.6 Error analysis . . . . . . . . . . . . . . . . . . . . . . . . . 40 6 Limitations, Scope and Extensions 42 7 Conclusions and further work 44 References 46 A List of POS tags 50 B Decision trees learned 51 B.1 Decision Tree no. 1: . . . . . . . . . . . . . . . . . . . . . . . . . . 51 B.2 Decision Tree no. 2: . . . . . . . . . . . . . . . . . . . . . . . . . . 51 CONTENTS 5 B.3 Decision Tree no. 3: . . . . . . . . . . . . . . . . . . . . . . . . . . 52 B.4 Decision Tree no. 4: . . . . . . . . . . . . . . . . . . . . . . . . . . 53 B.5 Decision Tree no. 5: . . . . . . . . . . . . . . . . . . . . . . . . . . 53 B.6 Decision Tree no. 6: . . . . . . . . . . . . . . . . . . . . . . . . . . 54 B.7 Decision Tree no. 7: . . . . . . . . . . . . . . . . . . . . . . . . . . 54 B.8 Decision Tree no. 8: . . . . . . . . . . . . . . . . . . . . . . . . . . 55 B.9 Decision Tree no. 9: . . . . . . . . . . . . . . . . . . . . . . . . . . 59 B.10 Decision Tree no. 10: . . . . . . . . . . . . . . . . . . . . . . . . . 59 2 SEMANTIC RELATIONS 12 of the form [Noun Noun], 0.68 when classifying complex nominals of the form [Adjective Noun], and 0.65 when classifying genitives. [Chang and Choi, 2006] report an agreement of 0.73 when classifying manually a set of examples between only two classes: encoding causation and not encoding causation. The low coefficients give an idea of the difficulty of the task: even humans annotators often disagree when classifying manually. 3 CAUSATIONS 13 3 Causations Broadly speaking, causation can be defined as a relation between two events: cause and effect. Cause is the producer of the effect, and effect the result of the cause. This definition is obviously ambiguous, and it doesn’t draw a line between what is a causation and what is not. Causal relations have been studied in several fields. Different theories of causation coexist in Philosophy, Psychology and Linguistics. 3.1 In Philosophy In this presentation we follow the ideas of [White, 1990], which offers summary sketches of the most important points of each theory. In Regularity Theories [Hume, 1739], causation is seen as a constant conjunction between events, associated with priority in time and contiguity in time and, where relevant, space. The conjunction between events is thought to be imperfect, indeterminate, asymmetric and spurious. Therefore, causation is defined as a probable conjunction of two events and threated using Probability Theories. The basic form of the theory says that one event causes another if it is followed by it and is such that events of the first kind are regularly followed by events of the second kind. For others, causation is a condition for the occurrence of an event [Mill, 1843], [Sosa, 1975], [Mackie, 1980] [Suppes, 1970]: •A Sufficient Condition CAUSE(event1, event2), if the existence of event1implies the existence of event2. •A Necessary and Sufficient Condition CAUSE(event1, event2), if the existence of event1implies the existence of event2and the existence of event2implies the existence of event1. •Insufficient but Necessary part of an Unnecessary but Sufficient (INUS) Condition CAUSE(event1, event2) if event1doesn’t imply the existence of event2, but is part of a more complicated condition that imply event2. A good example to understand INUS conditions can be found in [Hitchcock, 2007]: “. . . an INUS condition for some effect is an insufficient but non-redundant part of an unnecessary but sufficient condition. Suppose, for example, that a lit match causes a forest fire. The lighting of the match, by itself, is not sufficient; many matches are lit without ensuing forest fires. The lit match is, however, a part of some constellation of conditions that 3 CAUSATIONS 14 are jointly sufficient for the fire. Moreover, given that this set of conditions occurred, rather than some other set sufficient for fire, the lighting of the match was necessary: fires do not occur in such circumstances when lit matches are not present.” The Counterfactual Approach [Lewis, 1973a,Lewis, 1973b] sees causation as what would have happened if something were the case than in fact is not the case. In other words, event1causes event2just in case it is true that if event1had not occurred, then event2would not have occurred. Several researchers criticize this approach and propose different solutions for the problems of the theory. For example, [Murali, 1997] proposes a solution for the preemption problem, the problem of fragile causes and indeterministic causation. 3.2 In Psychology Psychology is concerned with how people understand and perceive causation, as well as how they make causal inferences and attributions. What are the consequences of thinking that two events are causally related is also studied in Psychology. Different hypothesis have been made about the sort of things people may believe can be causes [White, 1990]: •Events or happenings. •Standing conditions or states of affairs, such as causal powers of material particulars. •Interactions between occurrences and stable properties of things. •Conditions, such as necessary and sufficient conditions or INUS conditions. Psychologists have performed several experiments in order to explain empirically the perception of causations by humans and the implications of considering two events casually related. [Glymour, 2003] offers a formalism based on Bayes nets which allows to reinterpret experiments on human judgment, offers a precise interpretation of mechanisms and allows generalizations of existing theories of causal learning. [Eagleman and Holcombe, 2002] present an experiment that relates causality and the perception of time. They note that if someone holds out his hand and snaps his fingers, he will not notice any difference in the time that he intends to move his fingers and the time he hears the snap. However, humans are able to perceive around 25 miliseconds differences in timing and the auditory signals corresponding to the snap were being processed by the nervous system for more than 100 miliseconds. 3 CAUSATIONS 15 In a very simple experiment, subjects voluntarily pressed a key that caused a tone to follow. When asked to judge the time of the keypress and the time of the tone, subjects perceived the keypress to occur later and the tone earlier than if these events had occurred alone. The authors conclude that when two events are casually related, the perceived time of these two events shifts towards each other. 3.3 In Theoretical Linguistics Causation has been investigated by linguistics under a variety of topics. In this section we will focus on the work done on Typological Linguistics, the Cognitive Approach and Discourse Theory. [Stefanowitsch, 2001] offers three principles of causation construal. These are conceptual strategies used to construe causal links between events and participants; they attempt to identify what in the human language thought leads to the perception of causal relations: 1. The Temporal Succession Principle Given two events, Aand B, a conceptualizer Cperceives Aas the cause of Bif and only if (a) A(regularly) precedes B, and (b) Aand Bare temporally contiguous, and/or (c) Aand Bare spatially contiguous. 2. The Counterfactuality Principle Given two events, Aand B, a conceptualizer Cperceives Aas the cause of Bif and only if (a) Aoccurs and Boccurs, and (b) (Cbelieves that) Bwould not have occurred if Ahad not occurred. 3. The Transmission of Energy Given two events, Aand B, a conceptualizer Cperceives Aas the cause of Bif and only if (a) Cknows that Acan transmit energy, and (b) Cbelieves that Atransmitted energy to some part of the real world and thereby generated B. Two events are considered to be causally related if one or more of the three principles apply. [Stefanowitsch, 2001] also offers a set of attributes of causations events. These attributes help specifying a causation and classify them: 3 CAUSATIONS 16 •The causer is the producer of the cause. We can characterize it using the following set of properties: –Animate ∗intentional or non intentional ∗volitional or non volitional ∗the degree of the causer’s control ∗whether or not the causer is in a position of authority over the causee –Inanimate ∗object or event Note that intentionality entails volitionality and control. •The causee is the producer of the effect. We can characterize it using the following set of properties: –Animate ∗the degree of control that the causee has over the result. ∗whether or not the causee resist the causer; if the causee resist there might be various motivations for doing so. –Inanimate ∗object or event •The causing event is the event performed by the causer. We can characterize it using the following properties: –Direct if involves contact between the causer and causee –Indirect if it doesn’t involve contact. •The resulting event is the event preformed by the causee. We can distinguish between: –immediate or nonimmediate, depending on the timing of the causing and resulting events. If they occur simultaneously with or the resulting event occurs right after the causing event, it is immediate; if a delay exists between both events it is nonimmediate. –static or dynamic, depending on the nature of the event. 3 CAUSATIONS 17 3.3.1 In Typological Linguistics [Shibatani, 1976] proposed several semantic parameters of causative constructions, including: •Coercive or noncoercive, depending on the degree of force that the causer imposes on the causee. The causer may force,persuade or gently suggest the causee. •Permissive or nonpermissive. If permissive, we can subclassify them in four types: –The causer forbears (or omits) prevention (or intervention). –The causer actively gives permission to the causee to do something. –The causer attempts but fails to prevent something from happening. –The causer gives up and does not intervene with the caused event. •Direct or indirect, depending on the previous planing of the causer about bringing a change. If direct, we can distinguish between directive and manipulative. If the causee functions as a volitional entity, it is directive, if the volition of the causee is absent then is manipulative. According to the author, this semantic classification has been proven relevant to many languages, and thus seems to be universally applicable. 3.3.2 In the Cognitive Approach Within the cognitive approach one of the most important theories is Force Dynamics [Talmy, 2000]. Force Dynamics is a semantic category covering how entities interact with respect to force. It started out as a generalization over the traditional notion of causative, analyzing causation into finer primitives and bringing the notions of letting,hindering, and helping into the discussion. The theory defines concepts such as exertion of force,resistance to such exertion and the overcoming of such resistance,blockage of a force and the removal of such blockage, etc. The basic conceptual primitives are [Talmy, 2000]: •Two entities exert a force each on the other. One is foregorunded or singled out for focal attention (the agonist); the other is considered for the effect it has on the agonist (the antagonist). •an entity is taken to exert a force by virtue of an intrinsic tendency towards manifesting it, towards motion (action) or towards rest (inaction). •opposed forces have different relative strength, the entity that is able to manifest its tendency at the expense of its opponent is the stronger. 3 CAUSATIONS 18 •according to their relative strengths, the opposing forces yield a resultant, assessed only for the agonist, as action or inaction. A simple example may help understanding the basis of the theory. “The woman fell because the floor wan slippery.” would be considered as a causation by most of the people. In the example, the agonist is the woman and the antagonist the floor. Both entities exert a force (the woman (suposedly) didn’t want to fall; the floor, by being slippery, makes people fall); the result is the woman falling, so the antagonist wins and the result is an action. The complete theory is much deeper and is considered one of the major contributions in the study of grammar and semantics. 3.3.3 Within Discourse Theory Discourse Theory tries to define the semantic relations that hold between different discourse units. Thus, casusation is threated as another semantic relation. Section 2.1 describes different sets of semantic relations proposed so far and section 3.4 states the differences between a causation and its closest semantic relations. 3.4 Causation and other semantic relations As we have already discussed, it’s not always easy to decide what semantic relation holds between two text spans. The limits between causations and other semantic relations are explained in this section [Talmy, 2000], [Lakoff, 1987]. The closest semantic relations to causation are condition,consequence, reason and influence. condition is a causation whose cause is hypothetical (e.g. “If he were handsome, he would be married.”). consequence is a causation whose effect is indirect or unintended (e.g. “His resignation caused regret among all classes.”); reason is a causation of decision, belief, feeling or acting (e.g. “I went because I though it would be interesting.”). In this work we consider all of them as causation. The distinction between influence and causation is a matter of degree. There is a continuum between totally independent events and totally casually related events. An influence holds between event1and event2if event1affects the manner or intensity of event2, but does not affect the occurrence of event2. If it affects the occurrence, then it is a causation. For example, “Targeting skin cancer relatives improves screening.” encodes an influence. purpose can be interpreted as an intended causation, i.e., event1is the purpose of event2if event1implies event2and the causer is willing the event2to exist. We 3 CAUSATIONS 19 don’t consider purpose in this work. As we have already mentioned, semantic relations may overlap, i.e., between two text spans more than one semantic relation may hold. A clear overlap exists between causation and temporal relations. By definition, the cause should always occur before the effect, so if event1causes event2, then event1should occur before than event2. We should make clear the distinction between two events causally related and two events correlated. The age of a human being and the size of his vocabulary tipically are correlated: the older, the more vocabulary. However, we all know that getting older does not cause to have a bigger vocabulary. 3.5 Encoding of Causation The basic arguments of a causation are the cause and the effect. From the point of view of detecting causations, the following distinctions may be useful: •Marked or unmarked A causation is marked if there is a specific linguistic unit that signal the relation. For example: –[marked]: I bought this book because I read a good review. –[unmarked]: Be careful. It’s unstable. Unmarked causations are more difficult to detect and usually it is not easy to decide whether they encode a causation or not. •Ambiguity If the causation is marked, the mark can signal always a causation or it can signal sometimes a causation. The former kind is unambiguous, the latter ambiguous. For example: –[unambiguous]: He died because of the extent of his injuries. –[ambiguous signaling causation]: The storm produced several small tornadoes and gustnadoes. –[ambiguous not signaling causation]: The factory produces two kinds of cable. •Explicit or implicit A causation is explicit if both arguments are present. If any of its arguments are missing, then it is a implicit causation. For example: –[explicit]: She was thrown out of the swanky Hotel Excelsior after she had run naked through its marble halls. 3 CAUSATIONS 20 –[implicit]: John killed Bob. Although it might not be obvious, the second example encodes a causation. The effect argument, Bob’s death, is not explicit. In this work, we focus in marked and explicit causations. 4 PREVIOUS WORK 21 4 Previous work In this section we present the work already done in Computational Linguistics on the detection and extraction of semantic relations and particularly on causations. 4.1 On Semantic relations One of the first proposed systems to perform semantic analysis was Tanka, [Barker and Szpakowicz, 1995]. The system performs semantic analysis in three levels: Clause-Level Relationships Analysis (semantic relationships between acts, events or states represented syntactically by syndetically connected finite clauses), Case Analysis (between the main verb and its syntactic arguments) and Noun-Modifier Relationships Analysis (between the head noun of a noun phrase and its modifiers). The Clause-Level Relationships they consider have been detailed in section 2.1. The system is based on a lexicon manually built and a set of rules that allows to decide the right relationships for a particular pair of text spans. The authors evaluate the system with a legal text; the vast majority of relationships suggested are correct. The evaluation is performed with 100 sentences randomly selected, 94 suggestions are correct and 2 wrong; the system cannot make a decision for 4 sentences and therefore the suggestion consists of a set of possible relationships. Semantic relations in Noun Phrases have been studied in [Moldovan et al., 2004]. Using the set of 35 semantic relations described in table 1 of section 2, they compare the performance of three learning models: Naive Bayes, Decision Trees and a new model, Semantic Scattering. The best results are obtained with Semantic Scattering, the F-measure varies from 0.33 to 0.75 depending on the syntactic pattern. As features, they only used the semantic classes of the head and modifier noun. In [Girju et al., 2004] the authors study the semantic relations in Nominalized Noun Phrases. They use a set of lexical, syntactic and semantic features. The best results are obtained with Support Vector Machines, the F-measure varies from 0.61 to 0.71 depending on the syntactic pattern. Other approaches work with Probability Theory and the lexico-syntactic patterns that express causations. These approaches almost don’t need supervision, but require a large corpus. In [Turney, 2006] the authors define the pertinence of a pattern for a word pair. When classifying noun-modifier pairs, they report a F-measure of 0.50. They only consider five semantic relations: causality,participant,quality,spatial and temporality. Their system is unsupervised and the only requirements are a large corpus (around 5 ×1010 words) and a set of words pairs with their corresponding labels (at least 600 pairs in their experiments). [Pantel and Pennacchiotti, 2006] also work with probabilities and propose two new measures: pattern reliability and instance reliability. The key idea is that in 5 THE METHOD 28 Pattern Matching Syntactic patterns that encode causation Corpus (SemCor 2.1) POS tagging and Chunker Shallow parsed sentences Matches No matches Figure 2: Flowchart of the pattern matching. and verbs the WordNet2.1 sense number. However, all the verb forms have the POS tag VB, which might not be enough if we want to tell the difference between different verb tenses. In order to get the right POS tag for all the verb forms, we use a POS tagger [Schmid, 1994]. While clustering the different examples in the syntactic pattern (1), we discovered that the most common relators expressing causation are as,after,because and since. Therefore, we decided to focus only in the causations that: •are instances of pattern (1), and •are signaled by the relator as, since, after or because. We found 1068 sentences that satisfy both conditions. The distribution of the relators is shown in table 5. Table 5: Distribution of the relators. Relator Cardinality because 381 as 330 after 228 since 129 Total 1068 The following is a real example contained in the corpus SemCor2.1. The format is the following: first column POS tag2, second lemma, third WordNet2.1 sense number and fourth word form: 2A list of tags and the parts of speech corresponding to them can be found in appendix A 5 THE METHOD 29 POS tag lemma wnsn word form NN ordinary 1 Ordinary NNP person 1 Williams VB say 1 said PRP he -1 he , , -1 , RB too 2 too , , -1 , VB was -1 was VB subject 1 subjected VB to -1 to JJ anonymous 1 anonymous NN call 1 calls RB soon 1 soon IN after -1 after PRP he -1 he VB schedule 1 scheduled DT the -1 the NN election 1 election . . -1 . After the POS tagging to attach the right POS tag for verbs and the chunking, the result is the following: POS tag lemma wnsn word form chunker NN ordinary 1 Ordinary B-NP NNP person 1 Williams I-NP VBD say 1 said B-VP PRP he -1 he B-NP , , -1 , O RB too 2 too B-ADVP , , -1 , O VBD be -1 was B-VP VBN subject 1 subjected I-VP IN to -1 to B-PP JJ anonymous 1 anonymous B-NP NN call 1 calls I-NP RB soon 1 soon B-ADVP IN after -1 after B-SBAR PRP he -1 he B-NP VBN schedule 1 scheduled B-VP DT the -1 the B-NP NN election 1 election I-NP . . -1 . O Note that the example shown matches the pattern we consider: Ordinary Williams said he, too, [was subjected]V P to anonymous calls soon [after]rel [he [scheduled]V PCthe election]C. Using the formula (2), the argument order is first effect and then cause, argumentsOrder =EC ∗middle =EC, so the effect is encoded in V P and the cause in V PC. Note that the four relators chosen not always signal a causation: 5 THE METHOD 30 •Rachel stayed on after the doctor had gone. •He has lived in many houses since he moved to the States. •She was trying to contain the bitterness of her voice as she enunciated her words too distinctly. 5.3 Machine Learning approach The rest of the tasks in order to create a model for the detection of causations are represented in figure 3. Matches Manual Clasification (causation / no causation) + feature selection Feature extraction ML algorithm Evaluation Training data Evaluation data Model Classified data Performance Figure 3: Flowchart of the learning process and evaluation. So far we have a set of sentences that are likely to express causation, since their syntactic structure is likely to encode one. Our goal is to classify them in sentences expressing causation and sentences not expressing causation. In order to do so, we decided to use Machine Learning techniques. The problem fits easily in the paradigm of classification. From now on, we will call instance the matches obtained after the syntactic pattern matching. Thus, an instance should be classified as cause if it encodes a causation, or ¬cause, if it doesn’t. Let us define Xas the set of all the instances and the set C={cause, ¬cause}. The function defined in (3) associates each instance with its right classification. Obviously fis unknown a priori. f:X−→ C(3) 5 THE METHOD 31 The challenge of the Machine Learning paradigm is to learn the ffunction based on some examples. In order to do so, the learning algorithm takes as input for each example a feature vector and its right classification. The algorithm is able to create a model that is an approximation of the function f. Once the model is learned, we need to estimate its performance. In order to do so, a new set of examples is needed. The performance is measured considering the differences between the classification according to the model obtained and the right classification. Typically the performance is not perfect; it varies a lot depending on the task. Formally, for each instance xi∈X, we define the vector Fi= (xi1, xi2. . . xim), corresponding to the features for the instance xi.xij denotes the feature jfor the instance i. We denote the set of examples used for training T r and the set used for testing T e.T r and T e are of the form {F1c1, F2c2. . . Fmcm};ci∈C. Since we have a finite number of instances, we need to divide them in the T r and T e sets at the very beginning. 5.3.1 Data preparation The data preparation basically consisted on classifying each instance as causation or not causation. Out of the 1068 instances, 517 were classified as cause and 551 as ¬cause3. This task is very slow and completely manual. A manual annotator has to examine each instance and decide if it encodes a causation or not. Furthermore, the task is not as easy as it may seem. Sometimes it is not clear whether a sentence encodes a causation or not; ambiguity occurs more often than expected. Consider the example “After four years of hard work, she graduated”. One could argue that the cause of her graduation is the fact that she worked hard for four years. However, taking into account only the information contained in the sentence we cannot conclude that a causation holds. 5.3.2 Feature selection At the same time than the manual classification, a feature selection was performed. It is not obvious what kind of features are the good ones for our problem. Typical features for Machine Learning approaches in the Natural Language Processing field are a mixture of lexical and syntactic; lately some researchers have achieved great results considering semantic features too. Following this approach, we look for clues in the three level of analysis that determine good features. 3This means the baseline for the classification task is 0.516, since 51.6% of the instances belong to ¬cause 5 THE METHOD 32 It may be argued that the effort needed to come up with a good set of features is too much. It is definitely a lot of work to do, specially considering that it is manual. However, the careful examination of the examples allows us to really understand the nature of causations. The set of features that were detected as potentially good features, their rationale, values, detection and examples are detailed below: 1. Relator •Rationale: Arelator can signal always, never or sometimes a causation. Furthermore, different combination of relators and other features can signal causation always, never or sometimes. •Values: because, as, after, since. •Detection: at the same time than the pattern matching •Examples: –Relators unambiguously causal: because ∗[cause]Part-time farmers generally must pay higher prices for supplies than full-time farmers because they buy in smaller quantities. ∗[cause]Leadership is lacking in our society because it has no legitimate place to develop. –Relators ambiguous: after,since,as ∗[¬cause]Frank and he had met about two years after she had arrived. ∗[cause]Marty stood for several moments with his mouth hanging open foolishly after it had happened. ∗[¬cause]They had been at lessons in the schoolhouse since they returned from Harpers Ferry. ∗[cause]The children are satisfied with smaller amounts of food since all of it is high in quality. 2. Relator Modifiers •Rationale: causations can hardly be signaled by a relator modified by an adverb or preposition. •Values: POS tags. •Detection: at the same time than the pattern matching. •Examples: –adverb + after almost always signals a temporal relation, not a causation: 5 THE METHOD 33 ∗Tom Brannon had caught up with the outfit shortly after the Maguires joined it. ∗This was long after Morse had left the house. –as + preposition can hardly signal a causation: ∗. . . he felt he was noting it, as if it were something he might think about when . . . ∗Alex nodded to the maid as though nothing unusual were taking place and entered the doctor’s room. 3. Semantic Class Cause Verb •Rationale: only certain verbs can express a cause. •Values: WordNet2.1 semantic class4. •Detection: at the same time than the pattern matching. •Examples: –If the relator is after and the cause verb semantic class is be-v-3 56, then it is a temporal relation, not a causation: ∗We heard him before he ever showed, and we heard him yelling after he was out of sight. 4. Cause Verb is Potentially Causal •Rationale: if a verb sense’s gloss contains the words cause to or change, or is subsumed by a verb sense that contains the words cause to or change, then is more likely to express a cause 7. •Values: yes, no. •Detection: examine gloss cause verbs and its subsumers •Examples: –ring-v-1 is subsumed by sound-v-2, which gloss is “cause to sound”. 5. Semantic Class Effect Verb •Rationale: only certain verbs can express an effect. •Values: WordNet2.1 semantic class. •Detection: at the same time than the pattern matching. 4By Semantic Class we mean the most common subsumer of the verb in WordNet2.1. For example, the semantic class of ruminate-v-1 is eat-v-2 5be-v-3 should be readed as the third meaning of the verb be 6The gloss of be-v-3 is “occupy a certain position or area; be somewhere; ‘Where is my umbrella?’ ‘The toolshed is in the back’; ‘What is behind this behavior?’ ” 7According with the definition, we found 7,370 out of 24,890 verb senses potencially causal 5 THE METHOD 34 •Examples: –react-v-1’s semantic class is act-v-1,allow-v-10’s semantic class is act-v-1 and marginalize-v-1 ’s semantic class is act-v-1. –If the relator is after and the effect verb semantic class is expressv-28, then is not a causation: ∗“My name’s Gisele”, the blonde said after she ordered a Scotch. 6. Effect Verb is Potentially Causal •Rationale: if a verb sense’s gloss contains the words cause to or change, or is subsumed by a verb sense that contains the words cause to or change, then is more likely to express an effect. •Values: yes, no. •Detection: examine gloss cause verbs and its subsumers •Examples: –walk-v-3 is subsumed by travel-v-1, which gloss is “change location’. 7. Verb Tense Cause and Effect Verb •Rationale: depending on the relator, some verb tenses are not likely to express causation •Values: present, past, future, perfective, progressive, conditional, obligation, possibility, . . . •Detection: examine the POS tags for the cause and effect verbs •Examples: –If the relator is as or after, the cause verb is not a copula (semantic class different than be-v-1 9), and the cause verb tense is present, then is not a causation: ∗[¬cause]Henrietta was discovering in the process of writing, as the born writer does, not merely . . . ∗[¬cause]To play the guitar as he aspires will devour his . . . ∗[¬cause]The Thayer Schools offers a year of postgraduate study in somewhat the same way, after a boy wins a B.S. in engineering. ∗[cause]. . . you must forgive me as I am so forgetful. 8The gloss of express-v-2 is “articulate; either verbally or with a cry, shout, or noise; ‘She expressed her anger’; ‘He uttered a curse’ ” 9The gloss of be-v-1 is “have the quality of being; (copula, used with an adjective or a predicate noun); ‘John is rich’; ‘This is not a good answer’ ” 5 THE METHOD 35 –If the relator is as and the effect verb is conditional, then is not a causation: ∗She wouldn’t go to New York as Maude suggested . . . –If the effect verb is progressive, then is not a causation: ∗The burden of his secret was pressing down on him, as it was on Lieutenant Beckstrom and his six enlisted men. ∗. . . said Juanita, holding her face very still, trying to contain the bitterness of her voice as enunciated her words too distinctly. –if the relator is as and the effect verb express obligation, then is a causation: ∗You must do that as I helped you. –if the effect verb es passive, then it is more likely to express a causation: ∗. . . and then Richard was shocked as, all at once, flames shot out from the sharp features of . . . 8. Cause and Effect Verb Same Tense •Rationale: if both the cause and effect verb are past simple, it can hardly be a causation, most of the time it is just a temporal relation. •Values: yes, no. •Detection: cause and effect verb same tense •Examples: –...failed to flourish in New England as it did in other parts of the country. Fletched nodded as he listened to the instructions and said he would . . . 5.3.3 Feature extraction The feature extraction is done extracting the information detailed below from the relator, the words surrounding it, and both VPs. Note that with the four relators chosen, VP express allways the effect and the VP contained in C (VPC) the cause. The final set of features are the following: •relator = {since, because, as, after} •relatorLeftModification = {POS tags} •relatorRightModification = {POS tags} •SemanticClassVCause = {WordNet2.1 sense number} 5 THE METHOD 36 •verbCauseIsPotentiallyCausal = {yes, no} •SemanticClassVEffect = {WordNet2.1 sense number} •verbEffectIsPotentiallyCausal = {yes, no} •For both Verb Phrases, the following features are extracted: –Present = {yes, no} –Past = {yes, no} –Modal = {conditional, obligation, possibility, future, no} –Perfective = {yes, no} –Progressive = {yes, no} –Passive = {yes, no} We exemplify the feature extraction with the sentence presented in section 5.2, Ordinary Williams said he, too, [was subjected]V P to anonymous calls soon [after]rel [he [scheduled]V PCthe election]C: relator after cPresent no ePresent no relatorLeftModification RB cPast yes ePast yes relatorRightModification PRP cModal no eModal no semanticClassVCause 1 cPerfective no ePerfective no verbCauseIsPotentiallyCausal yes cProgressive no eProgressive no semanticClassVEffect 1 cPassive no ePassive yes verbEffectIsPotentiallyCausal no In the example, ‘cPresent’ means “cause verb tense is present”, ‘ePresent’ means “effect verb tense is present”, and so on. The extraction of the non obvious features is depicted below: •We can easily get the main verb of a VP: it always corresponds to the last element of the VP [Quirk et al., 1985]. Since we have already identified the VP encoding cause and effect during the pattern matching, we can easily get their semantic classes looking for the information contained in the corpus. •Once we know the semantic class, we only need to examine the gloss to know if any of the verbs is potentially causal. •The boolean features that identify the tense of the VP are defined as follows: –Present: yes if the POS tags VBP or VPZ are present in the VP; no otherwise. –Past: yes if the POS tag VBD is present in the VP; no otherwise. 5 THE METHOD 37 –Modal: ∗conditional if the lemma would is present in the VP, ∗obligation if the lemmas must,ought or should are present in the VP, ∗possibility if the lemmas can,could,may or might are present in the VP, ∗future if the lemmas will or shall are present in the VP, ∗no otherwise. –Perfective: yes if the construction have + VBN (lemma have followed by participle) is present; no otherwise. –Progressive: yes if the POS tag VBG is present in the VP; no otherwise. –Passive: yes if the construction be + VBN (lemma be followed by participle) is present; no otherwise. Note that different combination of these boolean features can capture tenses like ‘past and passive’, e.g. was subjected, or ‘obligation, present and perfective’, e.g. should have gone. 5.3.4 Machine Learning algorithm We tried different Machine Learning algorithms using 10-fold cross-validation. The best results obtained are shown in table 6. They were obtained using Bagging with C4.5 trees. Table 6: Results obtained using Bagging with C4.5 trees during training. Class Precision Recall F-Measure cause 0.969 0.839 0.899 ¬cause 0.865 0.975 0.917 The measures for evaluating the performance of the Machine Learning algorithm are the most commonly used. P recision measures the portion of the assigned categories that were correct; Recall the portion of the correct categories that were assigned; F-measure is the weighted harmonic mean of P recision and Recall. The three measures range from 0 to 1. Formally, we can define the three measures as follows: Correct = X Correct = Y Assigned = X a b Assigned = Y c d 7 CONCLUSIONS AND FURTHER WORK 44 7 Conclusions and further work We have proposed a system which yields a high Precision and Recall for the detection of causations encoded by one of the most common syntactic patterns expressing a causation: [VP rel C], [rel C, VP]. The system is relatively simple and is able to detect causations encoded in an open domain text. So far research in causal relation detection has focused in causation expressed with noun phrases, e.g. “The [incident]NP1provoked [widespread protest]NP2’. We focused on the detection of causation between a verb and a subordinate clause. [Barker and Szpakowicz, 1995] also work with clauses, but their approach is very different. The system proposed uses shallow syntactic parsing. State of the art chunkers yield performance of 95 %, so we can assume we detect most of the pattern instances present in the input. However, chunkers take as input POS tagged text, and POS taggers also make mistakes. A key element to really see the potential of the system would be to integrate it with other systems that extract semantic relations. By doing so we could experiment with inference rules that combine causation and other semantic relations. For example, if event1causes event2and event3is subsumed by event1, we can conclude than event3causes event2; if event1causes event2and event2 entails event3, then event1causes event3. Another possible inference rule would express the transivity property of causations. An example may clarify the last two examples: •Being tired makes me snore. We all know that in order to snore you need to be asleep. causation(being tired, snore)∧entail(snoring, sleeping)⇒causation(being tired, sleep) •He died of cancer. Cancer is caused by smoking. causation(cancer, death)∧causation(smoking, cancer)⇒causation(smoking, death) Note that the inference rule allows us to extract more causal knowledge, but the causations extracted are indirect. The discovering of new rules and the validation of the new rules seem to be a great challenge. Another possible extension would be to deal with causal chains and intricate causal relations. A causal chain can be defined as a sequence of events that lead up to some final effect. Examples (1) and (2) exemplify a causal chain and a intricate causal relation respectively. 7 CONCLUSIONS AND FURTHER WORK 45 1. Artworks become art only when they transcend the simple facts of their existence, and they can do that only when they blend with the sensibility of the viewer. (they (artworks) blend with the sensibility of the viewer)⇒(they (artworks) transcend the simple facts of their existence)⇒(artworks become art)10. 2. It is lined primarily by industrial developments and concrete-block walls because the constant traffic and emissions do not make it an attractive neighborhood. ((constant traffic and emissions)⇒(not an attractive neighborhood)) ⇒ (lined up by industrial developments and concrete-block walls). 10‘x ⇒y’ should be readed as “x causes y” REFERENCES 46 References [Baker et al., 1998] Baker, C. F., Fillmore, C. J., and Lowe, J. B. (1998). The Berkeley FrameNet project. In Proceedings of the Thirty-Sixth Annual Meeting of the Association for Computational Linguistics and Seventeenth International Conference on Computational Linguistics, pages 86–90, San Francisco, CA, USA. Morgan Kaufmann Publishers. [Barker and Szpakowicz, 1995] Barker, K. and Szpakowicz, S. (1995). Interactive semantic analysis of clause-level relationships. In Proceedings of the Second Conference of the Pacific Association for Computational Linguistics (PACLING– 95), pages 22–30, Brisbane, Australia. [Blaheta and Charniak, 2000] Blaheta, D. and Charniak, E. (2000). Asigning function tags to parsed text. In Proceedings of the 1st Annual Meeting of the North American Chapter of Association for Computational Linguistics (NAACL), Seattle, WA, USA. [Chang and Choi, 2006] Chang, D.-S. and Choi, K.-S. (2006). Incremental cue phrase learning and bootstrapping method for causality extraction using cue phrase and word pair probabilities. Information Processing and Management: an International Journal, 42(3):662–678. [Eagleman and Holcombe, 2002] Eagleman, D. and Holcombe, A. (2002). Causality and the perception of time. Trends in Cognitive Sciences, 6(8):323–325. [Garcia, 1997] Garcia, D. (1997). Coatis, an nlp system to locate expressions of actions connected by causality links. In EKAW ’97: Proceedings of the 10th European Workshop on Knowledge Acquisition, Modeling and Management, pages 347–352, London, UK. Springer-Verlag. [Gildea and Jurafsky, 2002] Gildea, D. and Jurafsky, D. (2002). Automatic labeling of semantic roles. Computational Linguistics, 28(3):245–288. [Girju, 2003] Girju, R. (2003). Automatic detection of causal relations for question answering. In Proceedings of the 41st ACL, Workshop on Multilingual Summarization and Question Answering. [Girju et al., 2003] Girju, R., Badulescu, A., and Moldovan, D. (2003). Learning semantic constraints for the automatic discovery of part-whole relations. In NAACL ’03: Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology, pages 1–8, Morristown, NJ, USA. [Girju et al., 2006] Girju, R., Badulescu, A., and Moldovan, D. (2006). Automatic discovery of part-whole relations. Computational Linguistics, 32(1):83–135. REFERENCES 47 [Girju et al., 2004] Girju, R., Giuglea, A.-M., Olteanu, M., Fortu, O., Bolohan, O., and Moldovan, D. (2004). Support vector machines applied to the classification of semantic relations in nominalized noun phrases. In Proceedings of the HLT-NAACL Workshop on Computational Lexical Semantics. [Girju and Moldovan, 2002] Girju, R. and Moldovan, D. (2002). Mining answers for causation questions. In Proceedings of the American Association for Artificial Intelligence (AAAI) - Spring Symposium. [Glymour, 2003] Glymour, C. (2003). Learning, prediction and causal bayes nets. Trends in Cognitive Sciences, (7):43–48. [Hitchcock, 2007] Hitchcock, C. (Fall 2007). Probabilistic causation. In Zalta, E. N., editor, The Stanford Encyclopedia of Philosophy. The Metaphysics Research Lab. Center for the Study of Language and Information. Stanford University. [Hobbs, 2005] Hobbs, J. R. (2005). Toward a useful concept of causality for lexical semantics. Journal of Semantics, 22(2):181–209. [Hume, 1739] Hume, D. (1978, first published anonymously in 1739). A treatise of human nature. Oxford University Press. [Joskowiscz et al., 1989] Joskowiscz, L., Ksiezyk, T., and Grishman, R. (1989). Deep domain models for discourse analysis. In The annual AI systems in government conference, pages 195–200. [Kaplan and Berry-Rogghe, 1991] Kaplan, R. M. and Berry-Rogghe, G. (1991). Knowledge-based acquisition of causal relationships in text. Knowl. Acquis., 3(3):317–337. [Khoo et al., 2000] Khoo, C. S. G., Chan, S., and Niu, Y. (2000). Extracting causal knowledge from a medical database using graphical patterns. In ACL ’00: Proceedings of the 38th Annual Meeting on Association for Computational Linguistics, pages 336–343, Morristown, NJ, USA. Association for Computational Linguistics. [Kingsbury et al., 2002] Kingsbury, P., Palmer, M., and Marcus, M. (2002). Adding semantic annotation to the penn treebank. In Proceedings of the Human Language Technology Conference (HLT’02). [Lakoff, 1987] Lakoff, G. (1987). Women, Fire, and Dangerous Things. University of Chicago Press. [Lewis, 1973a] Lewis, D. (1973a). Causation. Journal of Philosophy, 70:556–567. [Lewis, 1973b] Lewis, D. (1973b). Counterfactuals. Oxford: Blackwell. REFERENCES 48 [Mackie, 1980] Mackie, J. L. (1980). The cement of the Universe. A Study of Causation. Oxford University Press. [Marcu and Echihabi, 2002] Marcu, D. and Echihabi, A. (2002). An unsupervised approach to recognizing discourse relations. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL’02), pages 368– 375. [Mill, 1843] Mill, J. (1967, original work published 1843). A system of logic ratiocinative and inductive. Longman. [Moldovan et al., 2004] Moldovan, D., Badulescu, A., Tatu, M., Antohe, D., and Girju, R. (2004). Models for the semantic classification of noun phrases. Proceedings of the Computational Lexical Semantics Workshop at HLT-NAACL. [Murali, 1997] Murali, R. (1997). A counterfactual analysis of causation. Mind, (106):263–277. [Pantel and Pennacchiotti, 2006] Pantel, P. and Pennacchiotti, M. (2006). Espresso: leveraging generic patterns for automatically harvesting semantic relations. In ACL ’06: Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL, pages 113–120, Morristown, NJ, USA. [Quinlan, 1993] Quinlan, J. (1993). C4.5: Programs for Machine Learning. Morgan Kaufmann. [Quinlan, 1996] Quinlan, J. (1996). Bagging, boosting and c4.5. In AAAI/IAAI, Vol. 1, pages 725–730. [Quirk et al., 1985] Quirk, R., Greenbaum, S., Leech, G., and Svartvick, J. (1985). A Comprehensive Grammar of the English Language. Longman. [Schmid, 1994] Schmid, H. (1994). Probabilistic part-of-speech tagging using decision trees. In International Conference on New Methods in Language Processing, Manchester, UK. [Shibatani, 1976] Shibatani, M. (1976). The Grammar of Causative Constructions: A Conspectus. Academic Press. [Sosa, 1975] Sosa, E. (1975). Causations and conditionals. Oxford University Press. [Stefanowitsch, 2001] Stefanowitsch, A. (2001). Construction Causation: A Counstructional Grammar Approach to Analytic Causatives. PhD thesis, Rice University. REFERENCES 49 [Suppes, 1970] Suppes, P. (1970). A probabilistic theory of causality. NorthHolland Publishing Co. [Talmy, 2000] Talmy, L. (2000). Toward a Cognitive Semantics. MIT Press. [Tatu, 2005] Tatu, M. (2005). Automatic discovery of intentions in text and its application to question answering. In Proceedings of the ACL Student Research Workshop, Ann Arbon, MI, USA. [Turney, 2006] Turney, P. D. (2006). Expressing implicit semantic relations without supervision. In ACL ’06: Proceedings of the 21st International Conference on Computational Linguistics and the 44th annual meeting of the ACL, pages 313–320, Morristown, NJ, USA. [White, 1990] White, P. (1990). Ideas about causation in philosophy and psychology. Psychological Bulletin, 108(1):3–18. [Witten and Frank, 2005] Witten, I. and Frank, E. (2005). Data Mining: Practical machine learning tools and techniques. Morgan Kaufmann. A LIST OF POS TAGS 50 A List of POS tags Tag Part of speech CC Coordinating conjunction CD Cardinal number DT Determiner EX Existential there FW Foreign word IN Preposition or subordinating conjunction JJ Adjective JJR Adjective, comparative JJS Adjective, superlative LS List item marker MD Modal NN Noun, singular or mass NNS Noun, plural NNP Proper noun, singular NNPS Proper noun, plural PDT Predeterminer POS Possessive ending PRP Personal pronoun PRP$ Possessive pronoun RB Adverb RBR Adverb, comparative RBS Adverb, superlative RP Particle SYM Symbol TO to UH Interjection VB Verb, base form VBD Verb, past tense VBG Verb, gerund or present participle VBN Verb, past participle VBP Verb, non-3rd person singular present VBZ Verb, 3rd person singular present WDT Wh-determiner WP Wh-pronoun WP$ Possessive wh-pronoun WRB Wh-adverb B DECISION TREES LEARNED 51 B Decision trees learned B.1 Decision Tree no. 1: relator = after: n relator = as: n relator = since | relatorLeftModification = yJJ: y | relatorLeftModification = yIN: y | relatorLeftModification = yJJR: y | relatorLeftModification = yVBZ: y | relatorLeftModification = yDT: y | relatorLeftModification = yTO: y | relatorLeftModification = yVBG: y | relatorLeftModification = yRB: n | relatorLeftModification = yVBP: n | relatorLeftModification = yVBD: y | relatorLeftModification = yPRP: y | relatorLeftModification = yCD: y | relatorLeftModification = yVBN | | ePerfective = yes: n | | ePerfective = no | | | lexicalClue = yes: n | | | lexicalClue = no: y | relatorLeftModification = yCC: n | relatorLeftModification = yVB: y | relatorLeftModification = yNNP: y | relatorLeftModification = yNNS: n | relatorLeftModification = yNN | | relatorRightModification = yPP: n | | relatorRightModification = yJJ: y | | relatorRightModification = y*: n | | relatorRightModification = yIN: y | | relatorRightModification = yVBZ: n | | relatorRightModification = yDT | | | verbCausePotCausal = yes: y | | | verbCausePotCausal = no: n | | relatorRightModification = yTO: n | | relatorRightModification = yVBG: n | | relatorRightModification = yRB: y | | relatorRightModification = yWDT: n | | relatorRightModification = yVBP: n | | relatorRightModification = yVBD: n | | relatorRightModification = yPRP: n | | relatorRightModification = yVBN: n | | relatorRightModification = yCD: n | | relatorRightModification = yWRB: n | | relatorRightModification = yVB: n | | relatorRightModification = yNNP: n | | relatorRightModification = yNNS: n | | relatorRightModification = yEX: n | | relatorRightModification = yMD: n | | relatorRightModification = yPDT: n | | relatorRightModification = yNN: n | | relatorRightModification = yWP: n | | relatorRightModification = yPUNCT: n | relatorLeftModification = yPUNCT: y relator = because: y Number of Leaves : 49 Size of the tree : 55 B.2 Decision Tree no. 2: relator = after: n B DECISION TREES LEARNED 52 relator = as: n relator = since | ePerfective = yes: n | ePerfective = no | | relatorLeftModification = yJJ: y | | relatorLeftModification = yIN: y | | relatorLeftModification = yJJR: y | | relatorLeftModification = yVBZ: y | | relatorLeftModification = yDT: y | | relatorLeftModification = yTO: y | | relatorLeftModification = yVBG: y | | relatorLeftModification = yRB: y | | relatorLeftModification = yVBP: n | | relatorLeftModification = yVBD: y | | relatorLeftModification = yPRP: y | | relatorLeftModification = yCD: y | | relatorLeftModification = yVBN: y | | relatorLeftModification = yCC: n | | relatorLeftModification = yVB: y | | relatorLeftModification = yNNP: y | | relatorLeftModification = yNNS: n | | relatorLeftModification = yNN | | | cPast = yes: n | | | cPast = no: y | | relatorLeftModification = yPUNCT: y relator = because: y Number of Leaves : 24 Size of the tree : 28 B.3 Decision Tree no. 3: relator = after: n relator = as: n relator = since | ePerfective = yes: n | ePerfective = no | | relatorLeftModification = yJJ: y | | relatorLeftModification = yIN: n | | relatorLeftModification = yJJR: y | | relatorLeftModification = yVBZ: y | | relatorLeftModification = yDT: y | | relatorLeftModification = yTO: y | | relatorLeftModification = yVBG: y | | relatorLeftModification = yRB: n | | relatorLeftModification = yVBP: n | | relatorLeftModification = yVBD: y | | relatorLeftModification = yPRP: y | | relatorLeftModification = yCD: y | | relatorLeftModification = yVBN | | | relatorRightModification = yPP: y | | | relatorRightModification = yJJ: y | | | relatorRightModification = y*: y | | | relatorRightModification = yIN: y | | | relatorRightModification = yVBZ: y | | | relatorRightModification = yDT: y | | | relatorRightModification = yTO: y | | | relatorRightModification = yVBG: y | | | relatorRightModification = yRB: y | | | relatorRightModification = yWDT: y | | | relatorRightModification = yVBP: y | | | relatorRightModification = yVBD: y | | | relatorRightModification = yPRP: y | | | relatorRightModification = yVBN: y | | | relatorRightModification = yCD: y B DECISION TREES LEARNED 53 | | | relatorRightModification = yWRB: y | | | relatorRightModification = yVB: n | | | relatorRightModification = yNNP: y | | | relatorRightModification = yNNS: y | | | relatorRightModification = yEX: y | | | relatorRightModification = yMD: y | | | relatorRightModification = yPDT: y | | | relatorRightModification = yNN: y | | | relatorRightModification = yWP: y | | | relatorRightModification = yPUNCT: y | | relatorLeftModification = yCC: n | | relatorLeftModification = yVB: y | | relatorLeftModification = yNNP: y | | relatorLeftModification = yNNS: n | | relatorLeftModification = yNN | | | cPast = yes: n | | | cPast = no: y | | relatorLeftModification = yPUNCT: y relator = because: y Number of Leaves : 48 Size of the tree : 53 B.4 Decision Tree no. 4: relator = after: n relator = as: n relator = since | ePerfective = yes: n | ePerfective = no | | relatorLeftModification = yJJ | | | lexicalClue = yes: n | | | lexicalClue = no: y | | relatorLeftModification = yIN: n | | relatorLeftModification = yJJR: y | | relatorLeftModification = yVBZ: y | | relatorLeftModification = yDT: y | | relatorLeftModification = yTO: y | | relatorLeftModification = yVBG: y | | relatorLeftModification = yRB | | | cPast = yes: n | | | cPast = no: y | | relatorLeftModification = yVBP: n | | relatorLeftModification = yVBD: y | | relatorLeftModification = yPRP: y | | relatorLeftModification = yCD: y | | relatorLeftModification = yVBN | | | lexicalClue = yes: n | | | lexicalClue = no: y | | relatorLeftModification = yCC: n | | relatorLeftModification = yVB: y | | relatorLeftModification = yNNP: y | | relatorLeftModification = yNNS: n | | relatorLeftModification = yNN | | | cPast = yes: n | | | cPast = no: y | | relatorLeftModification = yPUNCT: y relator = because: y Number of Leaves : 27 Size of the tree : 34 B.5 Decision Tree no. 5: relator = after: n B DECISION TREES LEARNED 60 | | relatorLeftModification = yCC: n | | relatorLeftModification = yVB: y | | relatorLeftModification = yNNP: y | | relatorLeftModification = yNNS: n | | relatorLeftModification = yNN | | | cPast = yes | | | | relatorRightModification = yPP: n | | | | relatorRightModification = yJJ: n | | | | relatorRightModification = y*: n | | | | relatorRightModification = yIN: n | | | | relatorRightModification = yVBZ: n | | | | relatorRightModification = yDT: y | | | | relatorRightModification = yTO: n | | | | relatorRightModification = yVBG: n | | | | relatorRightModification = yRB: y | | | | relatorRightModification = yWDT: n | | | | relatorRightModification = yVBP: n | | | | relatorRightModification = yVBD: n | | | | relatorRightModification = yPRP: n | | | | relatorRightModification = yVBN: n | | | | relatorRightModification = yCD: n | | | | relatorRightModification = yWRB: n | | | | relatorRightModification = yVB: n | | | | relatorRightModification = yNNP: n | | | | relatorRightModification = yNNS: n | | | | relatorRightModification = yEX: n | | | | relatorRightModification = yMD: n | | | | relatorRightModification = yPDT: n | | | | relatorRightModification = yNN: n | | | | relatorRightModification = yWP: n | | | | relatorRightModification = yPUNCT: n | | | cPast = no: y | | relatorLeftModification = yPUNCT: y relator = because: y Number of Leaves : 48 Size of the tree : 53