Torii: An aspect-based sentiment analysis system that can mine conditions

Torii: An aspect-based sentiment analysis system that can mine conditions Fernando O. Gallego Rafael Corchuelo Escuela Técnica Superior de Ingeniería Informática, University of Seville, Seville, Spain Correspondence Fernando O. Gallego, Escuela Técnica Superior de Ingeniería Informática, University of Seville, Avenida Reina Mercedes, s/n, 41012 Seville, Spain. Email: [email protected] Funding information Spanish R&D programme, Grant/Award Number: TIN2013-40848-R and TIN2016-75394-R; Opileak Summary Aspect-based sentiment analysis systems are a kind of text-mining systems that specialize in summarizing the sentiment that a collection of reviews convey regarding some aspects of an item. There are many cases in which users write their reviews using conditional sentences; in such cases, mining the conditions so that they can be analyzed is very important not to misinterpret the corresponding sentiment summaries. Unfortunately, current commercial systems or research systems neglect conditions; current frameworks and toolkits do not provide any components to mine them; furthermore, the proposals in the literature are insufficient because they are based on handcrafted patterns that fall short regarding recall or machine learning procedures that are tightly bound with a specific language and require too much configuration. In this article, we present Torii, which is a system that loads a collection of reviews, discovers the aspects on which they report, and summarizes the sentiment that is conveyed on them taking into account the existing conditions, if any. We also describe its architecture, our approach to mine conditions, and our experimental analysis on a large multilingual data set with reviews from multiple categories. To the best of our knowledge, Torii is the first proposal that addresses aspect-based sentiment analysis taking conditions into account. KEYWORDS deep learning, identification of aspects, mining conditions, sentiment analysis 1INTRODUCTION The Web is growing at an ever increasing pace, which confirms that we are in the age of data. Customer reviews are a major source of such data. It is then not surprising that many companies are increasingly interested in analyzing them automatically.1Aspect-based sentiment analysis systems2are a particular kind of text-mining systems that can help in this task. They specialize in discovering the different aspects of an item that are discussed in a collection of reviews and whether the sentiment that is conveyed on them is positive, negative, or neutral. For instance, analyzing a collection of reviews like “It's a pity that they don't provide a virtual credit card when you register at their offices” unveils a negative sentiment that may help a bank discover a new market need; analyzing a collection of reviews like “My first impression was that I didn't feel too excited about the new box design” also unveils a negative sentiment that may help a sushi company forecast the impact of a business decision; and analyzing a collection of reviews like “The check-out page is very responsive on desktop computers” unveils a positive sentiment that may help a retailer understand their strengths. However, note that the sentiment about the credit cards and the new box design cannot be generally flagged as negative due to the influence of conditions “when you register at their offices” and “my first impression”; it is the fact that no virtual credit card is provided when someone registers at a bank's office what is considered negative. Similarly, the initial negative sentiment about the new box does not necessarily entail that the customer feels unhappy with it now; analogously, condition “on desktop computers” entails that the positive sentiment about the check-out page is valid on desktop computers, not necessarily on tablets or mobile phones. Clearly, mining conditions and including them in the sentiment summaries is very important, since, otherwise, they might be very easily misinterpreted. Simply put: The sentiment that a review without conditions conveys may be globally flagged as either positive, negative, or neutral; the sentiment that a review with conditions conveys cannot. Unfortunately, current commercial or research text-mining systems3-8 neglect conditions. Endowing such systems with the ability to mine conditions is not trivial because existing natural language processing frameworks9,10 and toolkits11-15 do not provide any appropriate components and the results in the literature do not provide a conclusive solution. The naivest approaches search for user-defined patterns that build on syntactic anchors.16,17 They generally attain good precision, but fall short regarding recall because there are a few usual ways to introduce conditions that can be easily modeled with user-defined patterns, but too many unusual ways that cannot be easily modeled using such an approach (see the previous examples). There is only one machine-learning proposal18 that attempts to deal with such unusual conditions. Unfortunately, it is bound with the Japanese language, it must be configured with several specific-purpose dictionaries, taxonomies, and heuristics, and it was evaluated on a small data set with 3 155 sentences from hotel reviews. In this article, we present Torii, which is an aspect-based sentiment analysis system whose most salient feature is that it is the only system that can mine conditions. It can easily scale due to its microservice, message-driven architecture and its user interface was designed to facilitate analyzing collections of reviews by means of aspect and keyword filters. Its condition mining process relies on a deep neural approach that performs sequence labeling on the input sentence. We have performed a comprehensive experimental analysis on a public data set with 4.7 million sentences in English, Spanish, French, and Italian that were sampled from a variety of reviews on 16 common categories. The rest of the article is organized as follows. Section 2 reports on the related work, Section 3 presents our system architecture, Section 4 describes our solution to mine conditions, Section 5 reports on our experimental analysis, and finally, Section 6 presents our conclusions. 2RELATED WORK There are many commercial text-mining services, eg, Khoros,*Sprout Social,†Lexalytics,‡Brand Watch,§Sysomos,¶or Opileak.#They all can analyze text from sources like blogs, eg, WordPress or BlogSpot; social networks, eg, Twitter or Facebook; customer review sites, eg, LeGuide or Kelkoo; or news sites, eg, New York Times or NBC News, just to mention a few. Their focus is on extracting topics, measuring the impact of some contents, performing aspect-based sentiment analysis, or computing volumetric insights. There are also several research text-mining systems. For instance, entity-relation extractors3,4 mine entities and relations that are used to populate a knowledge base, aspect-based sentiment analysis systems5,6 analyze the sentiment that people express in social media regarding various aspects of a product or a service, and recommenders7,8 make suggestions building on a user's search history, previous orders, or the messages that he or she has exchanged with other people. Unfortunately, none of the previous text-mining systems can mine conditions. Regarding text-mining frameworks, UIMA9and GATE10 range among the most popular ones. They both help design and implement pipelines in which each stage gets an input message plus the annotations produced by the previous stages and computes further annotations. The annotations range from the word stems or part-of-speech tags of the input words to their dependency tags and sentiment scores, to mention a few. Both frameworks have a variety of components available, which eases using them in real-world projects. They both support multiple languages and can read text from a variety of sources, including plaintext, HTML documents, PDF documents, or databases. There are also many toolkits that provide components to implement specific natural-language-processing tasks, eg, Stanford Core NLP,11 Freeling,12 and NLTK.13 *Khoros is available at https://khoros.com/ †Sprout Social is available at https://sproutsocial.com/ ‡Lexalytics is available at https://www.lexalytics.com/ §Brand Watch is available at https://www.brandwatch.com/ ¶Sysomos is available at https://sysomos.com/ #Opileak is available at https://opileak.es Recently, toolkits like OpenNLP14 and spaCy15 have gained impetus due to their focus on efficiency without sacrificing effectiveness. Unfortunately, none of the previous frameworks or toolkits provide any components to mine conditions. In the literature, Narayanan et al,19 Skeppstedt et al,20 Mausam et al,16 Mausam,21 Chikersal et al,17 and Nakayama and Fujii18 have worked on related proposals. Narayanan et al19 highlighted the problems of not dealing with conditions in the field of opinion mining. Their proposal is a machine-learning model to compute the sentiment of a conditional sentence, but they did not report on a proposal to mine the conditions; they assumed that the sentences were previously labeled so as to make the conditions explicit. Recently, Skeppstedt et al20 presented a complementary proposal that can automatically classify a sentence as speculative, contrast, or conditional, but neither was it their goal to mine conditions. The naivest proposals to mine conditions are based on searching for user-defined patterns that rely on syntactic anchors. Mausam et al16 studied the problem in the field of entity-relation extraction; they realized that many usual conditions can be identified by locating adverbial clauses whose first word is one of the 16 one-word condition connectives in English; unfortunately, they did not report on the effectiveness of their proposal to mine conditions, only on the overall effectiveness of their proposal for entity-relation extraction. Their system was updated recently with new features,21 but their proposal to mine conditions was not. Chikersal et al17 reported on a similar, but simpler proposal: Search for sequences of words in between connectives “if,” “unless,” “until,” and “in case,” and the first occurrence of word “then” or a comma. Unfortunately, the previous proposals are not appealing because there are many unusual ways to introduce conditions, which makes handcrafting patterns with high recall very difficult; furthermore, it is not straightforward to adapt them to other languages in which common connectives are multiword or there is not a unique, context-agnostic translation for some English connectives. We confirmed the previous claims with our experimental analysis. The only existing machine-learning proposal was introduced by Nakayama and Fujii,18 who worked in the field of opinion mining in Japanese. They devised a model that is based on features that are computed by means of a syntactic parser and a semantic analyzer. The former identifies so-called bunsetus, which are Japanese syntactic units that consists of one independent word and one or more ancillary words, as well as their interdependencies; the latter identifies opinion expressions, which requires to provide some specific-purpose dictionaries, taxonomies, and heuristics. They used conditional random fields and support vector machines to learn classifiers that can check whether a bunsetu is a condition or not. Unfortunately, their proposal was only evaluated on a small data set with 3 155 sentences from hotel reviews, and the best F1score attained was 0.58. As a conclusion, this proposal is not generally applicable and its effectiveness seems to be poor. It is clear that some researchers have already realized that mining conditions is very important. Unfortunately, the state of the art does not provide a concluding solution, namely, existing text-mining services and frameworks do not provide any components to mine them and the results in the literature of condition mining fall short. This motivated us to work on Torii, which is the only aspect-based sentiment analysis system that can mine conditions. It has been designed using a microservice, message-driven architecture that helps it scale according to the workload. Its approach to mine conditions is based on a deep neural network that has proven to be as precise as handcrafted approaches, but it attains better recall due to its ability to project the input sentences onto a rich computer-generated feature space in which a machine can discover patterns that are not easy to spot for a person. 3SYSTEM ARCHITECTURE Our architecture relies on three components to which we refer to as Data Storage,Data Processors,andClient Application.TheData Storage component is responsible for persisting and indexing reviews; the Data Processors component provides a set of microservices to load reviews, plus microservices to mine conditions, discover new aspects, and summarize the sentiment; and the Client Application component provides user interfaces to interact with the system, both as an administrator and a user. We implemented our microservices using Spring-related technologies and we run them on a cluster of Tomcat 7.0 application servers; the communications were implemented using RabbitMQ 3.7. Note that we use a microservice approach to implement our components. They are fine-grained and they are distributed across a network, which makes message queuing the ideal approach to communicate them. This paradigm revolves around the following concepts: producers, consumers, messages, and queues. A producer is a microservice that produces a message and sends it to a queue, a consumer is a microservice that receives messages from a queue, a message encapsulates the data exchanged between a producer and a consumer, and a queue is an intermediate cache that helps producers and consumers work as asynchronously as possible. This paradigm has a number of advantages, including that it facilitates decoupling microservices, it ensures the reliability of messages by means of acknowledgements and timeouts, and it FIGURE 1 Domain model: actors and filters [Colour figure can be viewed at wileyonlinelibrary.com] allows systems to easily scale out. In the sequel, we refer to our microservices as simply services because there is no room for confusion. 3.1 The Data Storage component The Data Storage component is responsible for persisting and querying the data with which our system works. The domain model provides the classes and relations between them. We use two stereotypes in this model, namely, «persistent»and«transient». The difference is that the «persistent» stereotype implies that the objects of the corresponding class are persisted in our database, whereas the objects whose class has the «transient» stereotype are not; that is, they are computed and consumed on the fly. Figure 1 describes the domain model regarding the actors and the internationalized filters. There are two kinds of actors: administrators and users. Administrators are responsible for managing the configuration of the system. Users can visualize the results. There are two kinds of internationalized filters: Language and Category. Figure 2 describes the domain model regarding the reviews. A review is a piece of text that provides an assessment regarding an aspect of an item. Reviews contain one or more sentences. A sentence is a piece of text that is typically delimited by punctuation symbols and expresses a complete though. A sentence contains one or more tokens. A token is the minimum linguistic unit with a syntactic function. A sentiment token is a specialization of a token that is necessary to compute sentiment. A tagged token is a token that contains some linguistic tags. A sentence may contain one or more conditions. A condition is a clause that conveys a circumstance, a state, or a situation that must be true so that the whole sentence holds. A sentence may also contain one or more aspect samples. An aspect is a feature of an item, and an aspect sample is an instantiation of an aspect in a sentence. Figure 3 describes our term model. A term is an annotation that we use to characterize the tokens of which a sentence is composed. Every term has, at least, a lexical form. A lexical form has a stem that results from lowercasing a word and removing gender, number, and/or case prefixes and/or suffixes in the case of nouns and adjectives or gender, number, and tense forms in the case of verbs. Terms can be either aspects, sentiment words, negations, or context boosters. An aspect is a term that represents a feature of an item. For each aspect, there is a confidence score in range [0.00,+1.00]that determines how likely it is an actual aspect. Each aspect may be assessed by one or more sentiment words. A sentiment word is a term that represents an assessment that is made explicit by means of a sentiment weight in range −1.00 (very negative) up to +1.00 (very positive). A negation is a term that changes the sign of the sentiment weight regarding an aspect. A context booster is a term that boosts the sentiment weight regarding an aspect. A gazetteer represents a collection of samples of a specific kind of term. There are four gazetteers, namely, AspectGazetteer, which represents a collection of aspects; SentimentWordGazetteer, which represents a collection of SentimentWord;NegationGazetteer, which represents a collection of negations; and ContextBoosterGazetteer, which represents a collection of context boosters. Note that these gazetteers are readily available for most common languages, except for the gazetteer of aspects. To build the gazetteer of aspects, we start with an initial set of aspects that can be grown by means of an aspect identification process that we describe later. Figure 4 represents the booster model. Generally speaking, a booster is a word that increases the sentiment weight regarding an aspect. They can be classified as context boosters, which are terms, and inner boosters, which are writing styles. FIGURE 2 Domain model: reviews [Colour figure can be viewed at wileyonlinelibrary.com] FIGURE 3 Domain model: terms and gazetteers [Colour figure can be viewed at wileyonlinelibrary.com] FIGURE 4 Domain model: boosters [Colour figure can be viewed at wileyonlinelibrary.com] Our model was optimized in a number of ways. The first optimization is regarding the orientation of the associations; note that they are oriented from the many endpoint to the one endpoint since this can be implemented very efficiently using foreign keys. We have also added attribute lastUpdate to class Sentence and interface Term, which helps determine if a sentence or a term needs to be reanalyzed in cases in which a gazetteer is updated. An additional optimization consists in representing the tokens, the conditions, and the aspects using attributes beginIndex and endIndex, which provide their offsets in the sentences in which they are contained; this prevents us from creating many small duplicated strings. Note also that classes SentimentWord,Booster,andNegation are transient, that is, their objects need not be persisted because they can be computed on the fly very easily and they are only used in the process that computes sentiment scores, at the sentence or the aspect level, or the sentiment weight, the negation weight, and the booster weight of a token; we provide additional details on that process in the following section. Note also that the gazetteer classes are transient since they are used as entry points to the corresponding collections of terms only at run time; so, there is no need to persist them explicitly. Last but not the least, we created an ElasticSearch index to improve performance when retrieving objects of class Sentence. This component has three services: DataPersister, which is responsible for persisting the data from the services of the DataProcessors component; DataRetriever, which is responsible for retrieving the data of the services of the DataProcessors component; and ClientApplicationRepository, which is responsible for managing the queries issued by the ClientApplication component. Note that these services provide CRUD methods, that is, the basic methods to create, retrieve, update, and delete the entities of our domain model. In addition to them, the ClientApplicationRepository also provides some specific methods, namely: Select aspects of a given language and category; select the number of positive, negative, and neutral sentences of a given language, category, aspect, and keyword, if any; select aspects and their average sentiment of aspect samples of sentences in a given language, category, aspect, and keyword, if any; select sentences of a given language, category, aspect, and keyword, if any, and their sentiment; and select aspect samples and conditions of the sentences that result from the previous query. This component was implemented using the following technologies: (1) Spring Data to implement the repositories that handle writing and retrieving data; (2) MySQL 5.5 to persist our data; and (3) ElasticSearch to implement full-text indices. 3.2 The Data Processors component The Data Processors component is responsible for loading reviews, preprocessing them, and running the processes to mine conditions, discover new aspects, and summarize the sentiment. Figures 5 and 6 present the services; their relevant attributes, if any; their public methods, which can be called by an external service synchronously; their listener methods, which are subscribed to queues; and their private methods, which are used to execute their corresponding algorithms. Figure 7 shows the communications involved in loading new reviews into the DataStorage component. We assume that there is an external service that helps search for a collection of reviews on the Web and download them to a CSV file with the following columns: a unique identifier, the text of the review, its publication date, the language in which it was written, and its category. The CSV file is loaded into our system (1.1) by means of the ReviewLoader service, which forwards them to the Preprocessor service (1.2), which is responsible for splitting the input reviews into sentences, the sentences into tokens, and stemming them. The results are persisted to the Data Storage component (2.1), and they FIGURE 5 Services: loading reviews [Colour figure can be viewed at wileyonlinelibrary.com] FIGURE 6 Services: discovering aspects and summarizing sentiment [Colour figure can be viewed at wileyonlinelibrary.com] are also forwarded to the ConditionMiner service (2.2), which is responsible for mining conditions and persisting them to the Data Storage component (3). Note that the previous services run in a pipeline every time that an administrator loads a new collection of reviews into the system. There are two additional services that are run on request: the AspectDiscoverer service, which helps discover new aspects, and SentimentSummariser, which summarizes the sentiment regarding them. Example 1. Figure 8 shows an example of review, which we assume was retrieved from a website and stored in a CSV file. After reading from it, the ReviewLoader produces the Review object (1).Then,thePreprocessor service splits the review text into sentences (2). Finally, a sequence of tokens is produced for each sentence. Figure 9 shows the tokens of the first sentence. FIGURE 7 Communication diagram: loading reviews [Colour figure can be viewed at wileyonlinelibrary.com] :Review (1.1) :Review (1.2) :Sentence :Token (2.1) :Sentence :Token (2.2) :Condition (3) FIGURE 8 Review loading and preprocessing: example (1/2) [Colour figure can be viewed at wileyonlinelibrary.com] Example 2. Figure 10 shows the sequence of SentimentTokens of sentence “The argument isn't harsh, but the main character is too HORRIBLE.” Our method focuses on the two sentiment words: “harsh”, which changes its negation weight due to the nearby “n't”, and “HORRIBLE”, which changes its booster weight due to the context booster “too” and the inner booster that consists in writing the whole word with capital letters. Then, the sentence is split into two sequences of tokens that correspond to aspects “argument” and “main character”; token “but” does not belong to any sequences since it is just a nexus between them. The sentiment for the first aspect is 0.60 and −1.00 for the second aspect. The sentence sentiment is the sum of those scores, that is, 0.60 +(−1.00)=−0.40. FIGURE 9 Review loading and preprocessing: example (2/2) FIGURE 10 Sentiment analysis: example The ConditionMiner service uses the following approach: Given an input sentence, it first transforms it into vectors in which each element corresponds to the stem of a token; this vector is then passed onto a deep neural network that returns vectors whose elements are IOB tags, namely: tag Iindicates that a token is inside a condition, tag Oindicates that it is outside a condition, and tag Bindicates that it is the beginning of a condition. It is straightforward to map the resulting vectors onto the input sentences so as to create our Condition objects. We provide additional details on this service in the following section. Figure 11 shows the communications involved in discovering new aspects and summarizing sentiment. There is a service called AspectDiscovererLauncher that initiates the aspect discovery process. It first uses the DataRetriever service to retrieve the sentences to be analyzed (1.1). It then informs the AspectFilter service of the number of sentences (1.2) and starts passing them on to the AspectDiscoverer service (1.3), which discovers as many new aspects as possible. Every time the AspectDiscoverer service has discovered the aspects in a sentence, it passes them on to the AspectFilter service (1.4), which retains the most representative ones and stores them in the database by means of the DataPersister service (1.5). The aspect discovery process uses the following approach22: Given an input sentence, it iterates over the tokens of the sentence, creates the corresponding tagged tokens, and initializes their part-of-speech tags and their dependency tags. Then, it selects the nouns and noun phrases that are nonstop words and occur in, at least, 1% of the sentences; this results in a set of candidate aspects for which we compute a confidence score that normalizes the aspect frequency as follows: ck=𝑓k−min{𝑓i,…,𝑓 n} max{𝑓i,…,𝑓 n}−min{𝑓i,…,𝑓 n},(1) where ckrepresents the confidence score of aspect k,fkrepresents its frequency in the sentences that have been analyzed, min{𝑓1,…,𝑓 n}represents the minimum frequency of all aspects, and max{𝑓1,…,𝑓 n}represents the maximum frequency of all aspects. The previous computation results in a score in range [0.00,+1.00],sothattheclosertozerothe alternatives to implement the decoder (GRU or BiGRU) using both Softmax (𝜙=soft) and Sigmoid (𝜙=sig) as activation functions. For the sake of readability, we refer to them using the names of the alternatives and the activation function as subscripts: GRU-GRU𝜙, GRU-BiGRU𝜙, BiGRU-GRU𝜙, and BiGRU-BiGRU𝜙. We evaluated the baselines and our alternatives on our data set using four-fold cross-validation. We measured the standard performance measures: precision, recall, and the F1score. Figure 19 presents the results of our experiments. The conclusion is that the baselines can attain relatively good precision. Mausam et al's16 proposal attains a recall that is similar to its precision, but Chikersal et al's17 proposal falls short regarding recall. None of our proposals beat the baselines regarding precision, but most of them beat them regarding recall because they learn complex patterns that are very difficult to spot for a person. Note that the improvement regarding recall is enough for the F1score to improve the baselines. 6CONCLUSIONS In this article, we have first motivated the need to mine conditions in the context of aspect-based sentiment analysis. We have analyzed current commercial and research text mining systems, and we have found that they neglect conditions; unfortunately, existing natural language processing frameworks and toolkits do not provide any components to mine them; neither do the results in the literature provide a conclusive solution to the problem. This motivated us to work on Torii, which is an aspect-based sentiment analysis system that can mine conditions. We have presented its microservice-based architecture and the details behind our proposal to mine conditions; we have also performed an experimental analysis on a large multicategory, multilingual data set that proves that our proposal outperforms other proposals in the literature and achieves good enough precision and recall for practical purposes. ACKNOWLEDGEMENTS Our work was supported by the Spanish R & D programme (grants TIN2013-40848-R and TIN2016-75394-R) and Opileak.†† The computing facilities used to run our experiments were provided by The Andalusian Scientific Computing Centre (CICA). Special thanks also go to Opileak, since they also helped us validate our proposal in two real-world case studies. Due to nondisclosure agreements, we can only provide an overall picture of them. The first case study was in the context of the engineering department of a power utility company. Their projects revolve around green energy, power infrastructures, consulting, technology, and chemicals. One of their projects consists in analyzing power distribution data that comes from their distribution centers and target consumers. Frequently, they have to design and implement new dashboards to compare those data to a variety of factors, including the power intake of smart cities or the evolution of registered customers. Social media data caught their eye. They started to pay attention to new measures like the number of tweets regarding their company or how many likes their last Facebook publication got. They hired Opileak's opinion mining services and integrated them into their dashboards. The integration was performed by means of the REST API that Opileak provides to retrieve the data. They were interested in exploring the opinion of their customers regarding topics like smart cities, public lights, or the company itself in social media like Twitter or Facebook. This customer used the data to display statistics about the topics in which they were interested. Our proposal helps improve the results by means of a finer-grained aspect-level analysis that takes conditions into account. The second case study was in the context of the town hall of a well-known city in Spain that is visited by millions of tourists every year. The town hall has a tourism division that manages guided tours, attractions, monuments, and tourist information offices. They had several sites related to tourism, and they wanted to integrate them all, improve the front-end design, improve the availability of their services, and implement new business rules. Furthermore, they wished to integrate a friendly assistant to help people find their tourism services by means of interactions in natural language. The town hall is also interested in learning about the opinion that people cast in social media about their services. The project is currently ongoing. The town hall will benefit from our results thanks to the finer-grained sentiment summaries that our aspect-based approach provides and the fact that conditions are taken into account. However, it is also expected that our deep neural approach to mine conditions be integrated in a friendly virtual assistant so that it can recognize conditional sentences properly. ††Opileak is available at https://www.opileak.es ORCID Fernando O. Gallego https://orcid.org/0000-0002-9002-0157 Rafael Corchuelo https://orcid.org/0000-0003-1563-6979 REFERENCES 1. Flory L, Osei-Bryson K-M, Thomas MA. A new web personalization decision-support artifact for utility-sensitive customer review analysis. Decis Support Syst. 2017;94:85-96. 2. Do HH, Prasad PW, Maag A, Alsadoon A. Deep learning for aspect-based sentiment analysis: a comparative review. Expert Syst Appl. 2019;118:272-299. https://doi.org/10.1016/j.eswa.2018.10.003 3. Etzioni O, Fader A, Christensen J, Soderland S, Mausam. Open information extraction: the second generation. Paper presented at: Twenty-Second International Joint Conference on Artificial Intelligence (IJCAI); 2011; Barcelona, Spain. 4. Mitchell TM, Cohen WW, Hruschka ER, et al. Never-ending learning. In: Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI); 2015; Austin, TX. 5. Pablos AG, Cuadros M, Rigau G. W2VLDA: almost unsupervised system for aspect-based sentiment analysis. Expert Syst Appl. 2018;91:127-137. 6. Yoo S, Song J, Jeong O. Social media contents based sentiment analysis and prediction system. Expert Syst Appl. 2018;105:102-111. 7. Costa-Montenegro E, Tsybanev A, Cerezo-Costas H, et al. In-memory distributed software solution to improve the performance of recommender systems. Softw: Pract Exper. 2017;47(6):867-889. 8. Nilashi M, Ibrahim O, Bagherifard K. A recommender system based on collaborative filtering using ontology and dimensionality reduction techniques. Expert Syst Appl. 2018;92:507-520. 9. Ferrucci D, Lally A, Verspoor K, Nyberg E. Unstructured information management architecture (UIMA). https://docs.oasis-open.org/ uima/v1.0/uima-v1.0.html. 2009. 10. Cunningham H, Tablan V, Roberts A, Bontcheva K. Getting more out of biomedical documents with GATE's full lifecycle open source text analytics. PLoS Comput Biol. 2013;9(2): e1002854. 11. Manning CD, Surdeanu M, Bauer J, Finkel JR, Bethard S, McClosky D. The Stanford CoreNLP natural language processing toolkit. In: Proceedings of 52nd Annual Meeting of the Association for Computational Linguistics: System Demonstrations; 2014; Baltimore, MD. 12. Padró L, Stanilovsky E. Freeling 3.0: towards wider multilinguality. Paper presented at: International Conference on Language Resources and Evaluation (LREC); 2012; Istanbul, Turkey. 13. Bird S. NLTK: the natural language toolkit. In: Proceedings of the COLING/ACL on Interactive Presentation Sessions (ACL); 2006; Sydney, Australia. 14. The Apache Software Foundation. Apache OpenNLP. https://opennlp.apache.org. 2018. 15. Explosion.ai. spaCy: Industrial-strength natural language processing. https://spacy.io. 2018. 16. Mausam Schmitz M, Bart R, Soderland S, Etzioni O. Open language learning for information extraction. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL); 2012; Jeju Island, South Korea. 17. Chikersal P, Poria S, Cambria E, Gelbukh AF, Siong CE. Modelling public sentiment in Twitter: using linguistic patterns to enhance supervised learning . Paper presented at: International Conference on Intelligent Text Processing and Computational Linguistics (CICLing); 2015 Cairo, Egypt. 18. Nakayama Y, Fujii A. Extracting condition-opinion relations toward fine-grained opinion mining. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing (EMNLP); 2015 Lisbon, Portugal. 19. Narayanan R, Liu B, Choudhary AN. Sentiment analysis of conditional sentences. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing (EMNLP); 2009; Singapore. 20. Skeppstedt M, Schamp-Bjerede T, Sahlgren M, Paradis C, Kerren A. Detecting speculations, contrasts and conditionals in consumer reviews. In: Paper presented at: Proceedings of the 6th Workshop on Computational Approaches to Subjectivity, Sentiment and Social Media Analysis (WASSA @EMNLP); 2015; Lisbon, Portugal. 21. Mausam. Open information extraction systems and downstream applications. In: Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence (IJCAI); 2016New York, NY. 22. Hu M, Liu B. Mining opinion features in customer reviews. In: Proceedings of the 19th national conference on Artificial intelligence; 2004; San Jose, CA. 23. Taboada M, Brooke J, Tofiloski M, Voll KD, Stede M. Lexicon-based methods for sentiment analysis. Computational Linguistics. 2011;37(2):267-307. 24. Sutskever I, Vinyals O, Le QV. Sequence to sequence learning with neural networks. Advances in NIPS. 2014:3104-3112. 25. Srivastava N, Hinton GE, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from over-fitting. J Mach Learn Res. 2014;15(1):1929-1958. 26. Caruana R, Lawrence S. Overfitting in neural nets: backpropagation, conjugate gradient, and early stopping. In: Proceedings of the 13th International Conference on Neural Information Processing Systems (NIPS); 2000; Denver, CO. 27. Kingma DP, Ba J. ADAM: a method for stochastic optimization. arXiv preprint arXiv:1412.6980. 2014. 28. Bengio Y, Simard PY, Frasconi P. Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Networks. 1994;5(2):157-166. 29. Pascanu R, Mikolov T, Bengio Y. On the difficulty of training recurrent neural networks. Paper presented at: International Conference on Machine Learning (ICML); 2013; Atlanta, GA. 30. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Computation. 1997;9(8):1735-1780. 31. Han H-G, Zhang S, Qiao J-F. An adaptive growing and pruning algorithm for designing recurrent neural network. Neurocomputing. 2017;242:51-62. How to cite this article: Gallego FO, Corchuelo R. Torii: an aspect-based sentiment analysis system that can mine conditions. Softw: Pract Exper. 2020;50:47–64. https://doi.org/10.1002/spe.2762