Real-time focused extraction of social media users

Received March 30, 2022, accepted April 15, 2022, date of publication April 20, 2022, date of current version April 27, 2022. Digital Object Identifier 10.1109/ACCESS.2022.3168977 Real-Time Focused Extraction of Social Media Users RODRIGO MARTÍNEZ-CASTAÑO, DAVID E. LOSADA , AND JUAN C. PICHEL CiTIUS, Universidade de Santiago de Compostela, 15782 Santiago de Compostela, Spain Corresponding author: Juan C. Pichel ([email protected]) This work was supported in part by the Ministerio de Ciencia e Innovación (MICINN) under Grant RTI2018-093336-B-C21 and Grant PLEC2021-007662; in part by Xunta de Galicia under Grant ED431G/08, Grant ED431G-2019/04, Grant ED431C 2018/19, and Grant ED431F 2020/08; and in part by the European Regional Development Fund (ERDF). ABSTRACT In this paper, we explore a real-time automation challenge: the problem of focused extraction of Social Media users. This challenge can be seen as a special form of focused crawling where the main target is to detect users with certain patterns. Given a specific user profile, the task consists of rapidly ingesting Social Media data and early detecting target users. This is a real-time intelligent automation task that has numerous applications in domains such as safety, health or marketing. The volume and dynamics of Social Media contents demand efficient real-time solutions able to predict which users are worth to explore. To meet this aim, we propose and evaluate several methods that effectively allow us to harvest relevant users. Even with little contextual information (e.g., a single user submission), our methods quickly focus on the most promising users. We also developed a distributed microservice architecture that supports real-time parallel extraction of Social Media users. This modular architecture scales up in clusters of computers and it can be easily adapted for user extraction in multiple domains and Social Media sources. Our experiments suggest that some of the proposed prioritisation methods, which work with minimal user context, are effective at rapidly focusing on the most relevant users. These methods perform satisfactorily with huge volumes of users and interactions and lead to harvest ratios 2 to 9 times higher than those achieved by random prioritisation. INDEX TERMS Big data, distributed systems, focused user extraction, supervised learning, information retrieval, real-time processing, social media. I. INTRODUCTION Focused Crawling techniques are oriented to extracting web pages that satisfy a given property or topic of interest. This is typically supported by supervised learning technology (e.g., text classifiers) that help to decide what links are worth to explore. Nowadays, it is feasible to run crawling tools that efficiently extract on topic contents from massive repositories such as the web or certain Social Media (SM) sources. Focused Crawling was first introduced in [1] and coined in [2]. It has been intensely discussed in the literature [3], [4] and it plays a fundamental role, for example, in building vertical search engines [5]. Nevertheless, to the best of our knowledge, the real-time extraction of SM users that are relevant to a target topic has received little attention. In this paper, we make a first attempt to fill this gap. Many studies in the literature have proposed different alternatives to crawl The associate editor coordinating the review of this manuscript and approving it for publication was Massimo Cafaro . or extract SM contents. However, the notion of user is often ignored and most existing methods do not try to anticipate which users are worth to explore. We claim that we need real-time technology that rapidly ingests SM contents and, given little context (e.g., the last user’s SM submission), moves quickly towards relevant users. We envisage intelligent forms of user crawling that aggregate the history of the target users with little delay and do not overload the system with users that are likely non-relevant. As reported in our Related Work section, some user-oriented SM methods exist but they are oriented to extract samples of users (rather than massively analysing the entire SM site in real-time) or do not implement any form of user prioritisation. Given a certain SM platform, we formally define the task of Focused User Extraction (FUE) and investigate the efficiency and effectiveness issues involved in extracting target users. FUE has important applications in numerous domains. The rapid growth of SM platforms and their content production dynamics demand Big Data solutions able to (1) efficiently VOLUME 10, 2022 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 42607 R. Martínez-Castaño et al.: Real-Time Focused Extraction of Social Media Users ingest contents in real-time, (2) quickly react to changes, and (3) intelligently anticipate which users are worth to explore. This task requires methods able to perform user extraction from large repositories of SM data. By smartly prioritising SM users who are of potential interest, we can early identify target individuals and, thus, support a number of risk assessment tasks such as grooming detection, criminal recruitment or early identification of psychological problems. This requires not only large-scale technologies but also smart predictive components that continuously rank users based on real-time evidence (e.g., last SM post). In popular SM sources, we constantly obtain new pieces of evidence. Each new SM post is typically short and gives only a partial description and context of the author of the post. The user crawling process needs to evaluate this real-time evidence and, for example, decide whether or not to crawl the entire user history. This prediction task resembles when focused web crawlers see a link to a new page and have to decide whether or not to download the linked page. However, in FUE, the full exploration of a given user often requires tens or hundreds of requests to the SM servers and, thus, effective prioritisation of users becomes crucial. Taking only into account the partial user context allows us to optimise the existing resources and extract the maximum number of relevant users per unit of time. In this paper, we propose several user prioritisation methods that guide this exploration process and we study which of these methods lead to high-speed FUE. In order to extract and process huge amounts of data in real time, it is necessary to design and implement an adequate architecture that can scale up horizontally in a cluster of computers. In this regard, we contribute here by developing an adaptable distributed crawling architecture based on Catenae1[10]–[12], Kafka [13] and Docker [14]. The architecture was developed to support FUE but it can be easily adapted to other scenarios such as real-time query-based filtering, topic extraction or summarisation. Summing up, the paper describes the practical application of user prioritisation methods in real-time intelligent extraction of SM users. The main contributions of this paper are: •A formal definition of the Focused User Extraction (FUE) task and a systematic analysis of the main effectiveness and efficiency issues involved. •Several heuristic-based methods that consistently increase the harvest ratio of target users and a comprehensive study of their relative importance under real-time SM experiments. To the best of our knowledge, this is the first study that analyses the ability of a set of user-related variables in user crawling prioritisation. •A modular SM crawler that performs real-time extraction and exploration of SM communities and user profiles. An experimental evaluation performed on Reddit demonstrates the real-time processing capabilities of the crawler. The competing methods are benchmarked 1https://github.com/catenae against each other under real-time experiments where the methods run in parallel. In this way, the relative benefits of each prioritisation method are not affected by seasonal factors. •A Docker-based distributed architecture that can be easily deployed and scales by increasing the number of instances of each module in a cluster of computers (publicly available2under a Free Software license). The paper is structured as follows. Section II describes the methodology followed by our crawler to extract SM contents and rank users following different strategies. Section III discusses the architecture of the system, its modules, connections and the scalability of the proposed solution. Sections IV and Villustrate how different user extraction methods run and evolve under real-time experiments. In Section VI the adaptability of our approach to other Social Media is discussed. Section VII discusses related work and, finally, Section VII contains the main conclusions of this study. II. METHODOLOGY A. FOCUSED USER EXTRACTION Given a profile of interest that defines target users (target user profile), the relevance of each newly detected user has to be estimated. If possible, this needs to be done using minimal contextual information. For example, crawling the entire history of user posts, even if possible, introduces a significant overhead on the user crawling process. And many of those users might be non-relevant. An effective user crawler must process light volumes of user data (e.g., last post) and only make a full extraction of user’s data for those users that seem to be on-target. This represents a two-stage process in which SM web pages are constantly explored to extract candidate usernames (e.g., from web pages with the newest threads of user posts) and the user crawler process predicts which candidate users are worth to explore (i.e., which users are fully explored). The target user profile can be represented with a driving query or with a set of examples (users who fulfil the target profile and which can be used to build a user classifier). Given a Social Media platform, the aim of the task is to extract as many target users per unit of time as possible. Effective algorithms that solve this problem should somehow prioritise those users that are presumably on target and optimise the use of available resources. There are two main useful metrics to evaluate the performance of this task. First, the harvest ratio (HR), or precision, as the number of relevant users extracted divided by the total number of extracted users. A user is deemed as relevant if it is cataloged as so with high confidence by a reference classifier. This reference classifier represents the target profile of users. This means that the entire user profile –concatenation of all their posts–, when passed to the classifier, exceeds a certain threshold of the classifier (prediction probability higher than 0.9 in the case of the logistic regression classifier). A second 2https://github.com/blind-sniper 42608 VOLUME 10, 2022 R. Martínez-Castaño et al.: Real-Time Focused Extraction of Social Media Users performance metric is the extraction speed expressed as the number of extracted users per unit of time. There is a huge number of active users on Social Media. In realistic scenarios, where hardware is limited, extracting and processing everything would be unfeasible due to the massive volume of contents produced in real-time. Furthermore, there are usually some access restrictions imposed by the source (e.g., robots.txt constraints). As a consequence, lightweight exploration needs to be performed in order to extract SM users. A crucial issue is how to estimate user’s relevance from the little context available. Note also that, even if we have extensive hardware resources and computational power at our disposal, a random or arbitrary exploration of SM users is not pertinent. Such näive extraction of users slows down the identification of relevant users and, thus, cannot support the early identification of target users. Typically, when exploring a Social Media site looking for new users, candidate usernames can be found on pages that list users’ descriptions or on pages that contain a thread of user’s posts (e.g., the entry page of a given user community or forum). The available posts often include a title or description, possibly a snippet, information about the author and other relevant social properties (such as the number of replies, votes, etc.). These types of listings serve as seed pages for extracting candidate users. Among the available evidence that can be used to guide the crawler, we have elements such as the title or the text of the last user’s post, the date and time of publication, the number of comments, the score or the number of times a post has been liked, the number of times the post has been shared or re-posted, etc. In this paper, we study which of the available pieces of evidence are predictive of relevance in order to guide the crawling process. B. FOCUSED USER EXTRACTION ON REDDIT Reddit is a Web platform where users submit content (posts) such as text, images or links and other users can comment and vote for or against them. Comments, at the same time, can also be commented and voted. The platform is subdivided into communities (subreddits) focused on specific topics. It is currently ranked in Alexa [15] as the 7th website with more traffic in the United States and 19th globally. The number of average daily active users is higher than 52 million, and there are more than 100,000 active communities [16]. We have chosen Reddit as our reference platform for experimenting with FUE algorithms. This decision was based on several criteria. First, it is one of the three largest Social Media platforms with increasing popularity. Second, the diversity of users and communities make Reddit a perfect place for user mining. Third, its terms of service state their open philosophy and willingness to support external applications or services connected to Reddit. This allows us to easily conduct crawling research on this platform. Furthermore, the lessons learned from our study can be potentially transferred to other social networks. The crawling process is composed of two main steps. First, the crawler has to choose the next subreddit from which to explore the latest posts and comments. This subreddit exploration step constantly detects new usernames that are candidates to be fully explored. The second step consists of ranking the candidate users based on the available evidence. To meet this aim, a user ranking is built (using a number of user prioritisation methods) and the top user is selected and fully explored (all his/her submissions are collected). The first stage (subreddit exploration) works with a subreddit frontier that stores the subreddit names that have been found so far. Initially, the main frontpage of Reddit is used as a seed to extract some subreddit names that are stored in the frontier. This frontier grows during the crawling process (any web page retrieved from Reddit potentially contains references to unseen subreddits) and the selection of the next subreddit is done randomly from the subreddits available in the frontier. The second stage consists of choosing the next user to be analysed in depth. When exploring subreddits, new users are discovered and ranked according to different methods (see below). After a user is fully explored (all his/her submissions are retrieved), he/she is marked as processed on the list of known users and, thus, he/she will not be processed again. Algorithm 1describes our focused user crawler in pseudocode. There are four main procedures. First, during the Initialisation, the Reddit’s front page is downloaded. This page is a list of the most recent posts on the platform from any subreddit. The names of the subreddits are collected and used as seeds. In addition, the features of the posts (for example, title or number of comments) are extracted. These features are used to update the ranking of candidate users (according to different methods, as we will explain below). After obtaining the seeds, three tasks are launched in parallel: •Community Exploration. A subreddit is picked at random from the list of known communities. The front page of the selected subreddit is visited, extracting the latest posts and their features. The user ranking is updated by aggregating the new features with the existing ones. •User Extraction. The top user from the user ranking is selected. All the available submissions and comments posted by this user are extracted from the SM platform. Note that this requires several calls to Reddit and, thus, the focused user crawler aims at extracting only those users who are likely relevant. •Evaluation. All the extracted users are queued to evaluate their relevance. To meet this aim, we employ a user classifier. This classification tool analyses the full user information and determines whether or not the user is on-topic. In this way, this evaluation component assesses whether or not the extraction of the entire user history was worthwhile. This evaluation strategy follows [2], [17], where the relevance of the extracted pages is evaluated with a classifier. The main reason to estimate relevance in this way is that there are practical impediments to perform a manual analysis of thousands of texts from hundreds of thousands of users. VOLUME 10, 2022 42609 R. Martínez-Castaño et al.: Real-Time Focused Extraction of Social Media Users Algorithm 1 Pseudo-Code of the Focused User Crawler for Reddit 1: procedure Initialisation 2: if length of subreddit_list == 0then 3: html_code =load_url(00https://reddit.com/r/all/new/00) 4: subreddits =extract_subreddits(html_code) 5: for each subreddit in subreddits do 6: if subreddit not in subreddit_frontier then 7: subreddit_frontier.add(subreddit) 8: posts_features =extract_posts_features(html_code) 9: for each post_features in posts_features do 10: users_ranking.update_user_features(post_features) 11: do in parallel 12: Community Exploration() 13: User Extraction() 14: Evaluation() 15: 16: procedure Community Exploration 17: while true do 18: selected_subreddit =pick_at_random(subreddit_frontier) 19: html_code =load_url(00https://reddit.com/r/00 +selected_subreddit +00/new/00) 20: posts_features =extract_posts_features(html_code) 21: for each post_feature in posts_features do 22: users_ranking.update_user_features(post_features) 23: 24: procedure User Extraction 25: while true do 26: selected_user =users_ranking.get_first_user() 27: html_code =load_url(00https://reddit.com/user/00 +selected_user) 28: user_posts =extract_posts(html_code) 29: for each user_post in user_posts do 30: subreddit_frontier.add(user_post[00subreddit00]) 31: users_posts_to_evaluate.add(user_posts) 32: 33: procedure Evaluation 34: while true do 35: user_posts =users_posts_to_evaluate.get_first() 36: probability =classification_model.predict(user_posts) 37: if probability >THRESHOLD then 38: positive_users.add(user_posts) 39: else 40: negative_users.add(user_posts) We now make a brief discussion on the computational complexity involved in the procedures described above: •Community Exploration: The community exploration procedure is quite lightweight. For each subreddit, it gets the subreddit’s page of new contents, extracts some metadata from the available posts and updates some user’s features. The time complexity of this procedure is O(s), where sis the number of communities or subreddits. With proper data storage structures, the cost of choosing a random subreddit from the subreddit frontier (line 18) and updating user’s features (lines 21-22) do not add further complexity to the process. •User Extraction: The computational load of the user extraction procedure grows linearly with the number of users and, for each user, the user’s posts available at the user’s frontpage need to be crawled and added to the corresponding data structures. This means that the time complexity of this procedure is O(u·pavg), where uis the number of users and pavg is the mean number of posts published in the user’s frontpage. Observe that, having proper data storage structures, the costs of extracting the next user from the available list of users (line 26), incorporating the post’s subreddit to the subreddit frontier (line 30), and incorporating the user’s posts to the 42610 VOLUME 10, 2022 R. Martínez-Castaño et al.: Real-Time Focused Extraction of Social Media Users FIGURE 1. Simplified architecture diagram of the real-time focused user crawler for Reddit. evaluation’s data structure (line 31) can be considered negligible. •Evaluation: This procedure essentially consists of passing users through the classification model. The construction of the classifier from (external) training data is not part of the crawling process (the crawling process simply reads the trained classifier and the text vectorizer from disk). The text vectorizer processes the user’s posts and produces a numerical representation that is then passed to the classifier. We work with a lightweight classifier (Logistic regression, see section IV-F) whose prediction cost grows linearly with the number of features. The number of features depends on the number of unique words in the training corpus and it is often larger than the number of words in the user’s posts. As a consequence, the time complexity essentially depends on the number of users and the number of classification features: O(u· ft), where uis the number of users and ft is the number of classification features, which is bounded the size of the vocabulary of the training collection. Again, with proper storage structures, the cost of extracting the first user (line 35) and incorporating the user to the positive or negative lists (lines 38-40) is insignificant. In terms of space complexity, the size of subreddit frontier is bounded by the number of available subreddits (less than 200k) and, in any case, we only need to store a string for each subreddit (the name). The users’ features and posts demand more space but the crawling process, if needed, can remove the features and posts of the users who have been already passed to the classifier. Note also that community exploration, user extraction and evaluation run in parallel (lines 11-14) and, furthermore, the operation of each of these three procedures can be easily parallelized. As argued in the next section, depending on the load of each procedure, we can add more computational resources (e.g., to do parallel extraction of multiple users). III. REAL-TIME CRAWLING ARCHITECTURE The focused user crawler is composed of multiple microservices, designed to facilitate the scalability of the system when running on a cluster of computers. Modules are interconnected to form an execution graph built on Catenae [10]–[12], a Python library for easy development of scalable stream execution graphs. Catenae uses the Kafka message broker to interconnect different microservices. Catenae-based systems can scale up horizontally by increasing the number of instances of any microservice without further configuration. Our architecture permits the deployment of this focused crawler in infrastructures where dynamic resources can be configured, such as computing clouds (e.g., Amazon Web Services, Microsoft Azure, Google Cloud Platform, Alibaba Cloud). In addition, when few hardware resources are available, the crawler can scale up or down without interrupting its execution. Due to this distributed architecture, the focused user crawler can be simply scaled by augmenting the number of instances (e.g., if there is a bottleneck in one of the modules). The main modules and connections of the system are shown in Figure 1. The modules of the focused user crawler for Reddit are explained below: •Subreddit Explorer. At this stage, subreddits are requested to the Subreddit Chooser through RPC (Remote Procedure Call). When a subreddit is assigned, this module explores it by extracting the last posts and VOLUME 10, 2022 42611 R. Martínez-Castaño et al.: Real-Time Focused Extraction of Social Media Users comments. For each submission, it extracts the available features and sends them to a User Updater instance (for example, if the crawler prioritises users based on the number of votes of the user submissions, then any new update on the user’s votes is handled by the User Updater). When initialising the system, the Subreddit Explorer module extracts the subreddits of the texts available under /all, a pseudo-community that holds together the posts from all communities. Note that, at the beginning of the process, there is no knowledge on which subreddits are more promising (potentially contain more on-topic users) and, thus, we need to start from a general page of the platform. •Subreddit Chooser. This module is responsible for selecting a new subreddit to be sent to the subreddit explorers. In this work, we experiment with a random selection of subreddits but the platform can easily implement more sophisticated selection methods. •User Updater. The User Updater is responsible for updating the user features that guide the focused crawling process. For example, it averages, for every user, the number of comments in his/her posts. A full description of user features and related requirements to the User Updater module are given in Section IV. •User Extractor. The User Extractor instances request users to the User Chooser module and extract the maximum possible number of user texts.3The extracted content is sent to the User Classifier, which evaluates the performance. Additionally, the subreddits discovered during this process are sent to the Subreddit Storer. •User Chooser. This module handles the selection of new users to be processed (according to the user prioritisation process that guides the focused crawler). User extractors invoke the User Chooser module through RPC once they finish the extraction of a user. Since the new users are requested only when needed, the user selection is coherent with the updated ranking state in every moment. The User Chooser assigns a unique user for each user extractor. It also marks the selected users as processed in order to avoid repeated processing. •User Classifier. The main goal of this module is to evaluate the performance of the focused user crawler. It receives all the available texts for every extracted user, classifies the user and stores the probability provided by the user classifier. •Subreddit Storer. In this phase, the previously unseen subreddit names are stored in the database. It receives new subreddits from a Subreddit Explorer instance during the initialisation and from the User Extractor instances. •Stats Dumper. This module is connected to all the other modules and stores events with information related to the real-time execution. Every event is timestamped, 3In our experiments, we respect Reddit’s robots.txt and, thus, each request gets a maximum of two hundred texts (posts and comments). so the behaviour of the system in several dimensions can be easily analysed during or after the execution. This module is not represented in Figure 1due to its number of interconnections. •Model Trainer. This module is responsible for training the user classifier. In our initial experiments, the user classifier is built once (from external data) and remains unchanged thereafter. •Batch Updater. This module is needed for one of the advanced prioritisation methods detailed in Section V. It updates user-related data after learning new user classification models. MongoDB [18] is used as a storage system and for managing the rankings. Aerospike [19], a memory-based key-value store, is used to avoid processing repeated posts during the crawling process. IV. BASIC PRIORITISATION METHODS In our attempt to early identify target users during crawling, we have defined several methods to guide the FUE process. Standard focused crawlers employ a number of strategies when, for example, they get a new link and they need to predict whether it is worth to download the linked web page. In FUE, the available evidence is different and, thus, we need to design new strategies for prioritising the users that are found. These methods work with a number of features that are available when browsing the SM website. The methods considered are: A. RANDOM A random (RND) selection of the next user. This is a näive baseline used for comparison against more sophisticated methods. B. AVERAGE SCORE OF USER’S TEXTS In SM platforms, users’ interactions are often scored, voted or liked. The popularity of user’s posts and comments might be a valuable clue to guide FUE. For example, a focused user extraction process might be interested in early identifying those users emitting highly influential contents. In Reddit, the score of a post reflects the utility and quality of the content within its context (the subreddit). The interaction of the users determines the total score of a post (users can add, upvote, or subtract, downvote, a point to each post or comment). The score of a Reddit’s post or comment is calculated by summing the positive and negative votes. Depending on the focus of the crawler, this feature might be indicative of target users (e.g., a focused extraction of offensive users might benefit from the existence of many negative votes). During the crawling process, this average score is accumulated for each user. Note that this is computed from the user texts seen so far (web pages downloaded) and, thus, it is usually an incomplete view of the overall score for this user. Let Ube the set of candidate users and Su= {Sut1,Sut2, . . .}be the set of scores for each observed text of a 42612 VOLUME 10, 2022 R. Martínez-Castaño et al.: Real-Time Focused Extraction of Social Media Users given user u. Let Sube the average score of the known texts of a given user. We propose two different ways to determine the chosen user (cuser ). These two variants select the next user based on the highest or lowest average score, respectively. The first approach builds (and constantly updates) a ranking of users by decreasing average score and extracts the top user. This will be referred to as Highest Average Score (HAS). Similarly, the Lowest Average Score (LAS) builds a ranking of users by increasing average score. cuser =arg max u∈U Su(1) cuser =arg min u∈U Su(2) C. AVERAGE NUMBER OF COMMENTS IN USER’S TEXTS This feature also weights users based on the impact of their texts. In this case, we employ the average number of comments associated to the posts of a given user. Users with long threads of comments to their posts might be particularly relevant for a given FUE process. Let Cube the number of comments for each observed text of a given user u. Let Cu be the average number of comments of the known texts of a given user u. Again, we define two approaches that build descending and ascending rankings with the average number of comments of every user: Highest Average Comments (HAC) and Lowest Average Comments (LAC), respectively. The ranking is updated every time new data is available. When the system needs to extract a new user, the top user of the ranking is selected. cuser =arg max u∈U Cu(3) cuser =arg min u∈U Cu(4) D. TIME-BASED USER SELECTION The post time of user’s texts might also be useful for discovering target users. For example, a FUE process aiming at extracting users showing signs of psychological problems (e.g., depression) could benefit from time-related trends (e.g., users might make more submissions during the night because sleeping problems is one of the symptoms of depression). To encode this feature, we extract the hour when each observed text was posted, we average these hours for each user and we compute how close the average is to four selected times (0h, 6h, 12h and 18h). Let T= {0,6,12,18}be the set of selected hours. Let Hube the set of hours for the seen texts of a given user u. Let Hube the average hour when the seen texts of a given user were posted. We have defined four time-related prioritisation methods: Near 00:00 UTC±00:00 (N00) gives more weight to users that post near midnight, Near 06:00 UTC±00:00 (N06) gives more weight to users that post near 6 in the morning, and so on. cuser =arg min u∈U min(|t−Hu|,24 − |t−Hu|),where t ∈T (5) E. CLASSIFIER-BASED PRIORITISATION A focused crawler is often guided by a driving query or classifier. Given the text observed from each user, a natural way to guide the FUE process is either to compute the matching between the driving query and the user’s text or to pass the user’s text to the classifier that guides FUE. At any point, the user’s texts available to the crawler are a subset of the user’s history of posts but, still, this partial representation of the user might be highly valuable to early identify target users. For example, the first time that the crawler sees a user, it probably reads a single comment or post from the user. This resembles when a focused crawling of web pages sees the first reference (link) to a new web page. In such a case, a single anchor text and the associated URL are the only pieces of evidence available but the focused crawler can still estimate topicality with respect to the target profile. In FUE, we can maintain (and constantly augment) a textual representation of each user. This consists of the concatenation of the titles and comments’ bodies of the posts observed for each user. Let Dbe the set of incremental user’s representations. Given D, we can pass it to the classifier and obtain a probability estimation of the user being on-target, Pu(D). We define two approaches to select target users: Highest Classification Probability (HCP) builds a ranking of users by decreasing probability, whereas Lowest Classification Probability (LCP) builds a ranking of users by increasing probability. LCP is supposed to perform poorly since this prioritisation makes little sense. However, LCP serves as a safe check in our empirical study. cuser =arg max u∈U Pu(D) (6) cuser =arg min u∈U Pu(D) (7) Table 1summarises all prioritisation methods described above. F. EXPERIMENTAL RESULTS In order to make fair comparisons among the proposed methods, we have designed an experimental framework where all crawling variants run in parallel. In this way, we can fairly evaluate their relative merits on identifying target users and we avoid undesirable biases from seasonal effects that might affect the comparison. The server where all the experiments in the paper were conducted has the following characteristics: •CPU: 2 x Intel R Xeon R CPU E5-2630 v4 @ 2.20 GHz - 20 cores (40 threads) •Memory: 12 x Hynix 32 GiB DIMM DDR4 @ 2400 MHz (384 GiB) •Disk: Toshiba MD04ACA400 4 TB @ 7,200 RPM, 64 MB cache •Internet bandwidth: ∼400/200 Mbps (download/upload) The first experiment lasted seven days. During this period, the total number of unique extracted users was higher than 110,000 (655 per hour) for each prioritisation method VOLUME 10, 2022 42613 R. Martínez-Castaño et al.: Real-Time Focused Extraction of Social Media Users TABLE 1. Basic prioritisation methods. FIGURE 2. Number of extracted users by each focused user crawling method. (see Figure 2). Note that the extraction performance can be highly scaled, while maintaining the politeness of the crawler, by mainly launching more instances of the User Extractor module, which is the main bottleneck. The crawling variants HAC and HAS are slower at extracting users. This is due to the way in which users are prioritised. These two variants extract users that have a high number of comments or points. These users tend to be highly active on the platform and, thus, when the crawler extracts them, the retrieval of all their submissions is costly. Let us now evaluate the ability of these prioritisation strategies in identifying target users. To meet this aim, we use a user classifier built from external training data. More specifically, we built a binary classifier from a sample of Reddit users that predicts if a given user is likely to have signs of depression. We worked with a collection on depression and natural language use [20]. The classifier is a Logistic Regression model with L1 regularisation implemented in Python with scikit-learn. It was built with a training set of 486 users (83 positives, 403 negatives) where users were represented with a single document consisting of the concatenation of all their writings. In order to decide if a user is on-target, we established a threshold equal to 0.9 since previous experiments demonstrated that this setting leads to high precision [20]. Figures 3and 4plot the harvest ratio of different prioritisation strategies and, thus, it helps to compare the relative merits of the proposed metrics in early identifying the target users. To facilitate readability, the methods have been separated into two independent plots but the random strategy is shown in both graphs. The highest harvest ratio is achieved by HCP: 2.8 times better than the second best (N06) and 4.9 times better than RND during the first day. On the last day these numbers decay to 1.4 and 1.8, respectively. After the first day, four of the proposals behave better than random. This number increases to five after the third day. At this point, many strategies behave similarly since most of them have already explored a high number of users. The most promising strategies are HCP, N06, HAC and N00 since they perform considerably better than random. On the other hand, HAS, N12, N18, LAC, LAS and LCP can be discarded. Observe that HCP effectively exploits the little pieces of textual evidence available. These small extracts seem to be indicative of user relevance. Note also that none of the other good performers employ the texts written by users to prioritise them. V. ADVANCED PRIORITISATION METHODS Given the results discussed above, we designed three new FUE methods. The first is based on re-training the classifier that guides the HCP crawler (from new tagged users obtained with pseudo-training data). A second method performs a fusion of two user rankings (HCP and HAC). Finally, the third method combines HCP, N06 and HAC in a hierarchical way. This section presents these new methods and discusses the associated changes in the FUE architecture. 42614 VOLUME 10, 2022 R. Martínez-Castaño et al.: Real-Time Focused Extraction of Social Media Users FIGURE 3. Harvest ratio of some focused user crawling methods. FIGURE 4. Harvest ratio of some (discarded) focused user crawling methods. HCP With Pseudo-Training: With HCP, the crawling system makes two types of user classifications. The evaluation classifier (see Section II) is the core evaluation tool of the FUE process and assesses whether or not user extraction was effective. The prediction classifier categorises the users based on the partial information available (e.g., last post in a recently crawled page), yielding a confidence score that is used for user prioritisation. With the standard HCP method discussed above, these two classifiers are the same and their classification model is built once (from the training set) and never changes thereafter. However, the classification model of the predictive classifier could be updated as we extract and process users. More specifically, those users that are chosen, fully explored, and classified by the evaluation classifier can be incorporated as pseudo-training examples. The main idea of HCP with pseudo-training consists of updating the prediction classifier based on augmenting the training examples with new examples and their pseudo-labels. Every five minutes, a new prediction classifier is built from the original training set plus the new pseudo-examples available. We experimented with three variants. With HCP-0.9-0.1 only those users classified with a probability higher than 0.9 or lower than 0.1 are incorporated (as positive or negative, respectively).4With HCP-0.5-0.5 we employ the standard threshold of the classifier (0.5) and, thus, any user classified above or below the threshold is incorporated into the training set as positive or negative, respectively. We also 4Note that we work with Logistic Regression classifiers and, thus, we have access to probabilities associated to the estimations. VOLUME 10, 2022 42615 R. Martínez-Castaño et al.: Real-Time Focused Extraction of Social Media Users [24] R. Guo, H. Wang, M. Chen, J. Li, and H. Gao, ‘‘Parallelizing the extraction of fresh information from online social networks,’’ Future Gener. Comput. Syst., vol. 59, pp. 33–46, Jun. 2016. [25] F. Erlandsson, P. Bródka, M. Boldt, and H. Johnson, ‘‘Do we really need to catch them all? A new user-guided social media crawling method,’’ Entropy, vol. 19, no. 12, p. 686, Dec. 2017. [Online]. Available: https://www.mdpi.com/1099-4300/19/12/686 [26] P. Wang, J. Zhao, J. C. S. Lui, D. Towsley, and X. Guan, ‘‘Fast crawling methods of exploring content distributed over large graphs,’’ Knowl. Inf. Syst., vol. 59, no. 1, pp. 67–92, Apr. 2019. [27] Y. Zhou, R. Ji, J. Su, and J. Yao, ‘‘Uncovering media bias via social network learning,’’ ACM Trans. Intell. Syst. Technol., vol. 12, no. 1, pp. 1–12, Feb. 2021. [28] X. Wang, L. Tokarchuk, F. Cuadrado, and S. Poslad, ‘‘Exploiting hashtags for adaptive microblog crawling,’’ in Proc. IEEE/ACM Int. Conf. Adv. Social Netw. Anal. Mining, Aug. 2013, pp. 311–315. [29] Z. Zhang and O. Nasraoui, ‘‘Profile-based focused crawler for social media-sharing websites,’’ in Proc. 20th IEEE Int. Conf. Tools With Artif. Intell., Nov. 2008, pp. 317–324. [30] C. H. C. Leung, A. W. S. Chan, A. Milani, J. Liu, and Y. Li, ‘‘Intelligent social media indexing and sharing using an adaptive indexing search engine,’’ ACM Trans. Intell. Syst. Technol., vol. 3, no. 3, pp. 1–27, May 2012, doi: 10.1145/2168752.2168761. [31] M.-Y. Kan, ‘‘Web page classification without the web page,’’ in Proc. 13th Int. World Wide Web Conf. Alternate Track Papers Posters (WWW Alt.), 2004, pp. 262–263. [32] R. Meusel, P. Mika, and R. Blanco, ‘‘Focused crawling for structured data,’’ in Proc. 23rd ACM Int. Conf. Conf. Inf. Knowl. Manage., Nov. 2014, pp. 1039–1048. [33] F. Menczer, G. Pant, and P. Srinivasan, ‘‘Topical web crawlers: Evaluating adaptive algorithms,’’ ACM Trans. Internet Technol., vol. 4, no. 4, pp. 378–419, Nov. 2004. [34] G. Gossen, E. Demidova, and T. Risse, ‘‘ICrawl: Improving the freshness of web collections by integrating social web and focused web crawling,’’ in Proc. 15th ACM/IEEE-CS Joint Conf. Digit. Libraries, Jun. 2015, pp. 75–84. [35] T. Gisselbrecht, L. Denoyer, P. Gallinari, and S. Lamprier, ‘‘Whichstreams: A dynamic approach for focused data capture from large social media,’’ in Proc. 9th Int. AAAI Conf. Web Social Media, 2015, pp. 130–139. [36] S. Agarwal and A. Sureka, ‘‘Spider and the flies : Focused crawling on Tumblr to detect hate promoting communities,’’ 2016, arXiv:1603.09164. [37] X. Tang and C. C. Yang, ‘‘Ranking user influence in healthcare social media,’’ ACM Trans. Intell. Syst. Technol., vol. 3, no. 4, pp. 1–21, Sep. 2012, doi: 10.1145/2337542.2337558. [38] J. Li, L. Wu, R. Hong, K. Zhang, Y. Ge, and Y. Li, ‘‘A joint neural model for user behavior prediction on social networking platforms,’’ ACM Trans. Intell. Syst. Technol., vol. 11, no. 6, pp. 1–25, Dec. 2020, doi: 10. 1145/3406540. [39] B.-Y. Hsu, C.-L. Tu, M.-Y. Chang, and C.-Y. Shen, ‘‘On crawling community-aware online social network data,’’ in Proc. 30th ACM Conf. Hypertext Social Media, Sep. 2019, pp. 265–266, doi: 10.1145/ 3342220.3344937. [40] J. Zhuang, T. Mei, S. C. H. Hoi, X.-S. Hua, and Y. Zhang, ‘‘Community discovery from social media by low-rank matrix recovery,’’ ACM Trans. Intell. Syst. Technol., vol. 5, no. 4, pp. 1–19, Jan. 2015. [41] K. Lee, J. Caverlee, Z. Cheng, and D. Z. Sui, ‘‘Campaign extraction from social media,’’ ACM Trans. Intell. Syst. Technol., vol. 5, no. 1, pp. 1–28, Dec. 2013, doi: 10.1145/2542182.2542191. [42] S. C. Guntuku, D. B. Yaden, M. L. Kern, L. H. Ungar, and J. C. Eichstaedt, ‘‘Detecting depression and mental illness on social media: An integrative review,’’ Current Opinion Behav. Sci., vol. 18, pp. 43–49, Dec. 2017. [43] E. A. Ríssola, D. E. Losada, and F. Crestani, ‘‘A survey of computational methods for online mental state assessment on social media,’’ ACM Trans. Comput. Healthcare, vol. 2, no. 2, pp. 1–31, Mar. 2021. [44] S. Agarwal and A. Sureka, ‘‘Using KNN and SVM based one-class classifier for detecting online radicalization on Twitter,’’ in Proc. Int. Conf. Distrib. Comput. Internet Technol., Cham, Switzerland: Springer, 2015, pp. 431–442. [45] B. S. Nandhini and J. I. Sheeba, ‘‘Online social network bullying detection using intelligence techniques,’’ Proc. Comput. Sci., vol. 45, pp. 485–492, Jan. 2015. RODRIGO MARTÍNEZ-CASTAÑO received the B.Sc. degree in computer science and the M.Sc. degree in big data and data analysis technologies from the University of Santiago de Compostela, Spain. He is currently pursuing the Ph.D. degree. His research interests include big data technologies, distributed systems, information retrieval, and blockchain. DAVID E. LOSADA is currently an Associate Professor in computer science and artificial intelligence with CiTIUS, University of Santiago de Compostela, Spain. His current research interests include a wide range of information retrieval (IR) and related areas, such as early risk detection, text mining, IR evaluation, IR probabilistic models, summarization, novelty detection, and sentence retrieval. He is an Active Member of the IR Community and he regularly serves on the Program Committee of prestigious international conferences, such as SIGIR or ECIR. In 2011, he was recognized with the ACM Senior Member Award. JUAN C. PICHEL received the B.Sc. and M.Sc. degrees in physics from the University of Santiago de Compostela, Spain, and the Ph.D. degree in computer science from the University of Santiago de Compostela, in 2006. He was a Visiting Postdoctoral Researcher with the University Carlos III de Madrid, Spain, and the University of Illinois at Urbana–Champaign, USA. He also worked as a Researcher and the Project Manager with the Galicia Supercomputing Center, Spain. He is currently an Associate Professor with CiTIUS, University of Santiago de Compostela. His research interests include parallel and distributed computing, big data technologies, programming models, and software optimization techniques for emerging architectures. 42622 VOLUME 10, 2022