Analyzing recommender systems for health promotion using a multidisciplinary taxonomy: A scoping review

Analyzing recommender systems for health promotion using a multidisciplinary taxonomy: A scoping review Santiago Hors-Fraile, Octavio Rivera-Romero, Francine Schneider, Luis Fernandez-Luque, Francisco Luna-Perejon, Anton Civit-Balcells, Hein de Vries Keywords: Recommender system Tailoring Health intervention Behavior change Patient Recommendation Taxonomy Health promotion ABSTRACT Background: Recommender systems are information retrieval systems that provide users with relevant items (e.g., through messages). Despite their extensive use in the e-commerce and leisure domains, their application in healthcare is still in its infancy. These systems may be used to create tailored health interventions, thus reducing the cost of healthcare and fostering a healthier lifestyle in the population. Objective: This paper identifies, categorizes, and analyzes the existing knowledge in terms of the literature published over the past 10 years on the use of health recommender systems for patient interventions. The aim of this study is to understand the scientific evidence generated about health recommender systems, to identify any gaps in this field to achieve the United Nations Sustainable Development Goal 3 (SDG3) (namely, “Ensure healthy lives and promote well-being for all at all ages”), and to suggest possible reasons for these gaps as well as to propose some solutions. Methods: We conducted a scoping review, which consisted of a keyword search of the literature related to health recommender systems for patients in the following databases: ScienceDirect, PsycInfo, Association for Computing Machinery, IEEExplore, and Pubmed. Further, we limited our search to consider only English-lan-guage journal articles published in the last 10 years. The reviewing process comprised three researchers who filtered the results simultaneously. The quantitative synthesis was conducted in parallel by two researchers, who classified each paper in terms of four aspects—the domain, the methodological and procedural aspects, the health promotion theoretical factors and behavior change theories, and the technical aspects—using a new multidisciplinary taxonomy. Results: Nineteen papers met the inclusion criteria and were included in the data analysis, for which thirty-three features were assessed. The nine features associated with the health promotion theoretical factors and behavior change theories were not observed in any of the selected studies, did not use principles of tailoring, and did not assess (cost)-effectiveness. Discussion: Health recommender systems may be further improved by using relevant behavior change strategies and by implementing essential characteristics of tailored interventions. In addition, many of the features required to assess each of the domain aspects, the methodological and procedural aspects, and technical aspects were not reported in the studies. Conclusions: The studies analyzed presented few evidence in support of the positive effects of using health recommender systems in terms of cost-effectiveness and patient health outcomes. This is why future studies should ensure that all the proposed features are covered in our multidisciplinary taxonomy, including integration with electronic health records and the incorporation of health promotion theoretical factors and behavior change theories. This will render those studies more useful for policymakers since they will cover all aspects needed to determine their impact toward meeting SDG3. T 1. Introduction In order to achieve the United Nations Sustainable Development Goals, particularly goal 3, “Ensure healthy lives and promote well-being for all at all ages”(SDG3), it is imperative to invest in health-promotion activities. Over the years, numerous health-promotion interventions have been developed that help people adopt a healthy lifestyle and independently manage their health behaviors. Even though these interventions have been proven to be effective [1], they are not suitable for all as populations tend to present high levels of variability. In order to account for these differences, it is important to tailor the interventions to suit the diverse characteristics of a given population (i.e., economic standards, schedules, and residential location). Given this variability, new technologies can be used to solve geographical-access problems, deliver timely interventions, reduce intervention costs, and to even help users exert better control over the intervention [2]. However, computer-based health interventions suffer from a high user attrition rate [3], which presents a severe problem in public-health actions related to medical informatics. This is why it is relevant to use tailored health interventions [4], which can increase user engagement [5]. Tailored health interventions can also ensure more effective outcomes as compared to non-tailor approaches [6–9], and the integration of computers can make them scalable and even more cost-effective [10,11]. As technology evolves, new ways to implement such tailored interventions are being adopted, and researchers and policymakers need access to the correct tools to help them assess their design and usage suitability. One such innovative approach to computer-based tailored health interventions is the use of recommender systems (RS) [12]. RS are machine-learning, information-retrieval software tools, which predict the relevance of an item (e.g., a health resource or a message) for a given user (e.g., a patient) [13]. RS can select, tailor, and send health messages that are relevant to users based on previously retrieved user information. Even though RS have gained popularity in the last decade [14] and have been applied in a wide range of domains, such as ecommerce and leisure, their application in the health-promotion domain—as health recommender systems (HRS)—is still in its infancy. Although some HRS are already in use, there is still a long way to go before they become commonly used in health-related environments [15]. One reason for this could be that the potential of these systems [16] is not clearly defined and known to health professionals. For instance, they could be used as clinical-decision support systems if the end user is a healthcare professional, and as engines to generate relevant healthy lifestyle recommendations when patients are the end users. This latter application could significantly contribute to the field of health promotion. Nevertheless, some challenges should be solved such as legal liability and regulatory compliance. Currently, the legislative frameworks are not fitted to deal with potential errors of HRSs [17]. When sending health-promotion messages to the population by running public health campaigns or, more specifically, by using healthpromotion interventions, researchers in social marketing have reported that tailoring the content of these messages to the user’s context can improve their efficacy, as compared to the use of general content [18,19]. The added value of this strategy is that the user will then receive highly tailored messages tailored to his attitudes, social support system, self-efficacy, and the action plans needed to realize a particular health behavior. Yet, eHealth programs, including tailored eHealth programs, suffer from high dropout rates [3] One strategy aimed to overcome this is to offer messages that are also optimally adapted to user preferences, a strategy used by HRSs. HRSs may optimize the message tailoring for each user by selecting the message contents as per the patient’s need, sending them on a timely manner, and adapting the messages with changes in the patients’situation over time. Therefore, HRS may be a useful innovation over the current tailored systems as they may increase user engagement with the intervention and reduce costs. Considering the immense potential in applying RS to health promotion interventions, it is necessary to present a multidisciplinary overview of the results of using HRSs. To map the existing research literature pertaining to the use of HRS for patients, we conducted a comprehensive scoping exercise by exploring five different databases from different fields (technical, medical, and psychological). A preliminary search for previous scoping reviews that adopt a multidisciplinary approach to the topic of HRS for patients was also conducted in a variety of databases of different fields, but we did not find any relevant occurrences. The primary objective of this scoping review was to create a body of knowledge about the current state of HRS for patients in the last 10 years, in an attempt to answer the following research questions: What are the actual experiences with HRS for patients? What aspects have been studied? What are the existing research gaps that still need to be covered? These questions will be comprehensively addressed by following a multi-disciplinary approach adopted previously by some authors [20]. We analyzed four aspects—their domain, methodology and procedures, the usefulness of health promotion theoretical factors and behavior change theories, and technical details—in performing an indepth analysis from all angles, which is required to ensure the success of a tailored, computer-based health intervention. We proposed a scheme of classification for this analysis. It constitutes a new taxonomy which integrates both principles of traditional HRSs, and principles used in computer tailored eHealth approaches. The I-Change Model [21] was used to identify whether the HRSs also address these needed factors for behavior change. This taxonomy intends to facilitate the HRS classification, as there is no other taxonomy covering the those or similar aspects relevant for HRS to our knowledge. Therefore, both policy makers and researchers may easily identify knowledge gaps and common successful patterns in previous studies. For future studies such identification may contribute to increase the study fidelity by minimizing the possibilities of having undisclosed parts or overlooked aspects of the study that reduces their replicability. Future studies that complete the proposed taxonomy will be going through an exercise to include many of the needed requirements to meet SDG3, as it covers not only technical aspects, but also health communication aspects, and domain, and methodologies. This paper aims to present a clearer picture about how the existing studies can help policymakers make better decisions in terms of publichealth actions, including computer-based tailored health interventions, and to help researchers design future studies by building upon the existing knowledge. 2. Materials and methods 2.1. Design We conducted a scoping review following the PRISMA framework [22] to identify studies relating to HRS in which the end users were patients who received recommendations that may influence their health. 2.2. Search approach The main eligibility criteria were that the studies had to be articles published in journals over the last 10 years (from January 1, 2007, to October 18, 2016, when the search was performed), written in English, and dealing with RS that provided some sort of health recommendations to patients. The information sources selected were five databases, namely, PubMed, PsycINFO, Association for Computing Machinery (ACM), IEEExplore, and ScienceDirect. Electronic searches were conducted using the following keywords: ((“recommender systems”)OR (“recommender system”)OR(“recommendation systems”)OR(“recommendation system”)) AND (health OR patient OR patients). When the recommendation interface is, and what type of feedback can users provide to the HRS. The details of the taxonomy for each the 19 studies were independently extracted by two researchers (SHF, ORR) in parallel. After their extraction, classification discrepancies were resolved by mutual agreement in a later phase. An “N/A”could also be entered against a given field if analyzing it did not make sense for a given study, as could “Unknown”if a study did not provide information about that field. 2.6. Data analysis A researcher (SHF) went through all the taxonomy tables created and analyzed the common patterns, contradictory results, and the gaps in all the studies. All the identified elements were presented and discussed with the four other researchers (ORR, LFL, FS, and HDV). 3. Results We retrieved 905 initial results from the database search. These included 10 duplicate articles and 5 misclassified results that were actually books and proceedings. From the 890 remaining results, 84 met the inclusion criteria in the title review, 42 met the abstract review criteria, and 19 of them the full-text reading selection [28–46]. We will highlight some of the most relevant findings in the paragraphs below. The results obtained show that some studies have already used HRS to support patients for different purposes, with different approaches, and using different recommendation techniques. However, there are studies that appear to have misunderstood the concept of an RS. Of the 19 analyzed studies, 3 did not include systems that could be classified as an RS. Instead, they used other kinds of systems that computed recommendations and did not base their recommendations on the user or item feature similarity, or in previous knowledge incorporated by experts. We present the results for each of the features of our taxonomy. Some features did not apply to certain studies. For example, if a study proposed a theoretical algorithm or conducted a review, we cannot consider whether it has been tested with patients. We highlight these non-applicable studies for each feature analyzed. A complete description of all the extracted data using our proposed HRS taxonomy (Table 1) can be found in Appendix B. 3.1. Aspects studied 3.1.1. Domain Of all 19 eligible studies, 76.32% had the domain aspects we looked for. Of these, most of them focused on generic health promotion rather than recommendations relating to specific diseases (i.e., diabetes). The most frequently covered target population comprised adults (including young and healthy adults). Derived from the target groups, the age ranges covered can be seen in Fig. 2. Please, note that one study can cover several age groups. For 9 studies, either the age was not applicable or the age range was not specified. At least 10 studies covered the young-adult age group. No study reported coverage for children specifically, and the three studies that covered a population under 19 years of age were designed for diabetics or the overweight population in general; therefore, we included these in the chart. More than half the studies used messages as their recommendation items. Other less frequently used recommendation items were people, and health resources. Similarly, more than half the studies reported at least a mobile-based interface through which the recommendations were delivered. The studies were conducted in six countries, in the United States, a country in Asia, and four countries in Europe. Further, 60% of the studies in which tailoring was applicable stated that they implemented some type of tailoring technique. offered the option, keywords were sought in the entire text (not only in titles; abstracts; and/or metadata). We did not systematically assess the methodological rigor of the articles included as reflected in the convention of scoping reviews [23]. An example of the search process can be found in Appendix A. 2.3. Study selection procedure The study selection was divided into four phases, as described in the PRISMA framework. The first phase (identification) consisted of gathering all the articles retrieved from the database (904 results). This process was done by three researchers (SHF, ACB, FLP) who examined each article in parallel; an article was considered to have passed to the following phase if least one reviewer marked it down as relevant. After removing the duplicates (10 articles) and filtering some publications that were initially retrieved but not published in journals (3 proceedings and 1 book), the three researchers ended up with 890 results. They considered the results indicating the same content in different editions of the same paper to be duplicates. During the second phase (screening), the three researchers screened all the titles of the entries, after which they checked all the studies for eligibility (third phase) using the present inclusion and exclusion criteria. Studies were included if they dealt with HRS and if the end users of the system were patients, irrespective of the type of analysis performed. Studies that did not meet these criteria were excluded. In case of doubt, for example, if the titles were not descriptive enough, the researchers were asked to accept the paper since it could be excluded in the later phases. Accordingly, a result selected by any of the three researchers passed to the next phase, the inclusion phase (84). In this phase, the same three researchers read the abstract of the papers and followed the same acceptance criterion. 2.4. Full paper review The selected publications (42 articles) were fully read to assess their eligibility for the quantitative analysis. Only those publications that all three researchers agreed to pass to the quantitative analysis phase did (19), as shown in (Fig. 1). 2.5. Data extraction Our proposed taxonomy intended to cover the relevant information to meet the requirements of SDG3 and was based on the intuitive approach described in the study of Nickerson et al. [24]. However, we followed a two-step approach to ensure that it had the five features that Nickerson et al. proposes for a useful taxonomy: namely, being concise, self-explanatory, robust, comprehensive, and extendible. The first step was to choose the aspects using expert opinion. One of the researchers (SHF) proposed the two first taxonomy aspects and their features, and these were discussed and completed by researchers ACB, ORR, and LFL. The second step was to complete the taxonomy using previous studies, deriving a third aspect from the MIRO study that used the I-Change Model [25], and a technical aspect from previously proposed classifications by Schafer [26] et al. and Montaner et al. [27]. As a result, our taxonomy has four aspects. The first one is the domain aspect, which help us understand the general features of the study, such as what therapeutic area is being addressed, who are the target population, and what items are being recommended. The second one is methodological and procedural aspect, which let us identify the robustness of the study using features such as the number of test users, the system integration with an Electronic Health Record (EHR), and the study cost-effective- ness. The third aspect is the health promotion theoretical factors and behavior change theories, which assesses how much the intervention is grounded in health promotion and psychological techniques. The fourth and final one is the technical aspect, which determines the features of the HRS algorithm such as the used information filtering method, what 3.1.2. Methodology and procedures Upon analyzing the methodology and procedures, we found that 23.03% of the results we looked for were applicable and actually reported. The metrics used to assess the performance of the interventions were associated with the technical performance of the HRS (i.e., precision, recall, F-measure, and accuracy). In a lower percentage of the studies, the user perception (i.e., satisfaction, perceived usefulness, value, and trust) and health-related outcomes (i.e., weight loss) were also considered. Seven studies included tests with users. Two studies measured effectiveness in terms of patient outcomes, one of them not reporting its effectiveness, and the other reporting a positive effect with the control group’s average measure for weight loss doubling after the intervention; this outcome applied to 12% of the study population. Regarding the length of the intervention and the session frequency, the studies reported interventions lasting from 14 days to 4 months long, involving one session where the patient interacted with the HRS and received recommendations. Thirteen studies could have benefited from being connected to an EHR, and two of them reported having a connection with an EHR. No study reported the cost-effectiveness of the intervention. 3.1.3. Health promotion theoretical factors and behavior change theories In the studies we analyzed, 100% of the results did not find evidence of features of this aspect. Fig. 1. Methodology flow diagram. 3.1.4. Technical aspects When we analyzed the technical aspects, we found that 45.27% of the studies contained the information we sought. The results of these studies showed that the Top-N interface (a list of the N most probably relevant recommendations) was used the most for the recommendations. The most frequently used recommendation technology features were people-to-people correlation and user inputs, either standalone or in combination with other recommendation technologies. The “request recommendation list”technique was the most used for finding recommendations. In 70% percent of the studies, the user profiles were manually generated. The techniques to represent the user-profile analysis were applicable to 12 studies. The most commonly repeated profile representation technique was the vector space model, followed by the history-based model and user-item ratings. In almost 77% of the cases, no profile learning technique was needed because they already had a database with a user profile or had implemented collaborative filtering algorithms. In addition, among all the studies, four reported a profile adaptation technique. Half the studies analyzed did not included any feedback system, 40% included an explicit feedback system, and 10% implemented an implicit feedback system. The most common method of filtering information was pure collaborative filtering (Fig. 3), followed by hybrid methods, content-based filtering, and knowledge-based techniques. Five studies reported their user-profile-item matching technique, and 80% of them had implemented the nearest neighbor approach. This approach recommends new items to a given user among the items other similar users—who are called ‘neighbors’. The neighbor similarity can be computed in different ways such as using demographic data, or the users’item rating history. 4. Discussion Using our taxonomy to extract the features of the studies helped us to identify some relevant issues for discussion. Domain Therapeutic area The targeted disease or recommendation topic. Target population Description of the users, and other exclusion and inclusion criteria Type of recommendation (items) Messages, people, hospitals, paths,… Device interface Mobile, web, mobile and web, other (i.e., smartwatch display) Tailoring Yes/No Country Country or region where the intervention was conducted Methodology and procedures Used metrics to assess performance Metrics can be technical (F-score, precision, recall,…) or not (quit smoking,…) Number of test users 800, 45, 230,…(detail intervention and control groups, if applicable) Effectiveness on patients Quantitative measure of the aim of the study (i.e., 30% more physical activity in the intervention than in the control group, average weight loss during the study for obese patients, …) Success percentage % of patients that met the objectives of the study (i.e., quit smoking) Duration of the total intervention Total length of the period that the users were exposed to the HRS Number of sessions Average number of times the users interacted with the HRS during the intervention Electronic Health Record connection Yes/No Cost-effectiveness Yes/No (If yes, include the details) Health promotion theoretical factors and behavior change theories Attitude Yes/No Social influence Yes/No Self-efficacy Yes/No Action and Coping planning Yes/No Supporting Identity change Yes/No Rewarding abstinence Yes/No Advising on changing routines Yes/No Advising on coping Yes/No Advising on medication use Yes/No Technical aspects Recommendation interface a Recommendation technology a Finding recommendations a Initial profile generation techniques b Profile representation technique b Profile learning technique b Relevance feedback b Profile adaptation technique b Information filtering method b User-profile item matching technique b a These technical aspects were retrieved directly from the proposed classification of Schafer et al. b These technical aspects were retrieved directly from the proposed classification of Montaner et al. Fig. 2. Number of studies for each age group. Table 1 Taxonomy of health interventions using HRS. 4.1. Domain Although all the therapeutic areas extracted have a direct bearing on the achievement of SDG3, most of them focus on healthier nutrition and generic healthy-lifestyle promotion. More disease-specific HRSs are needed, which address non-generic topics and conditions. In particular, we believe that an excellent area would be substance abuse, one of the issues targeted by SDG3. As tailored messages have proved to be useful in reducing the intake of harmful substances [47], it would be feasible to design and implement an HRS that addresses this issue. Most of the studies were concentrated in two countries, Spain and Taiwan, which together represented more than 30% of all the studies. In order to achieve a comprehensive vision of the impact of HRS, more countries, especially low- and middle-income ones, should conduct studies on HRS since culture and perceptions of digital elements entering the healthcare loop may affect their actual effectiveness. Although the most recommended type of item in these studies were messages, none of the studies described anything about them. Consequently, it was not possible to assess them for inclusion of communication and behavior change features. This may have been due to the fact that they overlooked the importance of the message content or that they were not allowed to share the content. The large proportion of the mobile interfaces used in the reported HRS point in the right direction, toward universal access to healthcare services and resources, especially in low- and middle-income countries. 4.2. Methodology and procedures The effectiveness of HRS on patients was not described in 17 out of the 19 studies. This may be a consequence of the fact that several studies presented theoretical systems, reviews, or descriptions of systems whose results are yet to be achieved in the future. We also noticed that none of the studies reported on the cost-effectiveness of these systems, highlighting the need for further analysis on this feature in health interventions involving HRS. In addition, few studies in our sample used tests in order to assess user acceptance, and suitability of the system to meet its purpose with real users. Finally, sample sizes were low; only one study involved more than 90 users. We were therefore unable to determine the clinical or health outcomes since the statistical power of the samples are very low. Only one study reported health outcomes with a two-fold improvement when using the HRS [42]. However, this result is severely compromised since only three testers completed the study. EHRs can be used to define the profile of each user such that the recommendations are based on their previous health records. However, only two studies used EHR. We acknowledge that privacy and legal barriers may be the reason for such a small figure. Integration with user data may require additional effort at both the management and technical levels. In addition, the EHR usage is a good way to reduce the manual data entry of user profiles in the HRS, and to increase the extent of information on user characteristics to yield more accurate recommendations. We should take into consideration that it is more common to use alternative platforms and not integrate the experimental system with EHRs until they are mature and at the final phases before being explored. This confirms that the use of HRS is in its infancy and that they are a potential tool to achieve SDGs that have not yet been met. 4.3. Health promotion theoretical factors and behavior change theories We were unable to assess the extent of usage of health behavior theories and factors, because of the complete lack of information about how these messages were designed. Since a description of the messages and the length of intervention are the key elements in replicating the studies and building upon their experience, the utility of the existing evidence is limited. There is a need to develop and analyze additional studies with a more complete description of the intervention and how messages were designed. 4.4. Technical aspects Although the HRS concept was not correctly applied in some situations, only 4 of the 15 studies that were not reviews or theoretical descriptions comprehensively described the technical specifications of the HRS in terms of the classifications and categories used [48,49]. Consequently, there is little evidence of HRS characteristics that have been tested in the healthcare domain (Table 2). An important technical aspect concerns the limited description of applications of any profile adaptation technique. Only 3 out of 10 applicable studies implemented this technique. In order to provide more accurate recommendations over time, HRS need to evolve with the users. This means that these systems should ensure that user information is updated. Similarly, only 5 out of 14 studies implemented some kind of user feedback. Both the profile adaptation and user feedback are key factors for computer health education because the recommendations sent to the users need to be adapted to their current status and Fig. 3. Collaborative filtering recommender system concept diagram, the most used in the analyzed studies. updated based on their answers. Otherwise, we will rely on the user’s initial status, which will probably not yield accurate results in terms of behavior change interventions that need time to work (i.e., smoking cessation). 5. Conclusions This paper presents a comprehensive scoping review of HRS to explore the current experiences of health interventions for patients using these systems. Due to the lack of a defined taxonomy for these purposes, we also propose a multidisciplinary taxonomy to classify these systems and determine the aspects analyzed and the gaps that should be addressed. We encourage future HRS studies to make sure they follow this taxonomy, assessing domain, methodology, health promotion strategies, and technical aspects. It has been useful to discover some unmet SDG3 needs when using HRS. We consider this taxonomy may be relevant for future use as reporting the domain aspects will contribute an easy context categorization. The methodology and procedures aspects will make easier to understand the robustness and fidelity of the study. Reporting the health promotion theoretical factors and behavior change theories will explain whether how the behavior change the HRS wants to provide is backed by actual theories. Finally, the technical aspects reporting will break down the necessary details to repeat and evolve successful studies. Future HRS studies should cover at least all aspects proposed in our taxonomy when disseminating their results. As a result, policy makers will be able understand their impact towards SDG3. Although the studies analyzed present interesting approaches that could help meet SDG3, there remain several challenges. In terms of domain, we saw that most of the studies targeted the adult population, were oriented to generic health promotion and nutrition, and were conducted in a reduced number of countries. For the methodological and procedural aspects, we identified a lack of reported results and costeffectiveness, few and limited patient-testing cases, and that not all studies made use of EHR data. In terms of the health promotion theoretical factors and behavior change theories aspects, we found a complete dearth of information. In terms of the technical aspects, we identified that the studies do not report complete information about the systems; that there are systems mislabeled as RS; and that most of the systems have limitations in terms of generating user profiles, adapting the profiles to changes in the user’s circumstances, and collecting feedback from patients. Consequently, many of the studies may still be considered blackboxes whose details about how recommendations are generated are unknown. Although machine learning algorithms are difficult to interpret, and sometimes the dissemination is not aimed towards a full description of the systems, it is necessary to expose their details for both facilitating future research, and providing the information to make informed decisions at a policy maker level. Some institutions are introducing laws to remedy this lack of transparency. For example, the EU have approved the ‘General Data Protection Regulation’, which will come into force in 2018. It will ban systems generating decisions based solely on automated processing, which may clearly affect HRS that have not doctors in-the-loop [50–53]. That is why we recommend including health care professionals in the design phase of the HRS algorithm and the actual items that are going to be sent, as well as making them part of the intervention with the HRS as some studies are doing [54]. Due to the lack of reported key data in many of the studies of this review, we conclude that it is not possible to provide a guide of specific recommendations in the design of HRS to meet SDG3 yet. Future researchers should strive to innovate in terms of research areas and target groups. They should design HRS-based health promotion interventions by taking into consideration health promotion theoretical factors and behavior change theories, and specifying how the recommended items are made: their contents and wording, the frequency at which they are sent, and the exact tailoring techniques they use. Outlining these factors is also needed in order to be able to understand why certain interventions were or were not effective. In addition, the studies should describe their health-related metrics and test them with a sufficient number of users to achieve statistically significant results. Otherwise, technologyrelated metrics (i.e., F-score, precision, and accuracy) may prove inadequate to justify the cost and usage in a real-world setting. In this sense, it is necessary to continue reporting results on the evolution of HRS studies, since much existing evidence comes from descriptive theoretical studies or introductory studies. Paying more attention to the technical aspects, such as using correct terminology and comprehensively describing the systems, would benefit other researchers and policymakers willing to build on the previous successful experiences. Policymakers should facilitate the secure usage of EHR that can feed into HRS and promote new studies that focus on analyzing the costeffectiveness of these systems. As long as this type of analysis is not conducted, we encourage policymakers to propose and support studies pertaining to HRS in other therapeutic areas apart from nutrition and general well-being. A focus on relevant areas that can help meet SDG3, such as smoking cessation, oncology, mental health, and pregnancy and the early maternity stages, could help population risk prevention and enable users to manage symptoms, thereby having a global impact. Implications and direct applications for researchers and policymakers: Below are some aspects to consider when applying HRS to computer-based tailored health interventions for public health promotion. •Implication 1: Policymakers should promote the use of HRS to meet SDG3 because they can potentially act as a tool for scalable health promotion interventions, especially those that use mobile interfaces. •Implication 2: Other therapeutic areas apart from the ones included in this study are can also benefit from HRS, such as mental health, substance abuse, chronic diseases management, or health education for maternal care and childcare. •Implication 3. Policymakers should be aware that not all systems that claim to be an HRS are correctly defined. This may be misleading when assessing HRS-related results and making decisions about them. A deeper analysis to validate the correct classification by an IT expert is recommended. •Implication 4: Wherever possible, policymakers should facilitate Domain Methodology and procedures Health promotion theoretical factors and behavior change theories Technical Research on sparse therapeutic areas Specific cohorts not usually addressed Completely unreported Terminology misconception Lack of studies targeting teenagers and children Lack of reported results Limited profile adaptation techniques implemented No experience in low- and mediumincome countries Few patient experiences and limited number of participants Limited patient feedback systems included Few cases with EHR integration Manual initial user-profile generation Unreported cost-effectiveness Generic, superficial details used for RS classification Table 2 Gaps in HRS in terms of meeting SDG3. EHR integration with HRS for user-profile creation, which will help tailor the system’s recommendations to the user’s context. This can be done by, for instance, adopting secure computer communications protocols and providing a sample EHR for executing validation tests. •Implication 5: When using a public computer to run tailored health promotion interventions through HRS, policymakers should ensure that the team leading the intervention is a multidisciplinary one, including experts in behavior change, tailored health promotion, healthcare professionals, statisticians and technicians, who can collaboratively come up with a detailed design. In tailored interventions, special care should be taken to include a feature where the user profile is updated as the system adapts to the users’changing situation over time. •Implication 6: Although there is immense potential in the use of HRS in health interventions, there is no information on the effectiveness nor cost-effectiveness thus far, indicating the need for further studies to address these aspects. In sum, to better identify interventions in computer-based health promotion with HRS that covers all relevant aspects—the domain, methodological and procedural aspects, health promotion theoretical factors and behavior change theories, and technical aspects—policymakers can apply our taxonomy for each intervention. 6. Limitations This scoping review analyzed journal articles from five databases, but additional results may be obtained by taking into consideration conference proceedings and grey literature and by using other databases. The methodological rigor of the articles included was not systematically assessed as per the convention of scoping reviews. Authors contributions Santiago Hors-Fraile led the scoping review, contributed to all phases of the analysis, and wrote the main body of the manuscript. Octavio Rivera-Romero contributed to the quantitative analysis and supported with manuscript writing. Antón Civit-Balcells and Francisco Luna-Perejón contributed to the identification, screening, and eligibility phases. Finally, Luis Fernández-Luque, Francine Schneider, and Hein de Vries contributed to the writing, structure organization, and revision of the manuscript. Conflicts of interest None identified. Summary points What was already known about the topic: •HRS can be used to automatically tailor health information. •There is a growing interest in the scientific community about the use of HRS, and some studies have already been conducted for health promotion. •The application of tailoring and health communication theories are effective for behavior change. What this study contributed to existing knowledge: •HRS adoption to foster healthy lifestyles and promote wellbeing is currently lacking in terms of scientific evidence and only a few experiences that involve a sufficient number of users. This poses a challenge for policymakers and researchers to make decisions regarding the use of such systems. HRSs have been applied to very few areas that would meet the requirements of SDG, indicating that such systems need to be applied to new unexplored areas. •Despite the apparent interest in tailoring messages, the data reported is insufficient to determine whether the messages are indeed tailored using health communication theories. Besides, there is little information about the application of behavior change theories in HRS. •In order to achieve effective behavior change or to maintain a healthy lifestyle, it is necessary to take into account the current status of the user and the subsequent evolution of their circumstances. The current HRS do not place much emphasis on receiving feedback and adapting according to the user’s context. •This paper has contributed a taxonomy for classifying HRS intended for patients, which can be used by researchers and policymakers in future studies to visualize and understand each HRSs approach. Acknowledgements All authors acknowledge the valuable feedback sent by the two editors of this especial issue. The present study was co-funded by the Project SmokeFreeBrain “Multidisciplinary tools for improving the efficacy of public prevention measures against smoking”of the European Union's Horizon 2020 research and innovation programme under the grant agreement No 681120. Appendix A. Example of the search process Researchers who wish to repeat the search in Science Direct, will have to click on “expert search”and then introduce the following text without the brackets: [(“recommender systems,”OR “recommender system,”OR “recommendation systems,”OR “recommendation system”) AND (health OR patient OR patients)]. Next, they should select a year range between 2007 and 2016 and make sure that the checkboxes against journals and books are ticked. Some extra publications may be retrieved, since it is likely that some publications were released from October 14 to December 31. Similarly, the same query can be introduced in the PubMed search bar and filtered by publication date “January 1, 2007 and October 18, 2016.” When using to the ACM digital library, this query was adapted to the database library as follows: +(“recommendation system”“recommender systems”“recommender system”“recommendation systems”) + (patients patient health); next, we filtered all results between 2007 and 2016. Similar searches were performed in the remaining databases. Table A1 Studies domain analysis. Title Therapeutic area Target population Type of recommendations (items) Interface Implemented tailoring elements Country of the study A smart mirror to promote a healthy lifestyle Cardio-metabolic risk Healthy adults (25–60) & Non-pregnant or breastfeeding & not claustrophobia & no mental disabilities & no overt disease Messages Other (mirror) Yes Italy and France Collective-intelligence recommender systems: Advancing computer tailoring for health behavior change into the 21st century. Generic health promotion N/A Messages N/A N/A N/A Constructing recommendation systems for effective health messages using content, collaborative, and hybrid algorithms. Generic health promotion N/A Messages N/A Yes N/A Consumers' intention to use health recommendation systems to receive personalized nutrition advice. Nutrition Not specified Messages Digital (E-mail) vs Fitness Clubs and Doctors Yes The Netherlands Design and evaluation of a cloud-based Mobile Health Information Recommendation system on wireless sensor networks Generic health promotion Young adults Messages Mobile and Web Yes Taiwan Design of a real-time and continua-based framework for care guideline recommendations. General chronic patients preventive care Caregivers of chronic patients Messages Mobile N/A Taiwan glUCModel: a monitoring and modeling system for chronic diseases applied to diabetes. Diabetes Diabetics Messages Web No Spain Health recommender systems: concepts, requirements, technical basics and challenges. Generic health promotion N/A N/A N/A Yes N/A Mobile peer support in diabetes. Diabetes Diabetics Messages & People and communities Mobile Yes N/A Multimodal hybrid reasoning methodology for personalized wellbeing services Generic health promotion Healthy adults & Non-pregnant & not disabilities & no medical complications Messages Mobile Yes Unknown Nutrition for elder care: A nutritional semantic recommender system for the elderly Nutrition Elderly Messages Web Yes Spain Personalized healthcare cloud services for disease risk assessment and wellness management using social media None (technicalonly) Not specified Doctors Web No N/A Predicting potential side effects of drugs by recommender methods and ensemble learning Drug side effects N/A Drug side effects N/A No N/A Rethinking Health: ICT-Enabled Services to Empower People to Manage Their Health Generic health promotion N/A N/A N/A N/A N/A Social networks for improving healthy weight loss behaviors for overweight and obese adults: A randomized clinical trial of the social pounds off digitally (Social POD) mobile app Weight loss Overweight and obese adults with Android smartphones/tablets & not psychiatric illness & not receiving treatment for drug or alcohol dependency & not eating disorder & not pregnant & not breastfeeding & not heart condition & not chest pain & lose consciousness People (Other users) Mobile No USA Supporting self-management of obesity using a novel game architecture. Obesity Overweight Alternative strategies to coping with factors influencing obesity (i.e. stress) N/A, but mobile is suggested N/A N/A TPLUFIB-WEB: A fuzzy linguistic Web system to help in the treatment of low back pain problems Low back pain Adults Messages Web Yes Spain Ubiquitous Multicriteria Clinic Recommendation System. Generic health services General public Clinic and paths Mobile No Taiwan Which Doctor to Trust: A Recommender System for Identifying the Right Doctors. Generic health services General public Doctor profiles Mobile and Web (Web app) No USA