Artificial Intelligence Algorithms for Collaborative Book Recommender Systems

Annals of Data Science (2024) 11(5):1705–1739 https://doi.org/10.1007/s40745-023-00474-4 Artificial Intelligence Algorithms for Collaborative Book Recommender Systems Clemens Tegetmeier1·Arne Johannssen1·Nataliya Chukhrova2 Received: 24 November 2022 / Revised: 10 May 2023 / Accepted: 13 May 2023 / Published online: 8 June 2023 © The Author(s) 2023 Abstract Book recommender systems provide personalized recommendations of books to users based on their previous searches or purchases. As online trading of books has become increasinglyimportantinrecentyears,artificialintelligence(AI)algorithmsareneeded to recommend suitable books to users and encourage them to make purchasing decisions in the short and the long run. In this paper, we consider AI algorithms for so called collaborative book recommender systems, especially the matrix factorization algorithm using the stochastic gradient descent method and the book-based k-nearest-neighbor algorithm. We perform a comprehensive case study based on the Book-Crossing benchmark data set, and implement various variants of both AI algorithms to predict unknown book ratings and to recommend books to individual users based on the highest predicted ratings. This study aims to evaluate the quality of the implemented methods in recommending books by using selected evaluation metrics for AI algorithms. Keywords Artificial intelligence ·Book recommender systems ·knn algorithm · Machine learning ·Matrix factorization algorithm ·Stochastic gradient descent method BArne Johannssen [email protected] Clemens Tegetmeier [email protected] Nataliya Chukhrova [email protected] 1University of Hamburg, Hamburg, Germany 2HafenCity University, Hamburg, Germany 123 1706 Annals of Data Science (2024) 11(5):1705–1739 1 Introduction Book recommender systems are often used by companies to present interesting and personalized book recommendations to their customers. The recommendations are supposedtoconvincethecustomertobuybooksintheshortrunandtousethebookrecommender system for further purchases in the long run. As online trading of books has considerably increased in recent years [1], book recommender systems have become more important. Online booksellers such as Amazon, Barnes & Noble, Waterstones, and Thalia have played an important role in this development. For instance, current challenges for recommender systems are taking into account the user’s context, e.g., time or mood [2–4], ensuring diversity [5] and including implicit ratings to a greater extent [6] when generating book recommendations. Generally, the research focus has turned to Artificial Intelligence (AI) algorithms, e.g., the number of papers considering deep learning techniques has increased significantly in recent years and is frequently applied to recommender systems [7–9]. Note that AI, like machine learning, deep learning, data mining, and Big Data analytics, is based on Data Science techniques, so these areas are closely related [10–12]. While AI refers to the development of intelligent techniques that can perform tasks that typically require human intelligence [13], Data Science is an interdisciplinary field that involves the extraction, processing, analysis, and interpretation of large and complex data sets. In particular, deep learning and various variants of neural networks offer a new way to address current challenges of recommender systems [14] and beyond [15–19]. However, when using these black box algorithms, the problem of missing explainability of how the recommendations are generated needs to be considered [20–22]. Inbookrecommendersystems, AI algorithms havethe task ofsuggestingbooksthat buyers are potentially interested in and that have not been read by them. Depending on how the AI algorithms are supposed to recommend books, a distinction is made between collaborative,content-based, and hybrid book recommender systems. In a collaborative book recommender system, AI algorithms access all book ratings that havebeensubmittedbyusersofthebookrecommendersystem.Basedonthesubmitted book ratings, the AI algorithms predict for each user the ratings for the books they have not yet rated. Then, the books with the highest predicted ratings can be recommended to each user [23]. Users mostly have rated a very small proportion of the books that exist in the data. Thus, AI algorithms have to predict more of the book ratings than are known. In this paper, we focus on AI algorithms in collaborative recommender systems. Two popular AI algorithms in collaborative recommender systems are the matrix factorization algorithm using the stochastic gradient descent method and the bookbased k-nearest-neighbor (knn) algorithm [24]. In this paper, both these algorithms are considered in the framework of the modified Book-Crossing data set. This data set from Cai–Nicolas Ziegler [25] is a kind of benchmark data basis for research on AI algorithms in collaborative recommender systems [26]. We investigate a subset consisting of 42,137 explicit ratings of the Book-Crossing data set that forms the data basis. The task of both AI algorithms is to predict the unknown book ratings of the modified Book-Crossing data set, and then to recommend the books with the highest predicted ratings to each user. By using different variants of both AI algorithms, this 123 Annals of Data Science (2024) 11(5):1705–1739 1707 paper aims to evaluate both these algorithms in recommending books based on the modified Book-Crossing data set. For this aim, the quality of both AI algorithms is measured by selected evaluation metrics for AI algorithms. This paper is organized as follows. In Sect.2, after introducing the basics of collaborative book recommender systems, a short overview of AI algorithms and common evaluation metrics is given. Section3presents the book-based knn-algorithm and the matrix factorization algorithm using the stochastic gradient descent method. In Sect.4, we provide a comprehensive case study based on the Book-Crossing data set. In particular, we establish modifications to the data basis, present our methodology and proposed procedure, give the results of the study, and discuss them in detail. In the frameworkofthecasestudyweshowhowthequalityofbothAIalgorithmsismeasured using selected evaluation metrics. For this purpose, the statistical software Rand the corresponding package Recommenderlab, which was developed for collaborative recommender systems [27], is used. Finally, Sect.5concludes the paper. 2 AI Algorithms in Collaborative Book Recommender Systems 2.1 Essentials AI algorithms in collaborative book recommender systems are equivalent to AI algorithms that are generally used in entertainment recommender systems (e.g., movie recommender systems), where the entertainment products are referred to as items. The research on book recommender systems depends very much on the research flow on entertainment recommender systems, and most of the results are transferable to book recommender systems. AI algorithms need data about the book ratings by users, which can be explicit or implicit book ratings. A book rating is called an explicit book rating if a user actively assigns a rating on a specific scale (e.g., a scale from 1 to 10, where 10 represents the most positive experience and 1 is the most negative experience) to a book, see Table 1. In contrast, an implicit book rating is not directly given by a user. Instead, the book ratings are predicted based on the user’s behavior [28]. For example, a rating of 1 is assigned to a book if a user reads the complete book whereas the book gets a rating of 0 if a user only spends a short time with the book. In the following, however, we focus on explicit book ratings. Table 1 Example of explicit book ratings by users (on a scale from 1 to 10) Book 1 Book 2 Book 3 Book 4 Book 5 User19––1– User2–––64 User31010–1– User4–109–9 User5–109–– 123 1708 Annals of Data Science (2024) 11(5):1705–1739 A collaborative book recommender system includes data about pbook ratings. There are nusers u1,...,unwho have rated books, and mbooks i1,...imthat have received a rating by a user. The pbook ratings are represented in an n×m user-book matrix B,see(2.1). B= i1i2··· im u1 u2 . . . un ⎛ ⎜ ⎜ ⎜ ⎝ r11 r12 ···r1m r21 r22 ···r2m . . ..... . . rn1··· ···rnm ⎞ ⎟ ⎟ ⎟ ⎠ (2.1) A user can rate a book only once. Every row shows the ratings by one user and every column represents the ratings for one book. Thus, every entry in the user-book matrix is a rating by one user for one book. Formally this means that the entry rui is the rating by user ufor book i. Rating all mbooks is the maximum amount of ratings a user can give. A book can receive a maximum of nratings meaning that every user has rated this book. In this paper, we consider rating predictions for users who have already rated at least one book and for books that have already received at least one rating by a user. Therefore, the cold start problem [29] that deals with the question of how to predict a ratingofauser or for a book without anyknowledgeabout past ratings is not addressed. Most users have rated only a small percentage of the mbooks, which implies that B is a sparse matrix. The density DB=p nm ·100 of the user-book matrix measures the percentage of the known pbook ratings in relation to the theoretically possible book ratings (i.e., nm ratings). It is important for the AI algorithms to be able to predict the large number of unknown ratings by a small number of known ratings. It should be noted that each user has a different view on the question of which rating corresponds to a certain book quality. One user may argue that the rating 6 is a good rating on a scale from 1 to 10, whereas another user considers only ratings greater or equal to 9 as good ratings. The mean rurepresents the average book rating by a user. Based on the standard deviation σ(ru)of a user’s book ratings, conclusions can be drawn whether a user has given similar book ratings (low standard deviation) or varying book ratings (high standard deviation). The mean of the ratings of a book riindicates how well users have rated the book on average. Additionally, the standard deviation σ(ri)describes the size of the rating range for a specific book. The mean of all pratings of the user-book matrix is given by μ. Based on the given book ratings, AI algorithms predict the unknown ratings of the user-book matrix B. This enables the AI algorithms to create an ordered list for each user with the Nbooks that received the highest prediction. As a consequence, the books on the list are recommended to each user. How AI algorithms deal with different perceptions of the rating scale by users is explained in Sects.3.1 and 3.2. 123 Annals of Data Science (2024) 11(5):1705–1739 1709 2.2 Memory-Based Versus Model-Based AI Algorithms The AI algorithms in book recommender systems are distinguished between memorybased and model-based AI algorithms. Memory-based AI algorithms access the entire user-book matrix to recommend books to users [23,30]. In contrast, model-based AI algorithms create a model from the user-book matrix. Based on this model, users get book recommendations. Within the memory-based AI algorithms, a distinction is made between user-based and book-based AI algorithms. They employ two different approaches to forecast the unknown ratings of the user-book matrix. User-based AI algorithms predict the missing ratings of every user based on similar user ratings. In contrast, book-based AI algorithms forecast the unknown ratings of every book by considering the ratings of similar rated books. Based on the forecasts of the unknown bookratingseveryusergetsthebooksrecommendedthatreceivedthehighestpredicted ratings. The knn-algorithm is a popular memory-based AI algorithm in collaborative book recommender systems [23,31]. Model-based AI algorithms can be classified into the fields of regression, clustering, neural networks, deep learning and dimensionality reduction [31]. Model-based AI algorithms are mostly dimension-reducing algorithms, and matrix factorization algorithms are often applied in this context. The matrix factorization algorithm using the stochastic gradient descent method and the matrix factorization algorithm using the alternating least squares method are two popular matrix factorization algorithms [23,24]. Additionally, research on neural networks and deep learning in collaborative recommender systems has increased significantly in recent years and can also be applied to collaborative book recommender systems [7–9]. 2.3 Evaluation Metrics for AI Algorithms in Book Recommender Systems AI algorithms in book recommender systems predict the unknown book ratings of the user-book matrix B. Based on the predictions, the AI algorithms suggest Nbooks to every user as an ordered list. The quality of the AI algorithms depends on the grade of satisfaction of the users in relation to the proposed books. However, the satisfaction is hardly measurable in reality. Thus, the quality of the algorithms can only be approximated by online or offline tests [28,32]. In the following, we focus on offline tests and the corresponding evaluation measures. The evaluation metrics can be divided into the fields of prediction accuracy,classification accuracy and diversity [23]. The values of the evaluation metrics depend on the characteristics of the considered data set (e.g., the range of the rating scale). Therefore, it is important to compare the quality of different AI algorithms using the same data set [28]. Split,bootstrapping and cross-validation are methods that can be used to evaluate the AI algorithms [27]. In particular, in a cross-validation, users are divided into a predetermined number of groups of equal size. The number of groups is equivalent to the number of iterations performed. In each iteration, one group is the test group and all other groups are considered to be the training groups. The test group is changed in everyiteration, so that after all iterations each user wasin thetest group once. The users of the training groups are referred to as training users and the users of the test group 123 1710 Annals of Data Science (2024) 11(5):1705–1739 Table 2 Overview of the classification accuracy (confusion matrix) Recommended Not recommended All Relevant books mre (TP) mrn (FN) mr Irrelevant books mie (FP) min (TN) mi All memnm are referred to as test users. In each iteration, a model is developed based on the given ratings of the training users. This model is then tested on the test users. In this process, some of the known ratings are employed to test the model and some of the known ratings of the test users are withheld to validate the model. Most offline evaluation metrics are measured by the predicted values for the withheld ratings. This is mostly done by taking the mean of the values regarding the offline assessment metrics in all iterations [27]. Prediction accuracy metrics measure how precisely an AI algorithm estimates the ratings. The larger the deviation of the predicted value from the true value is, the larger is the value of the metrics, and the worse the prediction accuracy of the AI algorithm. In the following, we consider three common prediction accuracy metrics, i.e., the Root Mean Square Error (RMSE), the Mean Square Error (MSE), and the Mean Absolute Error (MAE): RMSE =rui ∈rtest (ˆrui −rui)2 ptest MSE =rui ∈rtest (ˆrui −rui)2 ptest MAE =rui ∈rtest |ˆrui −rui| ptest Here, rtest denotes the ratings of the test data set that need to be validated and ptest denotes their number. The real ratings are referred to as rui, whereas ˆrui represents the predicted ratings of user ufor book i. Classification accuracy metrics determine whether a user receives book recommendations that are relevant to the user [23]. The larger the value of a classification accuracy metric, the larger the classification accuracy. Precision and Recall are the two most popular classification accuracy metrics and can be computed based on the confusion matrix given in Table 2: Precision =mre me Recall =mre mr Note that it holds 0 ≤Precision,Recall ≤1. 123 Annals of Data Science (2024) 11(5):1705–1739 1711 In Table 2, the following acronyms are used: True Positives (TP), False Negatives (FN), False Positives (FP), True Negatives (TN). Following Table 2, the recommended but irrelevant number of books mie corresponds to the type I error. Additionally, the number of books mrn that is not recommended but relevant can be interpreted as type II error. Precision measures how many of the recommended books are relevant, whereasRecall specifies howmanyoftherelevantbooksarerecommended. Therefore, Precision minimizes the type I error, while Recall minimizes the type II error. Note that minimizing one error type increases the other error type in many cases. This leads to a trade-off between the maximization of Recall on the one hand and the maximization of Precision on the other hand [23]. Another important evaluation metric is Diversity. The main idea is that users do not appreciate to have the same books suggested over and over again. Diversity can be measured in different ways [5,33]. One approach is to determine how many of the mbooks of the data set are recommended to the users. This ability is called Coverage. If there are many books that are not recommended to any user, this could mean a lack of diversification. Bobadilla et al. [34] defined a user’s coverage as the proportion of books not rated by the user that have been rated by one of the user’s nearest neighbors (a user’s nearest neighbors are users who have rated books similarly to the considered user). Yang et al. [23] presented a different approach to measure Diversity that takes into account the similarity of books recommended to one user and the similarity of books recommended to two different users. 3 AI Algorithms 3.1 Book-Based knn Algorithm The knn algorithm is a nonparametric algorithm [35]. In collaborative book recommender systems, it is used as a regression algorithm to estimate the missing values of the user-book matrix B. A distinction is made between the user-based and the bookbased knn-algorithm. We will mainly focus on the book-based knn-algorithm in the following. First, for every book, the similarity to all other books is measured by a similarity measure. Two books are considered to be similar, if users have given them a similar rating. The knearest neighbors of a book are the kbooks that are most similar to the book. Every book has users who have not rated the book. Based on one user’s ratings for the knearest neighbors of the book the user’s rating for the book can be predicted. Therefore, the user’s ratings for the knearest neighbors are weighted with the value of the corresponding similarity measure. For example, in Table 1, books 2 and 3 have received similar ratings by users 4 and 5. Book 2 has been highly rated by user 3. Therefore, the predicted rating of user 3 for book 3 could also be high. With this approach, the knn algorithm tries to predict all unknown book ratings. After obtaining the predictions, the Nbooks with the highest predicted ratings are suggested to each user [23]. 123 1712 Annals of Data Science (2024) 11(5):1705–1739 The similarity of two books is measured by a similarity measure. A common similarity measure is the Bravais-Pearson correlation coefficient [36] wi gh =u∈Urug −rg·(ruh −rh) u∈Urug −rg2·u∈U(ruh −rh)2(3.1) with−1≤wi gh ≤1,wherewi gh indicatesthesimilarityofthebooksigandihmeasured by the Bravais-Pearson correlation coefficient. Note that only the users who have rated both books (i.e., u∈U) are used for the calculation. Some books may generally have been rated higher than other books. This is taken into account by subtracting the respective mean. The mean of book igis given by rgand the mean of book ihis given by rh. Therefore, a user’s rating counts as a positive rating only if it exceeds the mean of the book’s ratings. The denominator contains the standard deviation of the ratings from the mean. A large standard deviation indicates that a book has received different ratings. In contrast, a low standard deviation means that a book has received mostly the same ratings. In this way the differences of the ratings from the mean considered in the numerator are scaled. Thus, the Bravais-Pearson correlation coefficient takes into account general rating differences between books. A value of “1” implies a high similarity between two books, whereas a value of “−1” means that two books have received opposite ratings by users and are therefore not similar. The adjusted cosine similarity and the Euclidean distance are also popular similarity measures but are not considered in this paper [37]. To improve the quality of the similarity measure, the number of users who have rated both books could be considered. This ensures that books are only counted as similar if they have been rated similarly by multiple users [23,37]. This approach can be represented for the calculation of the similarity of two books as follows [23]: wi gh =2·|Ug∩Uh| |Ug|+|Uh|·wi gh (3.2) In (3.2), |Ug|represents the number of users who have rated the book ig,|Uh|indicates how many users have rated the book ihand |Ug∩Uh|corresponds to the number of users who have rated both books. The fraction gets smaller if fewer users have rated bothbooks. Multiplying the fraction by wi gh ensuresthat veryfewcommonuserratings result in a lower similarity of two books [23]. After calculating the similarity measures for all books, the knearest neighbors are determined for each book. The knearest neighbors of a book are the books that have the highest similarity value [23]. The unknown ratings of users for a book are determined by weighting the ratings of these users at the knearest neighbors of the book with the similarity measure [36,37]: P(ug)=rg+σ(rg)·ga∈N(g)ruga−rga σ(rga)·wi gga ga∈N(g)wi gga (3.3) 123 Annals of Data Science (2024) 11(5):1705–1739 1713 Here, P(ug)is the prediction for the rating of a user for book ig.Thekmost similar books to book igareinthesetN(g). These books are denoted by ga,...,gk.Only the books that have been rated by a given user are considered as nearest neighbors in the prediction. To account for differences in ratings between books, the ratings of the books are normalized. Additionally, the normalized ratings are weighted by the similarity measure wi gga. The user’s weighted normalized ratings for the books are transformed into the book’s rating scale by multiplying the standard deviation σ(rg)of the book’s ratings. The resulting value is added to the mean rgof book ig. If book ighas a large standard deviation, the multiplication by the standard deviation ensures that a positive value should cause a greater deviation of the predicted value from the mean rgof book ig. Thisapproachisknownas z-score.Anothercommonapproachisgivenbythedeviation from the mean. This approach does not take into account the standard deviation of the considered book and the book’s nearest neighbors [36,37]. In this way, the unknown ratings of users are estimated for each book. As explained later in this section, it may not be possible to predict all ratings. For each user, the books not yet rated by the user are sorted in a descending order according to the height of the predicted rating. The book-based knn algorithm can be classified as either a memory-based [23,31] or a model-based AI algorithm [38]. It depends on whether each time a list of book recommendations is created for a user, the similarity measures are recalculated using the user-book matrix. If this is true, the book-based knn algorithm is a memory-based AI algorithm. In contrast, the book-based knn algorithm can be considered as a modelbased algorithm if the similarity measures are recomputed only at regular intervals. The model is the similarity matrix that contains the similarity between the books. The classification of the book-based knn algorithm as a model-based AI algorithm is supported by research results showing that the similarities between the books are stable over time [38]. Thequalityofthe knnalgorithm dependsonthe choiceof thesimilaritymeasure and the possible consideration of the number of common users of two books. Additionally, the choice of the number of knearest neighbors plays an important role: choosing a small number of nearest neighbors could result in an overfitting to the ratings of the nearest neighbors [28]. Moreover, there is a risk that the nearest neighbors have not receivedanyratings by the user. This impliesthat it is not possible to predict aratingfor the book. Coverage (see Sect.2.3) is a measure to determine the extent of the problem [34,36]. In contrast, choosing a large number of nearest neighbors could lead to the problem of underfitting [35]. User’s ratings for books that are not similar enough to the book might influence the prediction too much. In extreme cases this could lead to unsatisfactory book recommendations. Therefore, it is often suggested to take a value in the range between 20 and 50 for the number of knearest neighbors to solve the trade-off between overfitting and underfitting [36,37]. 3.2 Matrix Factorization Algorithm Using the Stochastic Gradient Descent Method The matrix factorization algorithm using the stochastic gradient descent method is a model-based AI algorithm. The main assumption behind the matrix factorization is 123 1720 Annals of Data Science (2024) 11(5):1705–1739 examine the impact of diversity in book recommendations on user’s satisfaction with the recommended books. Adamopoulos/Tuzhilin [42] considered different subsets of the data set and proposed unexpectedness as an additional opportunity to improve the recommendation quality. Park/Tuzhilin [43] presented an approach for solving the long-tail problem (e.g., how to deal with books with few ratings) of recommender systems. Deldjoo et al. [44] used the data set as part of their study which examined the influence of data characteristics on the accuracy and fairness (e.g., measuring to what extent the quality of the recommendation depends on being in a specific group as age) of recommender systems. 4.2 Procedure and Methodology In this section, the methodology and the procedure to analyze the quality of the book-based knn-algorithm and the matrix factorization algorithm using the stochastic gradient descent method to recommend books from the modified Book-Crossing data set is presented. 4.2.1 Procedure On the one hand, the quality of 31 variants of book-based knn-algorithms, in which the number of knearest neighbors is varied from 20 to 50, is measured. On the other hand, the quality of 11 variants of the matrix factorization algorithm using the stochastic gradient descent method, in which the number of latent factors is varied from 5 to 15, is measured. In order to measure the quality of the variants of both algorithms, the values of the prediction accuracy metrics RMSE, MSE, and MAE and of the classification metrics Precision and Recall (see Sect.2.3) are considered. As an additional check on the quality of the variants, they are compared with the values of the evaluation metrics in a “random” algorithm (recommends books randomly) and a “popular” algorithm (recommends frequently rated books). The values of the evaluation metrics are determined using the R-package Recommenderlab that was developed by Michael Hahsler to test and evaluate collaborative recommender systems [27]. To compare the variants of both AI algorithms, they need to have the same training and test data set. Additionally, more than one training data set and one test data set should be used for the evaluation. This may reduce the risk that the division into a training and a test data set would affect the quality of the algorithms. To satisfy these important requirements for quality comparability, the option of the Recommenderlab package to determine an evaluation scheme is employed. Using the command set.seed ensures that the evaluation scheme is the same for all tested variants. We apply cross-validation with 10 partitions and 10 iterations for each variant of both AI algorithms. Therefore, the 1842 users of the modified Book-Crossing data set were divided into 10 partitions consisting of about 184 users. In each iteration, users of 9 partitions form the training data set and develop a model. This model is tested using the test data set, which consists of the users of one partition. Thus, each user is nine times in the training data set and once in the test data set (see Table 6). 123 Annals of Data Science (2024) 11(5):1705–1739 1721 Table 6 Overview of the settings at the rating scheme Method Percentage of training users Iterations Given GoodRating Cross-validation 90% 10 9 9 4.2.2 Evaluation Metrics To be able to measure the evaluation metrics, the rating scheme uses the option Given to specify how many of a test user’s known ratings should be used for testing and how many should be used for validation. The value “9” is set for Given. Thus, from each test user, 9 of the known ratings are utilized to test the model developed by the training data set. Based on the estimation of the remaining known ratings of the test users, the algorithms are validated. Since each user has submitted at least 10 ratings in the modified Book-Crossing data set, at least one rating is used for validation for every test user. The prediction accuracy metrics RMSE, MSE, and MAE are measured by the known ratings used for validation. After 10 iterations, the prediction accuracy metrics are determined as the mean of their values from these iterations. To measure the classification accuracy metrics Precision and Recall, the value for GoodRating was decisive for the rating scheme. The value of GoodRating indicates the rating from which on a book belonging to a validating rating is so relevant for a test user that it should be recommended to the test user. This is a hypothetical assumption since, in reality, the user has already rated the book. The book ratings of the modified Book-Crossing data set are on a scale of 1 to 10, where a value of “9” is chosen for GoodRating. This choice is based on the assumption that the user would like to receive a recommendation for a book that the user has rated 9 or 10. Additionally, this assumes that the user would not know the book yet. The number of ratings to validate with a rating of 9 or 10 is determined for each test user. Then, for each test user, the ratings for the 2056 remaining books are predicted, since 9 of the known ratings are used to test the model. Of the 2056 books, each test user was recommended once the 10 and once the 20 books with the highest predicted ratings. The main assumption for choosing the two list sizes is that a user would mostly only look at the recommendations placed at the top of the list. For each test user, Precision is measured as the proportion of books in the list from recommended books that were previously determined to be relevant books to validate. Recall is measured as the proportion of the relevant books to be validated. After 10 iterations, the values for Recall and Precision are taken as the mean of the results from the 10 iterations. 4.2.3 Book-Based knn Algorithm Previous research on collaborative recommender systems (as discussed in Sect.3.1) considersanumberof20–50knearestneighborsasoptimum.Basedonthissuggestion, 31 variants with values for kfrom k=20 to k=50 of the book-based knn algorithm are tested. 123 1722 Annals of Data Science (2024) 11(5):1705–1739 Table 7 Overview of the variants of the book-based knn-algorithm na.as.zero kNormalize Method normalize_sim_matrix alpha FALSE 20–50 z-score Pearson FALSE 0.5 Table 8 Overview of the variants of the matrix factorization algorithm using the stochastic gradient descent method kγλmin_epochs max_epochs min_improvement Normalize 5–15 0.001 0.015 50 200 0.000001 z-score The Bravais-Pearson correlation coefficient (3.1) is chosen as similarity measure. For normalization, the z-score approach (3.3) is used. The unknown book ratings are not set to 0 (option na.as.zero) because the similarity of two books in the Bravais-Pearson correlation coefficient is only based on the users who rated both books. The meaning of alpha is not defined in the Recommenderlab package and related instructions, so the value was left at the default value of 0.5 (note that pre-tests showed no change in the scoring metrics at different values for alpha). The option to normalize the similarity matrix of the books is not set, as general differences in the ratings are already taken into account when calculating the similarities of the books (see Sect.3.1). For an overview of the settings see Table 7. 4.2.4 Matrix Factorization Algorithm Using the Stochastic Gradient Descent Method For the variants of the matrix factorization algorithm using the stochastic gradient descent method, the number of latent factors is varied from 5 to 15 latent factors. Funk [40], as the founder of the method, stated in his blog entry 25 and 40 as values for a reasonable number of latent factors for the Netflix data set, where the user-item matrix has a size of 8.5 billion entries. Koren et al. [24] mentioned a number of 20 to 100 latent factors for the same data set. Since the modified Book-Crossing data set has approximately 3.8 million entries, values between 5 and 15 are chosen for the number of latent factors. For normalization, the z-score is used, as for the variants of the book-based knnalgorithm. The z-score is chosen, since the common approach of considering the bias bu, the bias bi, and the mean of all known ratings μin the prediction of the ratings could not be selected, see Sect.3.2. The other parameters (see Table 8) are left at the default values [27,40]. 4.3 Results In this section, the results from all variants of both AI algorithms are presented for the prediction accuracy metrics RMSE, MSE, and MAE as well as the classification accuracy metrics Precision and Recall. In order to additionally check the quality of theseAIalgorithmscritically, theyare also compared with the results of the twocontrol 123 Annals of Data Science (2024) 11(5):1705–1739 1723 Table 9 Comparison of the values of the best variants of both AI algorithms for RMSE, MSE, and MAE with the values of the control algorithms RMSE MSE MAE Random_1 2.092 4.383 1.563 Popular_1 1.562 2.443 1.178 SVDF_15 1.555 2.420 1.148 kNN_24 2.040 4.180 1.396 kNN_50 2.036 4.157 1.407 algorithms“popular” and “random”. The results are rounded tothethirddecimalplace. The values of the evaluation metrics for all variants of both AI algorithms (see Tables 12–16) and the two control algorithms “popular” and “random” (see Tables 17–18) can be found in Appendix B. For the matrix factorization algorithm, the variant with 15 latent factors received the lowest values (RMSE =1.555, MSE =2.420, MAE =1.148, see Table 9) and the variant with 5 latent factors the highest values (RMSE =1.560, MSE =2.437, MAE =1.159, see Table 12) for all three predictive accuracy metrics. In each case, the increase in a latent factor slightly improved the predictive accuracy metrics. Therefore, the difference between the worst and the best variant is about 0.005 for RMSE, about 0.017 for MSE, and about 0.011 for MAE. For the book-based knn algorithm, the variant with 24 nearest neighbors received the best value for the prediction accuracy metric MAE with a value of 1.396. The variant with 50 nearest neighbors performed the best for the predictive accuracy metrics RMSE (2.036) and MSE (4.157) (see Table 9). Table 9and Fig. 3show the variants of both AI algorithms with the best values for RMSE, MSE and MAE and the values of the control algorithms. The matrix factorization algorithm using the stochastic gradient descent method achieved for all variants lower values in the prediction accuracy metrics compared to all variants of the book-based knn algorithm and than the control algorithms. All variants of the book-based knn algorithm had lower scores on the three predictive accuracy metrics than the control algorithm “ random”. Compared to the control algorithm “popular”, all variants of the book-based knn algorithm had higher values. The minimum difference between the prediction accuracy metrics of both AI algorithms is 0.476 for RMSE, 1.72 for MSE, and 0.248 for MAE. The maximum difference is 0.506 for RMSE, 1.854 for MSE, and 0.262 for MAE (see Tables 12 and 14). In the following, the results for Precision and Recall are discussed for both a list of 10 and 20 recommended books. As for the book-based knn algorithm, for the list with 10 recommended books, the variant with 29 nearest neighbors achieved the best values for Precision (0.006) and Recall (0.017). In contrast, for the list of 20 recommended books, the variant with 22 nearest neighbors received the highest value for Precision (0.006), and the variant with 26 nearest neighbors received the highest value for Recall (0.024) (see Tables 10–11). For the matrix factorization algorithm using the stochastic gradient descent method, the variant with 5 latent factors obtained the highest values for Precision (0.016) and Recall (0.035) for the list of 10 recommended books. For the list of 20 recommended books, the variant with 5 latent factors at Precision (0.013) and the variant with 9 latent factors at Recall (0.060) performed best (see Tables 10–11). 123 1724 Annals of Data Science (2024) 11(5):1705–1739 Fig. 3 Graphical comparison of the values of the best variants of both AI algorithms for RMSE, MSE, and MAE with the values of the control algorithms Table 10 Comparison of the values of the best variants of both algorithms for Precision and Recall at the top 10 list with the values of the control algorithms TP FP FN TN NPrecision Recall Random_1 0.020 9.980 5.740 2040.260 2056 0.002 0.003 Popular_1 0.249 9.751 5.511 2040.489 2056 0.025 0.059 SVDF_5 0.158 9.842 5.603 2040.397 2056 0.016 0.035 kNN_29 0.061 9.751 5.699 2040.489 2056 0.006 0.017 Table 11 Comparison of the values for Precision and Recall of the best variants of both AI algorithms at the top 20 list with the values of the control algorithms TP FP FN TN NPrecision Recall Random_1 0.054 19.946 5.706 2030.294 2056 0.003 0.010 Popular_1 0.360 19.640 5.400 2030.600 2056 0.018 0.082 SVDF_5 0.264 19.736 5.496 2030.504 2056 0.013 0.057 SVDF_9 0.264 19.736 5.496 2030.504 2056 0.013 0.060 kNN_22 0.111 19.511 5.649 2030.729 2056 0.006 0.023 kNN_26 0.104 19.519 5.656 2030.720 2056 0.005 0.024 All variants of the matrix factorization algorithm using the stochastic gradient descent method had higher values for Precision and Recall than all variants of the book-based knn algorithm. Additionally, all variants of both AI algorithms had higher values for Precision and Recall than the control algorithm random (see Tables 10–11). For the list of 10 recommended books, the minimum difference between both AI algorithms for Precision is 0.007, while the minimum difference for Recall is 0.012. The maximum difference is 0.011 for Precision and 0.021 for Recall. For the list of 20 recommended books, the minimum difference between the AI algorithms is 0.006 for Precision and 0.029 for Recall. The maximum difference is 0.008 for Precision and 0.038 is Recall (see Tables 13,15,16). 123 Annals of Data Science (2024) 11(5):1705–1739 1725 Fig. 4 Graphical comparison of the values of the best variants of both AI algorithms for Precision and Recall For both AI algorithms, a higher value for Recall is observed for all variants for the list of 20 recommended books. The value for Precision is higher for all variants in the matrix factorization algorithm using the stochastic gradient descent method for the list with 10 recommended books. For the book-based knn algorithm, this is true for most variants, although the difference is much smaller here. Figure 4shows this tendency by looking at the variants of the AI algorithms that scored the highest for Recall or Precision. 4.4 Discussion For the chosen settings of both AI algorithms, all variants of the matrix factorization algorithm using the stochastic gradient descent method show superior performance compared to all considered variants of the book-based knn algorithm. The matrix factorization algorithm using the stochastic gradient descent method led to better results for the prediction accuracy metrics compared to both control algorithms. As for Precision and Recall, the matrix factorization algorithm showed a better performance than the “random” algorithm and a worse performance than the “popular” algorithm. Thus, the quality of the matrix factorization algorithm applied to the modified Book-Crossing data set can be considered as good. The variants of the knn algorithm led to better scores than the control algorithm “random” and worse scores than the control algorithm “popular” on the prediction accuracy and classification accuracy metrics. One reason for the poor performance of the book-based knn algorithm could be a possibly low coverage of the books of the modified Book-Crossing data set [34]: the coverage of a book is the proportion of users who have not rated a book and at the same time have rated one of the knearest neighbors of the book (see 123 1726 Annals of Data Science (2024) 11(5):1705–1739 Sect.2.3). Low coverage means a high probability that if a user has not rated a book, the user has rated only a very small fraction of the knearest neighbors, or in extreme cases, none of the knearest neighbors of the book. In the first case, the problem of overfitting the predicted rating to the user’s rating at the few knearest neighbors may occur [28]. In the second case, no prediction can be made for the rating. The good results of the control algorithm “popular” on the classification accuracy metrics recall and precision and on the prediction accuracy metrics might be related to the fact that users might like books that have been rated by many users. Based on Fig.4, where the best values of both AI algorithms for the lists of 10 and 20 recommended books are plotted, the trade-off between a high value for Recall and a high value for Precision described in Sect.2.3 can be seen. Moreover, the computation of Precision and Recall in the Recommenderlab package can be considered to be critical: when creating the list of recommended books for a test user, the 10 or 20 books with the highest predicted ratings were recommended. This involves predicting ratings for books with a rating to be validated and ratings for books where the true rating is unknown. Here, an AI algorithm lead to a high value for Recall if a large proportion of the books are recommended with a relevant rating to validate and a high value for Precision if a large proportion of the recommended books are books with a relevant rating to validate (see Sect.4.2). Here, it is not possible to state with certainty, whether the user might find the recommended books with the predicted rating more interesting than the ratings to be validated of the books for which the user has given a rating of 9 or 10. The values for Recall and Precision were therefore calculated but should only be interpreted with caution. 5 Conclusions In this paper, we investigated the performance of two popular AI algorithms for collaborative book recommender systems using the Book-Crossing benchmark data set. We implemented different variants of the book-based knn algorithm and the matrix factorization algorithm using the stochastic gradient descent method based on selected prediction and classification accuracy metrics as well as using two control algorithms. These variants are characterized by variations in the number of knearest neighbors in the book-based knn algorithm and in the number of jlatent factors in the matrix factorization algorithm using the stochastic gradient descent method. We performed a comprehensivecase studyto analyzethe qualityofbothAIalgorithmsfor collaborative book recommender systems to recommend books from the modified Book-Crossing data set. For the investigated variants of both AI algorithms, the variants of the matrix factorization algorithm using the stochastic gradient descent method showed superior performance. In contrast, the book-based variants performed worse than the variants of the matrix factorization algorithm using the stochastic gradient descent method and than the control algorithm “popular”. It seems that the poor performance of the book-based knn algorithm might be related to the problem of poor coverage of the book-based knn algorithm. 123 Annals of Data Science (2024) 11(5):1705–1739 1727 This paper considered users who have already rated books and books that have already received ratings. For AI algorithms, there is also the question of how to deal with new users who have not yet submitted ratings and new books that have not yet received ratings. This problem is known as the cold start problem. It deals with the question of which books are suggested to a new user and to which users a new book is suggested. Another interesting question is how AI algorithms deal with the grey sheep problem. This problem deals with users whose rating behavior is difficult to explain by any patterns, which makes it very difficult for AI algorithms to recommend suitable books to them. In addition to these aspects, future research could focus on the performance of both algorithms when tested on other recent book data sets such as the Goodbooks-10k data set [45] and the Goodreads data set [46,47], which are also frequently used in research about book recommender systems [6,21].1 Acknowledgements The authors thank both anonymous reviewers for their valuable feedback and suggestions, which were important and helpful to improve the paper. Author contributions ClemensTegetmeier:Conceptualization, Methodology, Software, Validation, Formal analysis, Investigation, Writing - Original Draft, Visualization Arne Johannssen: Validation, Formal analysis, Investigation, Writing - Original Draft, Writing - Review & Editing, Supervision, Project administration Nataliya Chukhrova: Validation, Formal analysis, Investigation, Writing - Review & Editing, Supervision. Funding Open Access funding enabled and organized by Projekt DEAL. Data availibility The data that support the findings of this study are available from the respective references as mentioned in the main text. Code Availability The code is available from the authors upon request. Declarations Compliance with Ethical Standards This article does not contain any studies with human participants or animals performed by the authors. Conflict of interest The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Disclosure of potential conflicts of interest (i) This manuscript is the authors’ original work, which has not been published nor submitted simultaneously elsewhere; (ii) all authors have checked the manuscript and agreed to the submission, and (iii) there is no conflict of interest. Open Access ThisarticleislicensedunderaCreativeCommonsAttribution4.0InternationalLicense,which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. 1The data sets are available at http://fastml.com/goodbooks-10k-a-new-dataset-for-bookrecommendations/ and https://sites.google.com/eng.ucsd.edu/ucsdbookgraph/home?pli=1. 123 1728 Annals of Data Science (2024) 11(5):1705–1739 Appendix A See Figs. 5,6,7,8,9,10,11 and 12. Fig. 5 Frequencies of the number of ratings per user in the modified Book-Crossing data set (the user with 964 submitted ratings is not included in this figure) Fig. 6 Frequencies of the number of ratings per book in the modified Book-Crossing data set 123 Annals of Data Science (2024) 11(5):1705–1739 1729 Fig. 7 Frequencies of the medians of one user’s ratings in the modified Book-Crossing data set Fig. 8 Frequencies of the medians of one book’s ratings in the modified Book-Crossing data set 123 1736 Annals of Data Science (2024) 11(5):1705–1739 Table 16 Values for Precision and Recall for the book-based knn-algorithm for 36–50 neighbors TP FP FN TN NPrecision Recall n kNN_36 0.054 9.758 5.706 2040.482 2056 0.005 0.014 10 kNN_36 0.099 19.524 5.661 2030.716 2056 0.005 0.022 20 kNN_37 0.054 9.758 5.706 2040.482 2056 0.006 0.014 10 kNN_37 0.101 19.523 5.660 2030.717 2056 0.005 0.022 20 kNN_38 0.052 9.760 5.708 2040.480 2056 0.005 0.014 10 kNN_38 0.098 19.525 5.662 2030.715 2056 0.005 0.022 20 kNN_39 0.051 9.761 5.709 2040.479 2056 0.005 0.014 10 kNN_39 0.099 19.525 5.661 2030.715 2056 0.005 0.022 20 kNN_40 0.052 9.760 5.708 2040.480 2056 0.005 0.014 10 kNN_40 0.097 19.527 5.663 2030.713 2056 0.005 0.022 20 kNN_41 0.053 9.759 5.707 2040.481 2056 0.005 0.015 10 kNN_41 0.098 19.526 5.662 2030.714 2056 0.005 0.023 20 kNN_42 0.056 9.755 5.704 2040.484 2056 0.006 0.016 10 kNN_42 0.099 19.524 5.661 2030.716 2056 0.005 0.023 20 kNN_43 0.058 9.754 5.703 2040.485 2056 0.006 0.016 10 kNN_43 0.101 19.523 5.660 2030.717 2056 0.005 0.024 20 kNN_44 0.058 9.754 5.702 2040.486 2056 0.006 0.016 10 kNN_44 0.099 19.524 5.661 2030.716 2056 0.005 0.023 20 kNN_45 0.059 9.753 5.702 2040.487 2056 0.006 0.016 10 kNN_45 0.101 19.523 5.660 2030.717 2056 0.005 0.023 20 kNN_46 0.060 9.752 5.700 2040.488 2056 0.006 0.016 10 kNN_46 0.102 19.522 5.658 2030.718 2056 0.005 0.024 20 kNN_47 0.058 9.754 5.703 2040.485 2056 0.006 0.016 10 kNN_47 0.103 19.521 5.658 2030.719 2056 0.005 0.024 20 kNN_48 0.059 9.753 5.702 2040.487 2056 0.006 0.015 10 kNN_48 0.104 19.520 5.656 2030.720 2056 0.005 0.024 20 kNN_49 0.059 9.753 5.702 2040.487 2056 0.006 0.015 10 kNN_49 0.103 19.521 5.658 2030.719 2056 0.005 0.023 20 kNN_50 0.057 9.755 5.703 2040.485 2056 0.006 0.015 10 kNN_50 0.101 19.523 5.659 2030.717 2056 0.005 0.023 20 123 Annals of Data Science (2024) 11(5):1705–1739 1737 Table 17 Values for RMSE, MSE, and MAE for the control algorithms “random” and “popular” RMSE MSE MAE Random_1 2.092 4.383 1.563 Popular_1 1.562 2.443 1.178 Table 18 Values for Precision and Recall for the control algorithms “random” and “popular” TP FP FN TN NPrecision Recall n Random_2 0.020 9.980 5.740 2040.260 2056 0.002 0.003 10 Random_2 0.054 19.946 5.706 2030.294 2056 0.003 0.010 20 Popular_2 0.249 9.751 5.511 2040.489 2056 0.025 0.059 10 Popular_2 0.360 19.640 5.400 2030.600 2056 0.018 0.082 20 References 1. Börsenblatt (2021) Online-Buchhandel mit hohen Wachstumsraten. https://www.boersenblatt.net/ news/buchhandel-news/online-buchhandel-mit-hohen-wachstumsraten-185281 2. Adomavicius G, Tuzhilin A (2011) Context-aware recommender systems. In: Ricci F, Rokach L, Shapira B, Kantor PB (eds) Recommender systems handbook. Springer, Boston, pp 217–253 3. Villegas NM, Sánchez C, DÍaz-Cely J, Tamura G (2018) Characterizing context-aware recommender systems: a systematic literature review. Knowl Based Syst 140:173–200. https://doi.org/10.1016/j. knosys.2017.11.003 4. Raza S, Ding C (2019) Progress in context-aware recommender systems—an overview. Comput Sci Rev 31:84–97. https://doi.org/10.1016/j.cosrev.2019.01.001 5. Kunaver M, PoŽrl T (2017) Diversity in recommender systems—a survey. Knowl Based Syst 123:154– 162. https://doi.org/10.1016/j.knosys.2017.02.009 6. Ramakrishnan G, Saicharan V, Chandrasekaran K, Rathnamma MV, Ramana VV (2020) Collaborative filtering for book recommendation system. In: Das KN, Bansal JC, Deep K, Nagar AK, Ponnambalam P, Naidu RC (eds) Soft computing for problem solving. Springer, Singapore, pp 325–338 7. Da’u A, Salim N (2019) Recommendation system based on deep learning methods: a systematic review and new directions. Artif Intell Rev 53(4):2709–2748. https://doi.org/10.1007/s10462-019-09744-1 8. Zhang S, Yao L, Sun A, Tay Y (2019) Deep learning based recommender system: a survey and new perspectives. ACM Comput Surv 52:1. https://doi.org/10.1145/3285029 9. Wu L, He X, Wang X, Zhang K, Wang M (2021) A survey on neural recommendation: from collaborative filtering to content and context enriched recommendation. arXiv preprint arXiv: 2104.13030 10. Olson DL, Shi Y (2007) Introduction to business data mining. McGraw-Hill/Irwin, New York 11. Shi Y, Tian Y, Kou G, Peng Y, Li J (2011) Optimization based data mining: theory and applications. Springer, Berlin 12. Shi Y (2022) Advances in big data analytics. https://doi.org/10.1007/978-981-16-3607-3 13. Tien JM (2017) Internet of things, real-time decision making, and artificial intelligence. Ann Data Sci 4:149–178. https://doi.org/10.1007/s40745-017-0112-5 14. Batmaz Z, Yurekli A, Bilge A, Kaleli C (2018) A review on deep learning for recommender systems: challenges and remedies. Artif Intell Rev 52(1):1–37. https://doi.org/10.1007/s10462-018-9654-y 15. FoerchBrenesR, JohannssenA, ChukhrovaN(2022) Anintelligent bankruptcy predictionmodel using a multilayer perceptron. Intell Syst Appl 16:200136. https://doi.org/10.1016/j.iswa.2022.200136 16. Yeganeh A, Shadman A, Abbasi SA, Pourpanah F, Johannssen A, Chukhrova N (2022) An ensemble neural network framework for improving the detection ability of a base control chart in non-parametric profile monitoring. Expert Syst Appl 204:117572. https://doi.org/10.1016/j.eswa.2022.117572 17. Kurani A, Doshi P, Vakharia A, Shah M (2023) A comprehensive comparative study of artificial neural network (ANN) and support vector machines (SVM) on stock forecasting. Ann Data Sci 10(1):183– 208. https://doi.org/10.1007/s40745-021-00344-x 123 1738 Annals of Data Science (2024) 11(5):1705–1739 18. Yeganeh A, Chukhrova N, Johannssen A, Fotuhi H (2023) A network surveillance approach using machine learning based control charts. Expert Syst Appl 219:119660. https://doi.org/10.1016/j.eswa. 2023.119660 19. Yeganeh A, Johannssen A, Chukhrova N, Abbasi SA, Pourpanah F (2023) Employing machine learning techniques in monitoring autocorrelated profiles. Neural Comput Appl. https://doi.org/10.1007/ s00521-023-08483-3 20. Zhang Y, Chen X (2020) Explainable recommendation: a survey and new perspectives. Found Trends Inf Retrieval 14(1):1–101. https://doi.org/10.1561/1500000066 21. Ghazimatin A, Balalau O, Saha Roy R, Weikum G (2020) PRINCE: provider-side interpretability with counterfactual explanations in recommender systems. In: Proceedings of the 13th international conference on web search and data mining. Association for Computing Machinery, pp 196–204 22. Li X, Xiong H, Li X, Wu X, Zhang X, Liu J, Bian J, Dou D (2022) Interpretable deep learning: interpretation, interpretability, trustworthiness, and beyond. Knowl Inf Syst 64(12):3197–3234. https:// doi.org/10.1007/s10115-022-01756-8 23. Yang Z, Wu B, Zheng K, Wang X, Lei L (2016) A survey of collaborative filteringbased recommender systems for mobile internet applications. IEEE Access 4:3273–3287. https://doi.org/10.1109/access. 2016.2573314 24. Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37. https://doi.org/10.1109/mc.2009.263 25. Ziegler C-N, McNee S, Konstan J, Lausen G (2005) Improving recommendation lists through topic diversification. In: Proceedings of the 14th international World Wide Web conference. ACM Press, London 26. Alharthi H, Inkpen D, Szpakowicz S (2017) A survey of book recommender systems. J Intell Inf Syst 51(1):139–160. https://doi.org/10.1007/s10844-017-0489-9 27. HahslerM(2021)recommenderlab: lab fordevelopingandtestingrecommenderalgorithms.Rpackage version 0.2-7. https://github.com/mhahsler/recommenderlab 28. HerlockerJL,KonstanJA, TerveenLG,RiedlJT(2004)Evaluatingcollaborativefilteringrecommender systems. ACM Trans Inf Syst 22(1):5–53. https://doi.org/10.1145/963770.963772 29. Lika B, Kolomvatsos K, Hadjiefthymiades S (2014) Facing the cold start problem in recommender systems. Expert Syst Appl 41(4 Part 2):2065–2073. https://doi.org/10.1016/j.eswa.2013.09.005 30. Breese JS, Heckerman D, Kadie C (1998) Empirical analysis of predictive algorithms for collaborative filtering. In: Proceedings of the fourteenth conference on uncertainty in artificial intelligence. Morgan Kaufmann, pp 43–52 31. Wenga C, Fansi M, Chabrier S, Mari J-M, Gabillon A (2021) A comprehensive review on non-neural networkscollaborativefilteringrecommendationsystems.In:arXivpreprintarXiv:2106.10679.https:// doi.org/10.48550/arXiv.2106.10679 32. Beel J, Langer S (2015) A comparison of offline evaluations, online evaluations, and user studies in the context of research-paper recommender systems. In: Kapidakis S, Mazurek C, Werla M (eds) Proceedings of the 19th international conference on theory and practice of digital libraries. Lecture Notes in Computer Science, vol 9316, pp 153–168 33. Bradley K (2001) Improving recommendation diversity. In: Proceedings of the AICS ’01 34. Bobadilla J, Ortega F, Hernando A, Gutiérrez A (2013) Recommender systems survey. Knowl Based Syst 46:109–132. https://doi.org/10.1016/j.knosys.2013.03.012 35. James G, Witten D, Hastie T, Tibshirani R (2013) An introduction to statistical learning. Springer, New York 36. Desrosiers C, Karypis G (2010) A comprehensive survey of neighborhood-based recommendation methods. In: Recommender systems handbook. Springer, Berlin, pp 107–144 37. Herlocker J, Konstan JA, Riedl J (2002) An empirical analysis of design choices in neighborhoodbased collaborative filtering algorithms. Inf Retrieval 5(4):287–310. https://doi.org/10.1023/A: 1020443909834 38. Sarwar B, Karypis G, Konstan J, Reidl J (2001) Item-based collaborative filtering recommendation algorithms. In: Proceedings of the tenth international conference on World Wide Web—WWW ’01. ACM Press, London 39. Ng A, Soo K (2017) Numsense! Data science for the Layman: no math added. Lightning Source Inc 40. Funk S (2006) Netflix update: try this at home. https://sifter.org/~simon/journal/20061211.html 41. BookCrossing, About BookCrossing. (2021). https://www.bookcrossing.com/about 123 Annals of Data Science (2024) 11(5):1705–1739 1739 42. Adamopoulos P, Tuzhilin A (2014) On unexpectedness in recommender systems: or how to better expect the unexpected. ACM Trans Intell Syst Technol 5/4:1–32. https://doi.org/10.1145/2559952 43. Park Y-J, Tuzhilin A (2008) The long tail of recommender systems and how to leverage it. In: Proceedings of the 2008 ACM conference on recommender systems. Association for Computing Machinery, pp 11–18 44. Deldjoo Y, Bellogin A, Di Noia T (2021) Explaining recommender systems fairness and accuracy through the lens of data characteristics. Inf Process Manag 58:5. https://doi.org/10.1016/j.ipm.2021. 102662 45. Zajac Z (2017) Goodbooks-10k: a new dataset for book recommendations. In: FastML 46. Wan M, McAuley JJ (2018) Item recommendation on monotonic behavior chains. In: Pera S, Ekstrand MD, Amatriain X, O’Donovan J (eds) Proceedings of the 12th ACM conference on recommender systems. Association for Computing Machinery, pp 86–94 47. Wan M, Misra R, Nakashole N, McAuley JJ (2019) Fine-grained spoiler detection from large-scale review corpora. In: Korhonen A, Traum DR, Màrquez L (eds) Proceedings of the 57th conference of the association for computational linguistics. Association for Computational Linguistics, pp 2605–2610 Publisher’s Note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations. 123