Research Article Recommender Systems: A Comprehensive Review of Models, Approaches and Evaluation Metrics
Article Sidebar
Main Article Content
Abstract
With the advent of the current digital era, individuals across the developed world are commonly equipped with devices that can access vast amounts of information at their fingertips. What was considered an impossible feat was realized through remarkable technological advancements. This positive transformation has had a profound impact on education, where traditional knowledge management, such as libraries, are no longer a primary determinant of a student’s academic success. Instead, it has been replaced by the internet as a medium for learning, practicing, and topic exploration. However, the sheer volume of the ever-increasing information available online can easily overwhelm a user, particularly when conducting detailed research on a specific topic. Therefore, the need for a reliable research article recommender system cannot be understated, helping students and researchers to navigate the expansive knowledge space better and achieve their learning and research objectives. This review paper aims to study the most common types of recommendation system techniques in research articles recommender systems (RS). A total of ten related works and relevant evaluation metrics written by other researchers will be studied and accessed rigorously using comparative analysis, granting further insights into the current work similar or related to the domain of this paper. Finally, this paper will identify and elaborate their current trends and gaps in the discussion section.
Article Details
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
All articles published in JIWE are licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0) License. Readers are allowed to
- Share — copy and redistribute the material in any medium or format under the following conditions:
- Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use;
- NonCommercial — You may not use the material for commercial purposes;
- NoDerivatives — If you remix, transform, or build upon the material, you may not distribute the modified material.
References
G. Foley, “How information overload is killing your progress and what to do about it,” Medium, Jan. 30, 2024. [Online]. Available: https://medium.com/@gregzfoley/how-information-overload-is-killing-your-progress-and-what-to-do-about-it-9c28848871c6
I. Bouchrika, “Overcoming Information Overload in Higher Education: The Power of Document Summarization,” Research.com, Sep. 26, 2024.
S. Raza et al., “A Comprehensive Review of Recommender Systems: Transitioning from Theory to Practice,” 2024, arXiv preprint arXiv:2407.13699.
Z. Fayyaz, M. Ebrahimian, D. Nawara, A. Ibrahim, and R. Kashef, “Recommendation systems: Algorithms, challenges, metrics, and business opportunities,” Applied Sciences (Switzerland), vol. 10, no. 21, pp. 1–20, 2020, doi: 10.3390/app10217748.
S. Apathy, “History of recommender systems: overview of information filtering solutions,” Onespire - SAP and IT Services, Sep. 12, 2023. [Online]. Available: https://onespire.net/history-of-recommender-systems/.
D. Roy, and M. Dutta, “A systematic review and research perspective on recommender systems,” J. Big Data, vol. 9, no. 1, 2022, doi: 10.1186/s40537-022-00592-5.
M. Nura, and Z.A. Hamisu, “Author-Centric Scientific Paper Recommender System to Improve Content-Based Filtering Approach,” Int. J. Softw. Eng. Comput. Syst., vol. 10, no. 1, pp. 40–49, 2024, doi: 10.15282/ijsecs.10.1.2024.4.0122.
D. Curcic, “Number of Academic Papers Published Per Year – WordsRated,” Wordsrated, Jun. 01, 2023. [Online]. Available: https://wordsrated.com/number-of-academic-papers-published-per-year/.
R. Habib, and M.T. Afzal, “Sections-based bibliographic coupling for research paper recommendation,” Scientometrics, vol. 119, no. 2, pp. 643–656, 2019, doi: 10.1007/s11192-019-03053-8.
T. Dai, L. Zhu, Y. Wang, H. Zhang, X. Cai, and Y. Zheng, “Joint model feature regression and topic learning for global citation recommendation,” IEEE Access, vol. 7, pp. 1706–1720, 2019, doi: 10.1109/ACCESS.2018.2884981.
Ministry of Higher Education Malaysia, Malaysia Education Blueprint 2015–2025 (Higher Education), 2015. [Online]. Available: https://www.mohe.gov.my/en/download/publications-journals-and-reports/pppm-2015-2025-pt/102-malaysia-education-blueprint-2015-2025-higher-education/file.
Ministry of Higher Education Malaysia, Statistik Pendidikan Tinggi 2024: Bab 1 – Makro, 2024. [Online]. Available: https://www.mohe.gov.my/muat-turun/statistik/2024-4/1702-bab-1-makro-2024-update-pdf/file.
C.K. Kreutz, and R. Schenkel, “Scientific paper recommendation systems: a literature review of recent publications,” Int. J. Digit. Libr., vol. 23, no. 4, pp. 335–369, 2022, doi: 10.1007/s00799-022-00339-w.
Z. Ali, G. Qi, K. Muhammad, B. Ali, and W.A. Abro, “Paper recommendation based on heterogeneous network embedding,” Knowledge-Based Systems, vol. 210, 2020, doi: 10.1016/j.knosys.2020.106438.
N. Sakib et al., “A hybrid personalized scientific paper recommendation approach integrating public contextual metadata,” IEEE Access, vol. 9, pp. 83080–83091, 2021, doi: 10.1109/ACCESS.2021.3086964.
Y.-K. Ng, “CBRec: a book recommendation system for children using the matrix factorisation and content-based filtering approaches,” Int. J. Business Intell. Data Min., vol. 16, no. 2, 2020, doi: https://doi.org/10.1504/ijbidm.2020.104738.
N.W. Rahayu, R. Ferdiana, and S.S. Kusumawardani, “A systematic review of ontology use in E-Learning recommender system,” Computers and Education: Artificial Intelligence, vol. 3, 2022, doi: 10.1016/j.caeai.2022.100047.
“A better way to make the recommendations that power popular platforms,” Stanford Graduate School of Business, Sep. 4, 2024. [Online]. Available: https://www.gsb.stanford.edu/insights/better-way-make-recommendations-power-popular-platforms#:~:text=YouTube%20has%20attributed%2070%25%20of,to%2080%25%20of%20content%20consumption.
T. Zhong, Z. Wen, F. Zhou, G. Trajcevski, and K. Zhang, “Session-based recommendation via flow-based deep generative networks and Bayesian inference,” Neurocomputing, vol. 391, pp. 129–141, 2020, doi: 10.1016/j.neucom.2020.01.096.
Z. Li, and X. Zou, “A Review on Personalized Academic Paper Recommendation,” Comput. Inf. Sci., vol. 12, no. 1, p. 33, 2019, doi: 10.5539/cis.v12n1p33.
S. Lin, G. Lee, and S.-L. Peng, “Academic article recommendation by considering the research field trajectory,” in Proc. Int. Conf. Artificial Intelligence and Soft Computing, pp. 447–454, 2021, doi: 10.1007/978-3-030-65407-8_39.
A. Shahid et al., “Insights into relevant knowledge extraction techniques: a comprehensive review,” J. Supercomput., vol. 76, no. 3, pp. 1695–1733, 2020, doi: 10.1007/s11227-019-03009-y.
btd, “Serendipity: A new dimension in recommender systems,” Medium, Nov. 16, 2023. [Online]. Available: https://baotramduong.medium.com/recommender-system-serendipity-c40c052f8199.
S. Ahmad, and M.T. Afzal, “Combining metadata and co-citations for recommending related papers,” Turkish Journal of Electrical Engineering & Computer Sciences, vol. 28, no. 3, pp. 1519–1534, 2020, doi: 10.3906/elk-1908-19.
M.P. Geetha, and D.K. Renuka, “Research on recommendation systems using deep learning models,” International Journal of Recent Technology and Engineering (IJRTE), vol. 8, no. 4, pp. 10544–10551, 2019, doi: 10.35940/ijrte.D4609.118419.
S. Malik, A. Rana, and M. Bansal, “A survey of recommendation systems,” Inf. Resour. Manage. J., vol. 33, no. 4, pp. 53–73, 2020, doi: 10.4018/IRMJ.2020100104.
W.-E. Kong, S.-C. Haw, N. Palanichamy, and S. H. A. Rahman, “An e-learning recommendation system framework,” Int. J. Adv. Sci. Eng. Inf. Technol., vol. 14, no. 1, pp. 10–19, 2024, doi: 10.18517/ijaseit.14.1.19043.
S.M. Al-Ghuribi, and S.A. Mohd Noah, “Multi-criteria review-based recommender system–the state of the art,” IEEE Access, vol. 7, pp. 169446–169468, 2019, doi: 10.1109/ACCESS.2019.2954861.
L. Chen, G. Chen, and F. Wang, “Recommender systems based on user reviews: the state of the art,” User Modeling and User-Adapted Interaction, vol. 25, no. 2, pp. 99–154, 2015, doi: 10.1007/s11257-015-9155-5.
R.D’Addio, M. Conrado, S. Resende, and M. Manzato, “Generating recommendations based on robust term extraction from users’ reviews,” in Proc. 20th Brazilian Symposium on Multimedia and the Web, 2014, pp. 55–58, doi: 10.1145/2664551.2664583.
C. Yang, X. Yu, Y. Liu, Y. Nie, and Y. Wang, “Collaborative filtering with weighted opinion aspects,” Neurocomputing, vol. 210, pp. 185–196, 2016, doi: 10.1016/j.neucom.2015.12.136.
P. Resnick, N. Iacovou, M. Suchak, P. Bergstrom, and J. Riedl, “GroupLens,” in Proc. 1994 ACM Conf. Comput. Support. Coop. Work (CSCW ’94), pp. 175–186, 1994, doi: 10.1145/192844.192905.
F. Hdioud, B. Frikh, and B. Ouhbi, “Multi-criteria recommender systems based on multi-attribute decision making,” in Proc. Int. Conf. Information Integration and Web-Based Applications & Services, pp. 203–210, 2013, doi: 10.1145/2539150.2539176.
K. Pachauri, “Semantic similarity for recommender system,” Medium, Dec. 15, 2021. [Online]. Available: https://medium.com/analytics-vidhya/semantic-similarity-for-recommender-system-d72c58dfe686.
H. Zhou, F. Xiong, and H. Chen, “A comprehensive survey of recommender systems based on deep learning,” Appl. Sci., vol. 13, no. 20, p. 11378, 2023, doi: 10.3390/app132011378.
“word_embeddings” [Online]. Available: https://web.engr.oregonstate.edu/~huanlian/teaching/ML/2024fall/unit4/word_embeddings.html.
T.R. Gruber, “A translation approach to portable ontology specifications,” Knowledge Acquisition, vol. 5, no. 2, pp. 199–220, 1993, doi: 10.1006/knac.1993.1008.
J. K. Tarus, Z. Niu, and G. Mustafa, “Knowledge-based recommendation: a review of ontology-based recommender systems for e-learning,” Arti6f. Intell. Rev., vol. 50, no. 1, pp. 21–48, 2018, doi: 10.1007/s10462-017-9539-5.
M. A. Paredes-Valverde, M. A. Rodriguez-Garcia, A. Ruiz-Martinez, R. Valencia-Garcia, and G. Alor-Hernandez, “ONLI: An ontology-based system for querying DBpedia using natural language paradigm,” Expert Syst. Appl., vol. 42, no. 12, pp. 5163–5176, 2015, doi: 10.1016/j.eswa.2015.02.034.
M. Harrathi, N. Touzani, and R. Braham, “A hybrid knowledge-based approach for recommending massive learning activities,” in Proc. 2017 IEEE/ACS 14th Int. Conf. Computer Systems and Applications (AICCSA), pp. 49–54, 2017, doi: 10.1109/AICCSA.2017.150.
G. George, and A. M. Lal, “Review of ontology-based recommender systems in e-learning,” Computers & Education, vol. 142, p. 103642, 2019, doi: 10.1016/j.compedu.2019.103642.
Xtreme1, “The 'ontology' in machine learning - multimodal data training,” Medium, Feb. 11, 2023. [online]. Available: https://medium.com/multisensory-data-training/the-ontology-in-machine-learning-e4ba59cd3fc4.
O.C. Santos, and J.G. Boticario, “Practical guidelines for designing and evaluating educationally oriented recommendations,” Computers & Education, vol. 81, pp. 354–374, 2015, doi: 10.1016/j.compedu.2014.10.008.
U.A. Bhatti, H. Tang, G. Wu, S. Marjan, and A. Hussain, “Deep learning with graph convolutional networks: An overview and latest applications in computational intelligence,” International Journal of Intelligent Systems, vol. 2023, pp. 1–28, Feb. 2023, doi: 10.1155/2023/8342104.
X. Wang, W. Lu, M. Ester, C. Wang, and C. Chen, “Social recommendation with strong and weak ties,” Proc. 25th ACM Int. Conf. Inf. Knowl. Manag., pp. 5–14, 2016, doi: 10.1145/2983323.2983701.
Q. Wang et al., “Learning domain-independent representations via shared weight auto-encoder for transfer learning in recommender systems,” IEEE Access, vol. 10, pp. 71961–71972, 2022, doi: 10.1109/ACCESS.2022.3188709.
L. Ye, H. Xie, Y. Lin, and J.C.S. Lui, “Rewarding social recommendation in OSNs: Empirical evidences, modeling and optimization,” IEEE Trans. Knowl. Data Eng., vol. 34, no. 9, pp. 4410–4424, 2022, doi: 10.1109/TKDE.2020.3038930.
A. Farmaki, H. Olya, and B. Taheri, “Unpacking the complex interactions among customers in online fan pages,” Journal of Business Research, vol. 125, pp. 164–176, 2021, doi: 10.1016/j.jbusres.2020.11.068.
R. van den Berg, T.N. Kipf, and M. Welling, “Graph convolutional matrix completion,” arXiv preprint, 2017. Available: http://arxiv.org/abs/1706.02263.
X. Wang, X. He, M. Wang, F. Feng, and T.-S. Chua, “Neural graph collaborative filtering,” Proc. 42nd Int. ACM SIGIR Conf. Res. Dev. Inf. Retr., pp. 165–174, 2019, doi: 10.1145/3331184.3331267.
X. Yang, H. Steck, and Y. Liu, “Circle-based recommendation in online social networks,” Proc. 18th ACM SIGKDD Int. Conf. Knowl. Discov. Data Min., pp. 1267–1275, 2012, doi: 10.1145/2339530.2339728.
M. O. Ayemowa, R. Ibrahim, and M. M. Khan, “Analysis of recommender system using generative artificial intelligence: A systematic literature review,” IEEE Access, vol. 12, pp. 87742–87766, 2024, doi: 10.1109/ACCESS.2024.3416962.
S. Feuerriegel, J. Hartmann, C. Janiesch, and P. Zschech, “Generative AI,” Business & Information Systems Engineering, vol. 66, no. 1, pp. 111–126, 2024, doi: 10.1007/s12599-023-00834-7.
I.J. Goodfellow et al., “Generative adversarial networks,” arXiv preprint, 2014. [Online]. Available: http://arxiv.org/abs/1406.2661.
M. Gao et al., “Recommender systems based on generative adversarial networks: A problem-driven perspective,” Information Sciences, vol. 546, pp. 1166–1185, 2021, doi: 10.1016/j.ins.2020.09.013.
H. Chen, S. Wang, N. Jiang, Z. Li, N. Yan, and L. Shi, “Trust-aware generative adversarial network with recurrent neural network for recommender systems,” International Journal of Intelligent Systems, vol. 36, no. 2, pp. 778–795, 2021, doi: 10.1002/int.22320.
J. An and S. Cho, “Variational autoencoder based anomaly detection using reconstruction probability,” Special lecture on IE, vol. 2, no. 1, pp. 1-18, 2015.
K. Haruna, M. A. Ismail, A. Qazi, H. A. Kakudi, M. Hassan, S.A. Muaz, and H. Chiroma, “Research paper recommender system based on public contextual metadata,” Scientometrics, vol. 125, no. 1, pp. 101–114, 2020, doi: 10.1007/s11192-020-03642-y.
M. Kartheek, and G.P. Sajeev, “Building semantic based recommender system using knowledge graph embedding,” 2021 Sixth International Conference on Image Information Processing (ICIIP), pp. 25–29, 2021, doi: 10.1109/ICIIP53038.2021.9702632.
L.J. Chew, S.C. Haw, S. Subramaniam, and K.W. Ng, “A hybrid ontology-based recommender system utilizing data enrichment and SVD approaches,” Journal of System and Management Sciences, vol. 12, no. 5, pp. 139–154, 2022, doi: 10.33168/JSMS.2022.0509.
W. Liu, and Q. Li, “Collaborative filtering recommender algorithm based on ontology and singular value decomposition,” 2019 11th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC), pp. 134–137, 2019, doi: 10.1109/IHMSC.2019.10127.
A. Chaudhuri, M. Sarma, and D. Samanta, “SHARE: Designing multiple criteria-based personalized research paper recommendation system,” Information Sciences, vol. 617, pp. 41–64, 2022, doi: 10.1016/j.ins.2022.09.064.
J. Zhang, and L. Zhu, “Citation recommendation using semantic representation of cited papers’ relations and content,” Expert Systems with Applications, vol. 187, p. 115826, 2022, doi: 10.1016/j.eswa.2021.115826.
I. Pinedo, M. Larranaga, and A. Arruarte, “ArZiGo: A recommendation system for scientific articles,” Information Systems, vol. 122, pp. 102367, 2024, doi: 10.1016/j.is.2024.102367.
X. Xiao, J. Xu, J. Huang, C. Zhang, and X. Chen, “TCRec: A novel paper recommendation method based on ternary coauthor interaction,” Knowledge-Based Systems, vol. 280, pp. 111065, 2023, doi: 10.1016/j.knosys.2023.111065.
S.J. Gharibi, K. BagheriFard, H. Parvin, S. Nejatian, and S.H. Yaghoubyan, “Ontology-based recommender system: a deep learning approach,” Journal of Supercomputing, vol. 80, no. 9, pp. 12102–12122, 2024, doi: 10.1007/s11227-023-05874-0.
P. Bahrani, B. Minaei-Bidgoli, H. Parvin, M. Mirzarezaee, and A. Keshavarz, “A hybrid semantic recommender system enriched with an imputation method,” Multimedia Tools and Applications, vol. 83, no. 6, pp. 15985–16018, 2024, doi: 10.1007/s11042-023-15258-4.
M. Nura, and Z. Adamu Hamisu, “Author-centric scientific paper recommender system to improve content-based filtering approach,” International Journal of Software Engineering and Computer Systems, vol. 10, no. 1, pp. 40–49, 2024, doi: 10.15282/ijsecs.10.1.2024.4.0122.
Murrell, and Zhai, “Is this the ChatGPT moment for recommendation systems? | Shaped Blog.” [Online]. Available: https://www.shaped.ai/blog/is-this-the-chatgpt-moment-for-recommendation-systems