A Hybrid RoBERTa–Rule-Based Aspect-Based Sentiment Analysis Model for Hospital Patient Reviews: Development, Validation, and Case Application in India

Authors

Akash Gajanan Prabhune, ADMIRE Centre for Advancing Digital Health, Institute of Health Management Research BangaloreFollow
Karpaga Priya P S, ADMIRE Centre for Advancing Digital Health, Institute of Health Management Research Bangalore
Vinay R Sri Hari, ADMIRE Centre for Advancing Digital Health, Institute of Health Management Research Bangalore
Akhila Rayampalli, ADMIRE Centre for Advancing Digital Health, Institute of Health Management Research Bangalore
Ankur S Thakur, ADMIRE Centre for Advancing Digital Health, Institute of Health Management Research Bangalore
Ebin M Yohannan, ADMIRE Centre for Advancing Digital Health, Institute of Health Management Research Bangalore

Abstract

Background: Patient Online Reviews (PORs) provide real-time, unsolicited feedback that complements traditional satisfaction surveys. However, existing sentiment analysis tools often handle only monolingual data, broad sentiment categories, or lack review authenticity checks. This study develops a hybrid Aspect-Based Sentiment Analysis (ABSA) model combining RoBERTa with a rule-based aspect detection layer to extract fine-grained sentiment insights from Indian hospital reviews.

Methods: We curated and manually annotated 4,862 Google Maps reviews from 25 Indian hospitals across six aspects—Doctor Behavior, Staff Behavior, Cleanliness, Facility Experience, Cost, and Waiting Time. Though the scheme included positive/negative/neutral labels, the final model used binary polarity (positive vs. negative) for reliability given sparse neutral data. The hybrid pipeline integrated RoBERTa fine-tuning with a keyword dictionary to enhance recall on low-frequency or implicit mentions. Review authenticity was verified using Type-Token Ratio, Jaccard similarity, and bigram overlap. External validation used 670 independent reviews.

Results: The model achieved an average F1 score of 0.923 and 92.67% accuracy. Sensitivity and specificity averaged 0.905, with PPV 94.27% and NPV 87.82%. Comparative testing showed superior performance over lexicon-based and monolingual transformer models. A case study revealed strong satisfaction with clinical care but concerns over waiting time.

Conclusions: The hybrid ABSA framework converts unstructured patient feedback into structured, actionable insights for hospital dashboards, audits, and digital health systems, supporting quality improvement in resource-limited, multilingual settings.

Keywords: Sentiment analysis, Healthcare, Patient experience, Hybrid model, India

Recommended Citation

Prabhune A, P S K, Hari V, Rayampalli A, Thakur A, Yohannan E. A Hybrid RoBERTa–Rule-Based Aspect-Based Sentiment Analysis Model for Hospital Patient Reviews: Development, Validation, and Case Application in India. J Health Res. 2025; 39(6):-. DOI: https://doi.org/10.56808/2586-940X.1170

References

[1] Greaves F, Ramirez-Cano D, Millett C, Darzi A, Donaldson L. Use of sentiment analysis for capturing patient experience from free-text comments posted online. J Med Internet Res 2013;15(11):e239. https://doi.org/10.2196/jmir.2721.

[2] Ranard BL, Werner RM, Antanavicius T, Schwartz HA, Smith RJ, Meisel ZF, et al. Yelp reviews of hospital care can supplement and inform traditional surveys of the patient experience of care. Health Aff (Millwood) 2016;35(4): 697—705. https://doi.org/10.1377/hlthaff.2015.1030.

[3] Woskie L, Kalita A, Bose B, Chakraborty A, Gupta K, Yip W. Patient satisfaction and value based purchasing in hospitals, Odisha, India. Bull World Health Organ 2024;102(7):509—20. https://doi.org/10.2471/BLT.24.290519.

[4] Doing-Harris K, Mowery DL, Daniels C, Chapman WW, Conway M. Understanding patient satisfaction with received healthcare services: A natural language processing approach. AMIA Annu Symp Proc 2016;2016:524—33.

[5] Schouten K, Frasincar F. Survey on aspect-level sentiment analysis. IEEE Trans Knowl Data Eng 2016;28(3):813—30. https://doi.org/10.1109/TKDE.2015.2485209.

[6] Medhat W, Hassan A, Korashy H. Sentiment analysis algorithms and applications: a survey. Ain Shams Eng J 2014;5 (4):1093—113. https://doi.org/10.1016/j.asej.2014.04.011.

[7] Lin CY. ROUGE: a package for automatic evaluation of summaries. In: Text summarization branches out. Barcelona, Spain: Association for Computational Linguistics; 2004. p. 74—81 [cited 2025 Aug 2]. Available from: https://aclanthology.org/W04-1013/.

[8] Chang IA, Wells MW, Chang IA, Arquette CP, Tang CJ, Gatherwright JR, et al. The positive patient experience: a comprehensive analysis of plastic surgery online reviews. Aesthet Surg J 2022;42(9):1083—93. https://doi.org/10.1093/asj/sjac092.

[9] Abbasi-Moghaddam MA, Zarei E, Bagherzadeh R, Dargahi H, Farrokhi P. Evaluation of service quality from patients' viewpoint. BMC Health Serv Res 2019;19(1):170. https://doi.org/10.1186/s12913-019-3998-0.

[10] Kamra V, Singh H, Kumar De K. Factors affecting patient satisfaction: an exploratory study for quality management in the health-care sector. Total Qual Manag Bus Excell 2016;27(9—10):1013—27. https://doi.org/10.1080/14783363.2015.1057488.

[11] Adhikary G, Shawon MSR, Ali MW, Shamsuzzaman M, Ahmed S, Shackelford KA, et al. Factors influencing patients' satisfaction at different levels of health facilities in Bangladesh: Results from patient exit interviews. PLoS One 2018;13(5):e0196643. https://doi.org/10.1371/journal.pone.0196643.

[12] Liu J, Zhao S, Wang G. SSEL-ADE: A semi-supervised ensemble learning framework for extracting adverse drug events from social media. Artif Intell Med 2018;84:34—49. https://doi.org/10.1016/j.artmed.2017.10.003.

[13] Zhao C, Feng R, Sun X, Shen L, Gao J, Wang Y. Enhancing aspect-based sentiment analysis with BERT-driven context generation and quality filtering. Nat Lang Process J 2024;7: 100077. https://doi.org/10.1016/j.nlp.2024.100077.

[14] Perikos I, Diamantopoulos A. Explainable aspect-based sentiment analysis using transformer models. Big Data Cogn Comput 2024;8(11):141. https://doi.org/10.3390/bdcc8110141.

[15] Covington MA, McFall JD. Cutting the gordian knot: the moving-average type—token ratio (MATTR). J Quant Linguist 2010;17(2):94—100. https://doi.org/10.1080/09296171003643098.

[16] Eka Diana N, Hanana Ulfa I. Measuring performance of N- gram and jaccard-similarity metrics in document plagiarism application. J Phys Conf Ser 2019;1196. https://doi.org/10.1088/1742-6596/1196/1/012069.

[17] Sharieff H, Sindhu T, SaiRamesh L. Comparison of machine learning techniques for sentimental analysis on restaurant reviews. Int J Adv Eng Manag 2020;2(6):740—3. https://doi.org/10.35629/5252-0206740743.

[18] Rahim AIA, Ibrahim MI, Chua SL, Musa KI. Hospital facebook reviews analysis using a machine learning sentiment analyzer and quality classifier. Healthcare (Basel) 2021;9(12). https://doi.org/10.3390/healthcare9121679.

[19] Li J, Yang Y, Chen R, Zheng D, Pang PC, Lam CK, et al. Identifying healthcare needs with patient experience reviews using ChatGPT. PLoS One 2025;20(3):e0313442. https://doi.org/10.1371/journal.pone.0313442.

[20] Pavithra A, Westbrook J. An assessment of organisational culture in Australian hospitals using employee online reviews. PLoS One 2022;17(9):e0274074. https://doi.org/10.1371/journal.pone.0274074.

[21] Li J, Pang PC, Xiao Y, Wong D. Changes in doctor-patient relationships in china during COVID-19: a text mining analysis. Int J Environ Res Public Health 2022;19(20):13446. https://doi.org/10.3390/ijerph192013446.

[22] Li J, Yang Y, Mao C, Pang PC, Zhu Q, Xu D, et al. Revealing patient dissatisfaction with health care resource allocation in multiple dimensions using large language models and the international classification of diseases 11th revision: aspect-based sentiment analysis. J Med Internet Res 2025; 27:e66344. https://doi.org/10.2196/66344.