Usability of an Immersive Augmented Reality Based Telerehabilitation System with Haptics (ARTESH) for Synchronous Remote Musculoskeletal Examination


  • Aleks Borresen University of Texas Southwestern Medical Center
  • Cody Wolfe University of Texas Southwestern Medical Center
  • Chung-Kuang Lin University of Texas Southwestern Medical Center
  • Yuan Tian The University of Texas at Dallas
  • Suraj Raghuraman The University of Texas at Dallas
  • Klara Nahrstedt University of Illinois
  • Balakrishnan Prabhakaran The University of Texas at Dallas
  • Thiru Annaswamy University of Texas Southwestern Medical Center, VA North Texas Health Care System



Augmented-reality, Haptics, Telerehabilitation, Telemedicine


This study describes the features and utility of a novel augmented reality based telemedicine system with haptics that allows the sense of touch and direct physical examination during a synchronous immersive telemedicine consultation and physical examination. The system employs novel engineering features: (a) a new force enhancement algorithm to improve force rendering and overcoming the “just-noticeable-difference” limitation; (b) an improved force compensation method to reduce the delay in force rendering; (c) use of the “haptic interface point” to reduce disparity between the visual and haptic data; and (d) implementation of efficient algorithms to process, compress, decompress, transmit and render 3-D tele-immersion data. A qualitative pilot study (n=20) evaluated the usability of the system. Users rated the system on a 26-question survey using a seven-point Likert scale, with percent agreement calculated from the total users who agreed with a given statement. Survey questions fell into three main categories: (1) ease and simplicity of use, (2) quality of experience, and (3) comparison to in-person evaluation. Average percent agreements between the telemedicine and in-person evaluation were highest for ease and simplicity of use (86%) and quality of experience (85%), followed by comparison to in-person evaluation (58%). Eighty-nine percent (89%) of respondents expressed satisfaction with the overall quality of experience. Results suggest that the system was effective at conveying audio-visual and touch data in real-time across 20.3 miles, and warrants further development.




Agostini, M., Moja, L., Banzi, R., Pistotti, V., Tonin, P., Venneri, A., & Turolla, A. (2015). Telerehabilitation and recovery of motor function: A systematic review and meta-analysis. Journal of Telemedicine and Telecare, 21, 202-213. doi: 10.1177/1357633x15572201

Amatya, B., Galea, M. P., Kesselring, J., & Khan, F. (2015). Effectiveness of telerehabilitation interventions in persons with multiple sclerosis: A systematic review. Multiple Sclerois and Related Disorders, 4, 358-369. doi: 10.1016/j.msard.2015.06.011

Aukstakalnis, S., Blatner, D., & Roth, S. (1992). Silicon Mirage: The Art and Science of Virtual Reality. Berkeley, CA: Peachpit Press.

Burdea, G. (2003). Virtual rehabilitation - benefits and

challenges. Yearbook of Medical Informatics, 1, 170-176.

Chen, J., Jin, W., Zhang, X. X., Xu, W., Liu, X. N., & Ren, C. C. (2015). Telerehabilitation approaches for stroke patients: Systematic review and meta-analysis of randomized controlled trials. Journal of Stroke & Cerebrovascular Diseases, 24, 2660-2668. doi: 10.1016/j.jstrokecerebrovasdis.2015.09.014

Cottrell, M. A., Galea, O. A., O'Leary, S. P., Hill, A. J., & Russell, T. G. (2017). Real-time telerehabilitation for the treatment of musculoskeletal conditions is effective and comparable to standard practice: A systematic review and meta-analysis. Clinical Rehabilitation, 31, 625-638. doi: 10.1177/0269215516645148

Holden, M. K. (2005). Virtual environments for motor rehabilitation: Review. Cyberpsychology Behavior, 8, 187-211; discussion 212-189. doi: 10.1089/cpb.2005.8.187

Howard, M. C. (2017). A meta-analysis and systematic literature review of virtual reality rehabilitation programs. Computers in Human Behavior, 70, 317-327. doi:

Iruthayarajah, J., McIntyre, A., Cotoi, A., Macaluso, S., & Teasell, R. (2017). The use of virtual reality for balance among individuals with chronic stroke: A systematic review and meta-analysis. Topics in Stroke Rehabilitation, 24, 68-79. doi: 10.1080/10749357.2016.1192361

Koceski, S., & Koceska, N. (2016). Evaluation of an assistive telepresence robot for elderly healthcare. Journal of Medical Systems, 40, 121. doi: 10.1007/s10916-016-0481-x

Larson, E. B., Feigon, M., Gagliardo, P., & Dvorkin, A. Y. (2014). Virtual reality and cognitive rehabilitation: A review of current outcome research. NeuroRehabilitation, 34, 759-772. doi: 10.3233/nre-141078

Lundell, S., Holmner, A., Rehn, B., Nyberg, A., & Wadell, K. (2015). Telehealthcare in COPD: A systematic review and meta-analysis on physical outcomes and dyspnea. Respiratory Medicine, 109(1), 11-26. doi: 10.1016/j.rmed.2014.10.008

O'Carroll, C. B., Hentz, J. G., Aguilar, M. I., & Demaerschalk, B. M. (2015). Robotic telepresence versus standardly supervised stroke alert team assessments. Telemedicine Journal and E-Health, 21, 151-156. doi: 10.1089/tmj.2014.0064

Parmanto, B., Lewis, A. N., Jr., Graham, K. M., & Bertolet, M. H. (2016). Development of the Telehealth Usability Questionnaire (TUQ). International Journal of Telerehabilitation, 8(1), 3-10. doi: 10.5195/ijt.2016.6196

Parmanto, B., Saptono, A., Pramana, G., Pulantara, W., Schein, R. M., Schmeler, M. R., . . . Brienza, D. M. (2010). VISYTER: Versatile and integrated system for telerehabilitation. Telemedicine Journal and E-Health, 16, 939-944. doi: 10.1089/tmj.2010.0033

Piggott, L., Wagner, S., & Ziat, M. (2016). Haptic neurorehabilitation and virtual reality for upper limb paralysis: A review. Critical Reviews in Biomedical Engineering, 44(1-2), 1-32. doi: 10.1615/CritRevBiomedEng.2016016046

Rizzo, A. (1997). Virtual reality and cognitive rehabilitation: A brief review of the future. The Journal of Head Trauma Rehabilitation, 12, 1-15.

Schutte, J., Gales, S., Filippone, A., Saptono, A., Parmanto, B., & McCue, M. (2012). Evaluation of a telerehabilitation system for community-based rehabilitation. International Journal of Telerehabilitation, 4(1), 15-24. doi: 10.5195/ijt.2012.6092

Segrelles Calvo, G., Gomez-Suarez, C., Soriano, J. B., Zamora, E., Gonzalez-Gamarra, A., Gonzalez-Bejar, M., . . . Ancochea, J. (2014). A home telehealth program for patients with severe COPD: the PROMETE study. Respiratory Medicine, 108, 453-462. doi: 10.1016/j.rmed.2013.12.003

Shull, P. B., & Damian, D. D. (2015). Haptic wearables as sensory replacement, sensory augmentation and trainer – a review. Journal of NeuroEngineering and Rehabilitation, 12(1), 59. doi: 10.1186/s12984-015-0055-z

Srinivasan, M. A., & Basdogan, C. (1997). Haptics in virtual environments: Taxonomy, research status, and challenges. Computers & Graphics, 21, 393-404. doi:

Tian, Y., Raghuraman, S., Annaswamy, T., Borresen, A., Nahrstedt, K., & Prabhakaran, B. (2017). H-TIME: Haptic-enabled Tele-Immersive Musculoskeletal Examination. Paper presented at the Proceedings of the 25th ACM international conference on Multimedia, Mountain View, California, USA.

van Delden, A. L., Peper, C. L., Kwakkel, G., & Beek, P. J. (2012). A systematic review of bilateral upper limb training devices for poststroke rehabilitation. Stroke Research and Treatment, 2012, 972069. doi: 10.1155/2012/972069

Venkatraman, K., Raghuraman, S., Tian, Y., Prabhakaran, B., Nahrstedt, K., & Annaswamy, T. (2014, 10-12 Dec. 2014). Quantifying and Improving User Quality of Experience in Immersive Tele-Rehabilitation. Paper presented at the 2014 IEEE International Symposium on Multimedia.



How to Cite

Borresen, A., Wolfe, C., Lin, C.-K., Tian, Y., Raghuraman, S., Nahrstedt, K., Prabhakaran, B., & Annaswamy, T. (2019). Usability of an Immersive Augmented Reality Based Telerehabilitation System with Haptics (ARTESH) for Synchronous Remote Musculoskeletal Examination. International Journal of Telerehabilitation, 11(1), 23–32.