Evaluation of Psychometric Properties of Thai Version Telehealth Usability Questionnaire (T-TUQ)

This cross-sectional validation study aimed to translate, cross-culturally adapt, and investigate the psychometric properties of a Thai version of the Telehealth Usability Questionnaire (T-TUQ). Two hundred and ten Thai participants, mean age of 61.2±15.2 years, were recruited from three specialty clinics: 50 (23.8%) hematology, 70 (33.3%) movement disorders, and 90 (42.9%) general neurology. The T-TUQ was translated from the original English version to produce a Thai language version. Back translation and pilot cognitive interviews were completed. All five subscales (usefulness, ease of use, effectiveness, reliability, and satisfaction) showed excellent internal consistency (alpha >0.80), displayed by Cronbach's alpha coefficient of 0.83, 0.94, 0.86, 0.83, and 0.92, respectively. For construct validity, exploratory factor analysis revealed two dimensions from eigenvalues and scree plot, defined as utility and accessibility subscales. In conclusion, the T-TUQ could be a reliable and valid instrument to evaluate the usability of telehealth with a Thai population.


International Journal of Telerehabilitation
Good to excellent content reliability from 53 participants, mainly Caucasians, of either patients or clinicians was reported for all constructs.Due to its reliability, acceptance and comprehensiveness, TUQ is considered an effective instrument to measure telehealth usability (Barsom et al., 2020;Hajesmaeel-Gohari & Bahaadinbeigy, 2021;Langbecker et al., 2017;Parmanto et al., 2016;Weaver et al., 2021).
Studies have reported the use of non-English language versions of the TUQ; however, the TUQ has never been translated or officially applied in Thailand (Bibiloni et al., 2020;Hvalič Touzery et al., 2020).The application of the original TUQ in Thailand would be limited because of the language barriers and cultural difference.Moreover, the objective of this study was to develop and validate a Thai version of the Telehealth Usability Questionnaire (T-TUQ) focusing on the patients' perspectives, and not including the clinicians' perspective.

Materials and Methods
This cross-sectional validation study was approved by the Institutional Review Board of the Faculty of Medicine, Chulalongkorn University (Certificate of approval number 1077/2021).The study was carried out in compliance with the Declaration of Helsinki.All participants were thoroughly informed about the study before providing a consent by action.Developers of the original TUQ gave their permission to translate the questionnaire into Thai without fee (Parmanto et al., 2016).

Participants
Inclusion criteria were native Thai speakers aged 18 years or older with at least one experience of a telehealth visit from the hematology, movement disorder, or general neurology clinics of King Chulalongkorn Memorial Hospital (KCMH), Thailand, and the ability to use the hospital's telehealth application either by themselves or with the help of their caregivers.Eligible participants were recruited between July 16, 2021 and November 10, 2022 within one month of their telehealth visit.Participants who refused to complete the questionnaire were excluded.
A total of 210 Thai participants, mean age of 61.2±15.2years, were recruited from three specialty clinics: 50 (23.8%)participants with hematologic conditions from hematology, 70 (33.3%)dystonia participants from movement disorders, and 90 (42.9%) participants with other neurological conditions such as facial dystonia, stroke, or seizure from general neurology.Nearly half of the participants completed the questionnaire on their own (n=95, 45.2%), while the remaining subjects were interviewed (n=115, 54.8%).

Instrument
Twenty-one items in the original TUQ were designed to comprehensively evaluate five usability factors of telehealth systems: usefulness, ease of use, effectiveness, reliability, and satisfaction.Usability is calculated by averaging the score of all items, rated by a 7-point Likert scale ranging from 1 (strongly disagree) to 7 (strongly agree).The TUQ was incorporated from three existing questionnaires: Telemedicine Satisfaction Questionnaire (TSQ), Telemedicine Patient Questionnaire (TMPQ), and Telemedicine Satisfaction and Usefulness Questionnaire (TSUQ).These questionnaires were widely used with proper content validity, but only partially cover the usability factors.Data for the TUQ content reliability, displayed in Table 1, was collected from self-reporting 53 participants of either patients or clinicians.The ethnicity of the TUQ's participants were 42 (79.2%)Caucasian, 5 (9.5%)Asian, 3 (5.6%)African American, 2 (3.7%)Hispanic, and 1 (2%) other (Parmanto et al., 2016).In this study, the T-TUQ responses were collected either by interview or self-reported, according to the participant's preference.

Translation and Adaption Process
The original English version of the TUQ was translated into Thai in accordance with the guidelines for the process of cross-cultural adaptation of self-report measures provided by Beaton et al. (2000).Forward and backward translations were conducted by native Thai-speaking and native English-speaking translators, respectively.Discussions with an expert committee were made throughout the process for proper cross-cultural adaptation.The main discussions were about the proper Thai word for "system" and "productive" in the original version.The back-translated TUQ version was later compared with the original TUQ for linguistic and conceptual accuracy.A pilot face validation was then carried out with ten native Thaispeaking participants from the general neurology clinic.These pilot participants were not included in the study sample.No changes were made for the final version as the pilot test reported the T-TUQ to be fully comprehensible.The final T-TUQ is available in the Appendix.Comrey (1988) and Guadagnoli and Velicer (1988) suggested a sample size of 200 to 300 for factor analysis.Conforming to the rule of thumb for questionnaire psychometric properties testing, one question required 10 samples (Nunnally, 1978).Considering the 21 items in the TUQ, the targeted sample size was 210 with no drop-out expected.

Statistical Analysis
All analyses were performed using Microsoft Excel and SPSS for Windows (version 28.0;IBM Corp) without imputation of missing data.

Descriptive Analysis
Continuous variables were displayed by mean with standard deviation (SD).Frequency with percentages were used to present categorical variables.

Reliability
Internal consistency within each subscale was determined by Cronbach's alpha coefficient.An alpha of at least 0.8 was considered as good consistency (Portney & Watkins, 2009).

Validity
Content validity remained from the original version due to adherence to the previously mentioned translation and adaptation process.Construct validity was assessed by exploratory factor analysis using principal axis factoring (PAF) with Oblimin and Kaiser normalization rotation methods.The factorability of the sample matrix was measured by the Kaiser-Meyer-Olkin (KMO) test requiring a minimum acceptable value of 0.6 (Kaiser,1974).Bartlett's test of sphericity was applied to assess the correlation matrix.

Results
From 210 T-TUQ responses of Thai participants with neurological or hematological disorders, the average summary score of Thai version TUQ was 6.5 (SD 0.95) out of 7. High mean usability scores were reported for all five subscales (Table 1).

Reliability
In terms of comparison with the original TUQ with five subscales, the T-TUQ (usefulness, ease of use, effectiveness, reliability, and satisfaction) had good to excellent internal consistency displayed by Cronbach's alpha of 0.83, 0.94, 0.86, 0.83, and 0.92, respectively (Table 1).

Constructed Validity
Since the KMO index was 0.91 and the Bartlett's test of sphericity was significant (< 0.001), this matrix of samples was deemed adequate for factor analysis.Exploratory factor analysis by PAF was done to determine the number of factors and the degree they were related.Rising the iteration to 9999 attempts to extract 5 factors as the original TUQ was terminated because the communality of a variable exceeded 1.0.While one factor extracted with eigenvalues >1 was responsible for 58.3% of variance, a two-factor structure explained a total of 63.9% of variance.Supported by the scree plot in Figure 1, twofactor structure was later evaluated by content.Nine items in the T-TUQ: items 4, 9, 16, 17, 5, 7, 10, 14, and 6, were loaded into factor 2, which can be defined as the telehealth accessibility subscale (Table 2).Twelve other items in factor 1 were defined as the utility subscale.High factor correlation of 0.76 was reported between two factors.Considering the two-factor constructs, internal consistency using Cronbach's alpha was 0.95 for both factors.

Discussion
The primary objective of this study was to translate, cross-culturally adapt, and assess the psychometric properties of a Thai version of the TUQ.The face validation process, along with the responses from 210 native Thai-speaking participants, confirmed that the T-TUQ was easily readable and exhibited content validity equivalent to the original English version.Furthermore, the T-TUQ demonstrated good to excellent reliability.
Before the onset of the COVID-19 pandemic, the TUQ gained recognition for its popularity, reliability, and comprehensiveness.With the global increase in telehealth services, the original questionnaire was translated into various languages.These translations extended to Spanish, Turkish, Slovene, Urdu, Portuguese, and Danish, aiming to capture user feedback on emerging telehealth technologies.In the context of Thailand, where there was no standard telehealth-related questionnaire in Thai, the development of a reliable T-TUQ would serve as an essential tool to enhance the implementation of telehealth services.
The internal consistencies of all five T-TUQ subscales displayed good to excellent reliability, aligning with reports from the original TUQ by Parmanto et al. (2016) (Table 1).A higher Cronbach's alpha of greater than 0.9 was reported in the Turkish version (n=107) (Özkeskin et al., 2021), but lower values of reliability (0.24) and usefulness (0.58) were reported from the Danish TUQ (n=34) (Bender et al., 2022).The Brazilian TUQ (n=64) also demonstrated excellent internal consistency (alpha=0.94)(Santos et al., 2023).Translations into Spanish and Slovene languages were conducted, but no reliability details have been reported.Our study, which included participants from hematology and neurology clinics, supports greater generalizability of the T-TUQ due to the range of diseases included.This study boasted the second-largest sample size, following the Urdu study that involved 350 participants.The analysis of construct validity demonstrated two-factor constructs and exhibited excellent internal consistency for both constructs.As the attempt to extract five factors as the original TUQ was terminated, this study data did not yet support five subscales.These results supported a reduction in the T-TUQ subscales from five to two.The reduced subscales were titled "Accessibility" and "Utility."In comparison to the original TUQ, the "Accessibility" subscale was composed of items from the "Usefulness," "Ease of use" (except item 8)."Effectiveness" (except items 13 and 14), while "Reliability" (except item 15) and "Satisfaction" were grouped under the "Utility" subscales (Table 3).However, further validation through confirmatory factor analysis should be conducted to support the T-TUQ.This reduction in factor constructs could be attributed to cultural and language differences.In the Thai language, items in factor 2 or "Utility" pertain more to feelings and inner self-benefit, whereas items in factor 1 are more related to the operating system and processes.Additionally, the high factor correlation of 0.76 between the two constructs and the scree plot in Figure 1 indicated that a one-factor structure could also be considered.
This study has limitations, including the restricted generalizability of the T-TUQ due to its exclusive focus on participants from hematology and neurology clinics at a university hospital in Bangkok.Moreover, test and re-test processes were not conducted, and the construct validation was carried out through exploratory, rather than confirmatory, factor analysis.
Future studies could expand participant characteristics to include individuals from other regions in Thailand with various diseases.Additionally, conducting test and re-test processes and confirmatory factor analysis would enhance the reliability and validity of the T-TUQ.It was easy to learn to use the system Accessibility 6 I believe I could become productive quickly using this system Accessibility 7 The way I interact with this system is pleasant Accessibility 8 I like using the system Utility 9 The system is simple and easy to understand Accessibility Effectiveness 10 This system is able to do everything I would want it to be able to do Accessibility 11 I could easily talk to the clinician using the telehealth system Accessibility 12 I could hear the clinician clearly using the telehealth system Accessibility 13 I felt I was able to express myself effectively Utility 14 Using the telehealth system, I could see the clinician as well as if we met in person Utility Reliability 15 I think the visits provided over the telehealth system are the same as in-person visits Accessibility 16 Whenever I made a mistake using the system, I could recover easily and quickly Utility 17 The system gave error messages that clearly told me how to fix problems Utility Satisfaction 18 I feel comfortable communicating with the clinician using the telehealth system Utility International Journal of Telerehabilitation

Conclusion
The development and validation of the Thai version of the Telehealth Usability Questionnaire (T-TUQ) represent a significant step toward evaluating telehealth services in Thailand.While the initial findings on its reliability and validity show promise, further research is needed to confirm its utility across diverse medical specialties and patient populations.The twofactor structure observed in the T-TUQ suggests potential cultural and language influences on user perceptions of telehealth.Future studies should expand the participant pool, conduct test-retest processes, and employ confirmatory factor analysis to enhance the questionnaire's reliability and validity.Ultimately, a validated T-TUQ holds great potential for improving telehealth implementation in Thailand, facilitating patient-centered care, and ensuring technology effectively serves its users.

Table 1 T
-TUQ Subscales Usability Score and Cronbach's Alpha Compared with Original TUQ Note.N = Valid cases per subscales for reliability analysis, SD = Standard deviation, T-TUQ = Thai version telehealth usability questionnaire, TUQ = Telehealth usability questionnaire