Functionality and end-user acceptability of the Internet-based Computerized Patient Assessment System (iComPAsS), a mobile symptom monitoring system
Introduction
The World Health Organization (WHO) has identified poor cancer symptom management as a global health problem, but under-treatment of cancer-related symptoms persists especially in the Philippines (1). Local data suggests that a significant number of patients needed consultation outside scheduled clinic visits (53%) and needed urgent care (31%) for symptom control (2). Critical components for the successful clinical management of pain include the patients’ ability to communicate their need for pain control, provide feedback on the effectiveness of treatment, and adherence to the requirements of therapy (2). Health information technology applications have been developed to overcome these difficulties. Several studies have demonstrated the practicality of administering symptom questionnaires to cancer patients through touch-tone telephones, mobile devices, or desktop computers (3-5) and have shown its influence in improving symptom control, especially in the US. A survey was done in our center to assess the readiness, acceptability and attitudes of cancer patients with regards to mobile health for symptom monitoring. The idea was received positively and the use of mobile phones was endorsed by 96% of the respondents for monitoring of symptoms. Further, 95% reported having access to the Internet from mobile devices, having necessary equipment (90%), and having knowledge (87%) to use the system (6).
Thus, our interdisciplinary group developed an internet-based computerized system for patient self-report of symptoms (iComPAsS), using a validated tool, Edmonton Symptom Assessment System (ESAS) (7), that shall allow the physician to view patient reports for easy and quick monitoring. In the Philippines, research is being conducted to gather data on the access and usage of information and communications technology (ICT), the perception of the utility of health-related ICT (6), and improvement in health outcomes, but has not been conducted to address healthcare among cancer patients. In addition, benefits of e-health applications may not translate in the local setting. The main objective of this study is to evaluate the usability of “IComPAsS” a mobile application for reporting cancer symptoms among patient and physician end-users.
Methods
Ethics statement
The study was approved by the University of Santo Tomas Hospital - International Review Board (No. IRB-MD-04-2015-054-A1), and all patients provided written informed consent.
Study design and location
This cross-sectional study was conducted at the University of Santo Tomas Hospital – Benavides Cancer Institute (USTH-BCI). The USTH-BCI is a tertiary care cancer center in Manila, Philippines that offers comprehensive cancer treatment for pediatric and adult cancer patients. In 2016, it registered more than 10,000 adult cancer cases.
Application development
Development of the mobile application
The mobile application was named iComPAsS which stands for “Internet-based Computerized Patient Assessment System.” The system employed the Edmonton Symptom Assessment System (ESAS) as the pilot tool (7) which was deemed face-valid after several focus group discussions (two oncologists, two pain specialists and an international symptom researcher). Application interface and system design was developed collaboratively with information technology consultants over several iterations and was in compliance with the Health Insurance Portability and Accountability Act (HIPPA) privacy and security policies. It requires a 1-time initialization with a unique patient identification number and password. The iComPAsS, a tool for patient self-reporting of pain and symptoms, allows patients to report the severity of their pain and symptoms through the Internet using a mobile phone, and physicians to view patient entries and send instructions back (e.g., to pick up a refill prescription or a revised prescription for purposes of dose titration, to follow-up in the clinic) and to generate graphical or statistical summaries (e.g., trend of pain over time from patient responses).
Features of the mobile application
The ‘Patient Interface’ is accessible from a mobile application, and is available in Android and iOS versions. The interface allows the patient to login, modify his username and password, access a menu, and log-out. The menu consists of the following options: Profile, Prescriptions, Take ESAS, List of Doctors, Messages, Change Password and Help (Figure 1). Profile allows the patient to view his demographic data and call the Pain Unit Hotline if data needs updating or correction; Prescriptions, to view physician instructions or prescriptions; Take ESAS, to access the Edmonton Symptom Assessment System (ESAS) which allows users to report 9 common symptoms in cancer (pain, nausea, vomiting, etc.). For each symptom, the severity is indicated using sliders from 0 (indicating no symptoms) to 10 (indicating worst possible symptom) as shown in Figure 2; List of Doctors to view a list of all his attending physicians, their specialties, schedules and office telephone numbers; Messages, to view instructions or messages from physicians and administrators; Change Password to change current password; and Help to access textual and a video tutorial on use, navigation and troubleshooting.
The Physician Interface allows the physician to login, modify his username and password, and access a menu, and log-out. The menu consists of the following options: Profile, List of Patients, Messages, Change Password, and Help. Profile allows the physician to view his profile (schedule, contact numbers, specialization) and to contact the administrator for necessary updates or corrections; List of Patients, to view a list of his patients, access their individual records (such as demographic data, clinical data, other attending physicians, and responses to ESAS or other tools, see Figure S1 for a sample of a completed patient report); Messages, to view messages that he has sent to patients or coming from administrators, and to send messages to patients, such as reminders, instructions, prescriptions, and appointment schedules; Change Password to change current password; and Help to access textual and a video tutorial on use, navigation and troubleshooting of the application.
Patient population and selection
The study included 20 participants which were comprised of 10 patients and 10 physicians. Patients were eligible to participate if they were cancer patients or care-givers of at least 18 years of age, could understand English, gave assent or consent to participate and were receiving or accompanying patients for treatment at the USTH-BCI. Affiliated physicians were invited if they were involved in cancer care; e.g., medical, surgical, radiation oncologists or palliative care specialists. Demographic data (age, gender, marital status, living set-up, employment status and educational attainment) were obtained from the patient and/or accompanying person/s.
Training
Participants attended a 2-hour training session conducted by the study investigators and app developers. Training for physicians and patients were done separately. Each participant was provided with a smartphone with the mobile app installed. Participants were allowed to use the application directly after a short demonstration and each were allowed sufficient time to go over through the app interface and menus. Each participant received a training booklet that included step-by-step references and facilitators were available throughout the session to guide the participants.
After a detailed exploration of the app’s content and functionalities, a training exercise for the participants was conducted for the MARS’ use. Since multiple raters were utilized, a 20-minute lecture using the training slides provided by the MARS authors was conducted as recommended (8).
Usability evaluation
Evaluation of the app’s usability was then done using the Mobile Application Rating Scale (MARS) (8). It is a validated 23-item simple and objective tool which provides a multidimensional, reliable, and flexible app-quality rating scale for researchers, developers, and health-professionals. The 23 Likert questions in MARS focused on 4 aspects of mobile applications—engagement, functionality, aesthetics and information.
For the physicians, the MARS was administered at the end of the training session. For the patients, the MARS was administered after a testing period of 7 days. During the 7-day period, the patients were instructed to use the app and to submit daily reports. Patients were given ample time to complete the MARS. All participants were interviewed regarding their experience, and other comments or suggestions for the mobile application.
Outcome measures
The main objective of this study is to evaluate the usability of “iComPAsS” a mobile application for reporting cancer symptoms among patient and physician end-users.
Statistical analysis
Demographic characteristics and MARS tool responses were encoded using MS Excel 2013, and analyzed using STATA 13. Descriptive statistics like mean and standard deviation, frequency and proportions were used to present quantitative and qualitative responses.
Results
Baseline characteristics
A total of 20 participants (10 patients and/or caregivers and 10 physicians) completed the training session and the MARS assessment tool. The demographic characteristics of the participants are summarized in Table 1. The mean age of the whole cohort was 39 years (range, 19–75 years). Majority of the patient participants were female, college graduates or higher, were employed, and were living with their family of origin. The physician participants were comprised of medical oncologists [3], radiation oncologists [3], surgical oncologists [2], and palliative care specialists [2].
Table 1
| Characteristic | Doctors (n=10) | Patients/caregivers (n=10) | Total (n=20) |
|---|---|---|---|
| Age, mean ± SD [range] | 39.8±11.55 [25–57] | 38.9±18.65 [19–75] | 39.35±15.10 [19–75] |
| Sex, n [%] | |||
| Male | 6 [60] | 3 [30] | 9 [45] |
| Female | 4 [40] | 7 [70] | 11 [55] |
| Marital status, n [%] | |||
| Never married | 5 [50] | 6 [60] | 11 [55] |
| Married | 5 [50] | 3 [30] | 8 [40] |
| Widow(er) | – | 1 [10] | 1 [5] |
| Religion, n [%] | |||
| Roman catholic | 9 [90] | 8 [80] | 19 [85] |
| Born again Christian | – | 1 [10] | 1 [5] |
| Church of Christ (Iglesia ni Cristo) | 1 [10] | – | 1 [5] |
| Others | – | 1 [10] | 1 [5] |
| Educational attainment, n [%] | |||
| Secondary school | – | 1 [10] | 1 [5] |
| College | – | 7 [70] | 7 [35] |
| Graduate school | 4 [40] | 2 [20] | 6 [30] |
| Doctorate | 5 [50] | – | 5 [25] |
| No answer | 1 [10] | – | 1 [5] |
| Occupation, n [%] | |||
| Employed | 5 [50] | 6 [60] | 11 [55] |
| Self-employed | 5 [50] | 2 [20] | 7 [35] |
| Housework | – | 1 [10] | 1 [5] |
| No answer | – | 1 [10] | 1 [5] |
| Living set-up, n [%] | |||
| Living alone | 1 [10] | 1 [10] | 2 [10] |
| With family of origin | 5 [50] | 7 [70] | 12 [60] |
| With partner with kids | 4 [40] | 1 [10] | 5 [25] |
| Living with kids | – | 1 [10] | 1 [5] |
The scores of the 4 main quality categories (engagement, functionality aesthetics and information) of the MARS tool is shown in Table 2. Scores are reported from 1 (least satisfactory) to 5 (most satisfactory) with a score of 3 signifying “acceptable”. The app was found highly satisfactory for all categories, scoring lowest for engagement (3.61±0.27) and highest for information (4.26±0.54). Both groups however scored engagement as the lowest category, with items under this section included: entertainment, interest, customization, interactivity and applicability to the target audience. Mean scores were lower in the patient group compared to the physician group (3.87±0.46 patients vs. 4.13±0.47 physicians).
Table 2
| Quality domain | Scores, mean ± SD (range) | ||
|---|---|---|---|
| Doctors (n=10) | Patients/caregivers (n=10) | Total (n=20) | |
| Engagement score | 3.70±0.56 (2.75–4.75) | 3.52±0.59 (2.60–4.60) | 3.61±0.27 (2.60–4.75) |
| Functionality score | 4.20±0.45 (3.50–4.75) | 3.98 +.66 (2.75–4.75) | 4.09±0.56 (2.75–4.75) |
| Aesthetics score | 4.17±0.57 (3.00–5.00) | 3.90±0.52 (3.00–4.67) | 4.03±0.55 (3.00–5.00) |
| Information score | 4.45±0.39 (3.75–5.00) | 4.07±0.63 (3.00–5.00) | 4.26±0.54 (3.00–5.00) |
| Over-all score | 4.13±0.47 (3.63–4.75) | 3.87±0.46 (3.26–4.55) | 3.99±0.43 (3.26–4.75) |
Majority of the participants (95%) would recommend and use the mobile app as shown in Table 3. From a 1–5 star rating, most scored the app as 4 stars (3.9 stars as mean score).
Table 3
| Review question | Doctors (n=10), n [%] | Patients/caregivers (n=10), n [%] | Total (n=20), n [%] |
|---|---|---|---|
| Would you recommend this app to people who might benefit from it? | |||
| 1. Not at all I would not recommend this app to anyone | – | – | – |
| 2. There are very few people I would recommend this app to | – | 1 [10] | 1 [5] |
| 3. There are several people I would recommend this app to | 1 [10] | 3 [30] | 4 [40] |
| 4. There are many people I would recommend this app to | 7 [70] | 3 [30] | 10 [50] |
| 5. I would recommend this app to everyone | 2 [20] | 3 [30] | 5 [25] |
| How many times do you think you would use this app in the next 12 months if it was relevant to you? | |||
| 1. 0 | – | 1 [10] | 1 [5] |
| 2. 1–2 times | 2 [20] | 2 [20] | 4 [20] |
| 3. 3–10 times | 4 [40] | 4 [40] | 8 [40] |
| 4. 11–50 times | 1 [10] | – | 1 [5] |
| 5. >50 times | 3 [30] | 3 [30] | 6 [30] |
| What is your overall (star) rating of the app? | |||
| One of the worst apps I’ve used | – | – | – |
| | – | – | – |
| Average | 1 [10] | 4 [40] | 5 [25] |
| | 7 [70] | 5 [50] | 12 [60] |
| One of the best apps I’ve used | 2 [20] | 1 [10] | 3 [15] |
The app’s perceived impact for patients and physicians is shown in Figures 3,4. Both patients and physicians mostly agree that the app can increase knowledge, awareness, promote change in attitude, increase motivation, and promote help-seeking and behavior change regarding cancer symptoms. Although none responded with a “disagree” or “strongly disagree”, more patients and/or caregivers were uncertain with the app’s impact (responding with “either”).
Discussion
This study showed that the iComPAsS was generally usable and acceptable by both physicians and patients. It was found satisfactory for all domains included in the MARS (engagement, functionality aesthetics and information).
Although several tools are widely available for evaluating mobile apps (9,10), we found the MARS most appropriate for several reasons: it was designed specifically to evaluate health applications, it included quality domains relevant for clinician and patient end-users, and lastly it was validated for use in non-health professionals (11). The MARS has been used to compare and contrast a wide array of mobile applications side by side, by a group of only two to three participants (12,13). In this study, all 20 respondents utilized the MARS to review one app, where it served as a guide to evaluate key quality domains. A potential utility of the MARS is to observe how the scores evolve as the app continues to be updated.
Participants rated “Information” as the highest category and “engagement” as the lowest category. Information pertained to the quality of the information from a credible source, whereas engagement pertained to how the app was fun, interesting, customizable, interactive and well-targeted to the audience. This was an anticipated finding, considering that functionality (over entertainment) was prioritized during the app’s conceptualization and development. It was also an intention to simplify the app interface to have it compatible with a wide variety of smartphone devices.
The participants responded positively to the app and found great interest and enthusiasm for its use. Patients have suggested that the app will allow them to report minor symptoms that are usually neglected for more concerning ones. Some patients have also praised the sliding scale, emphasizing that it is a helpful tool to measure the degree of severity of symptoms. Most of the patients found the app straightforward and easy to use. We found that despite some patients not having used a smartphone before, participants found it easy to use with minimal assistance. This suggests that a well-designed system can be made intuitive and easy to use regardless of previous computer experience (14).
The use of symptom diaries is valuable in cancer care since it allows clinicians to identify changes in the patients’ medical condition. It provides clinicians a realistic scenario of the patient’s situation outside of the hospital or clinics (15-17). Electronic diaries are gaining momentum as a means of symptom reporting as they have added benefits of faster completion, lesser recall bias, and instantaneous feedback (18,19). The advantages may even be more pronounced in the older and frailer subgroup (20).
Our ultimate goal is for our patients to receive help when it is most needed, preventing the onset of more severe problems. Web-based systems thus confer benefit because it intensifies symptom management in response to patient reports, it focuses clinic visits on important problems, and it engages patients as active participants, which may improve experience, efficiency and outcomes of care (21,22). In addition, timely management will translate to improved oncologic outcomes by catching problems early in order to necessitate prompt action. Hence, severe and potentially fatal events are prevented (21,23).
In our cohort of physicians, all had access to smartphones and the internet. Physicians mentioned that they supported an app that allowed them to check-up patient symptoms at their own convenience and agreed that and that it could enhance symptom management. Physicians also noted that the app can help speed up patient consults by allowing them to identify problems faster (24).
An interesting finding was that the MARS scores in most domains were lower in the patient group compared to the physician group. This may reflect greater perceived importance of the app for physicians rather than for patients (Figure 3,4). We assume that for patients, the app will gain relevance and importance after an extended monitoring period, especially when they encounter symptoms at home away from the clinic.
Criticisms regarding the app’s use were the need for a stable internet connection, and the security feature of requiring password entry every time the app is opened. Requiring a password on every access was reported to be a barrier to engagement in a similar study by Kenny and colleagues (25). Physicians have commented on the risk of over-reliance on the app for remote monitoring instead of personal clinic visits, and daily reporting may increase workload for the patients at home.
This study has several limitations. Our sample mostly consisted of educated and younger participants, who are assumed to adapt well with mobile technology. Also, the 2-hour training session may be inadequate for participants who are computer-inexperienced. Another problem encountered by some participants during the test period was connectivity issues, which prevented them from submitting and viewing reports.
To the investigators knowledge, this is the first time a mobile health application for cancer patients was evaluated in the Philippines. This study becomes relevant, since adopting new innovations require that it is first functional and usable in the local setting. The findings in this study however, can only be interpreted in the context of an urban based cancer center where access to mobile based technologies is readily available. Since the app was received positively by both patients and physicians, we recommend a larger trial to evaluate the impact of these innovations on oncologic outcomes such as survival or health related quality of life.
Conclusions
We conclude that the iComPAsS mobile application for patient self-reporting of cancer symptoms is useable and acceptable by both physician and patient end-users. We recommend that its impact be further evaluated in a clinical setting.
Acknowledgments
Mary Kristine A. Intia.
Funding: This project was supported by the ASCO Conquer Cancer Foundation Grant.
Footnote
Conflicts of Interest: All authors have completed the ICMJE uniform disclosure form (available at http://dx.doi.org/10.21037/jhmhp.2018.04.08). The authors have no conflicts of interest to declare.
Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. The study was conducted in accordance with the Declaration of Helsinki (as revised in 2013). The study was approved by the University of Santo Tomas Hospital - International Review Board (No. IRB-MD-04-2015-054-A1), and all patients provided written informed consent.
Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.
References
- Schmidt R. Cancer pain management: patient-related barriers. In: Schmidt R, Willis W. editors. Encyclopedic Reference on Pain. New York, NY: Springer Verlag; 2007.
- Que JC, Sy Ortin TT, Anderson KO, et al. Depressive Symptoms among Cancer Patients in a Philippine Tertiary Hospital: Prevalence, Factors, and Influence on Health-Related Quality of Life. J Palliat Med 2013;16:1280-4. [Crossref] [PubMed]
- Basch E, Artz D, Dulko D, et al. Patient online self-reporting of toxicity symptoms during chemotherapy. J Clin Oncol 2005;23:3552-61. [Crossref] [PubMed]
- Bielli E, Carminati F, La CS, et al. A Wireless Health Outcomes Monitoring System (WHOMS): development and field-testing with cancer patients using mobile phones. BMC Med Inform Decis Mak 2004;4:7. [Crossref] [PubMed]
- Hawkes AL, Hughes KL, Hutchison SD, et al. Feasibility of brief psychological distress screening by a community-based telephone helpline for cancer patients and carers. BMC Cancer 2010;10:14. [Crossref] [PubMed]
- Bacorro W, Gutierrez R, Escuadra C, et al. Internet-based Computerized Patient Assessment System (iComPAsS): a cross-sectional analysis of needs, acceptance and readiness among adult cancer patients in a tertiary care center. J Hosp Manag Health Policy 2018; In press. [Crossref]
- Bruera E, Kuehn N, Miller MJ, et al. The Edmonton Symptom Assessment System (ESAS): a simple method for the assessment of palliative care patients. J Palliat Care 1991;7:6-9. [PubMed]
- Stoyanov SR, Hides L, Kavanagh DJ, et al. Mobile App Rating Scale: A New Tool for Assessing the Quality of Health Mobile Apps. JMIR mHealth uHealth 2015;3:e27 [Crossref] [PubMed]
- Brooke J. SUS - A quick and dirty usability scale. Usability Eval Ind 1996;189:4-7.
- Lewis JR. IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use. Int J Hum Comput Interact 1995;7:57-78. [Crossref]
- Stoyanov SR, Hides L, Kavanagh DJ, et al. Development and Validation of the User Version of the Mobile Application Rating Scale (uMARS). JMIR mHealth uHealth 2016;4:e72 [Crossref] [PubMed]
- Chen J, Cade JE, Allman-Farinelli M. The Most Popular Smartphone Apps for Weight Loss: A Quality Assessment. JMIR mHealth uHealth 2015;3:e104 [Crossref] [PubMed]
- Mani M, Kavanagh DJ, Hides L, et al. Review and Evaluation of Mindfulness-Based iPhone Apps. JMIR mHealth uHealth 2015;3:e82 [Crossref] [PubMed]
- Hardinge M, Rutter H, Velardo C, et al. Using a mobile health application to support self-management in chronic obstructive pulmonary disease: a six-month cohort study. BMC Med Inform Decis Mak 2015;15:46. [Crossref] [PubMed]
- Wan Leung S, Lee TF, Chien CY, et al. Health-related quality of life in 640 head and neck cancer survivors after radiotherapy using EORTC QLQ-C30 and QLQ-H&N35 questionnaires. BMC Cancer 2011;11:128. [Crossref] [PubMed]
- Dickinson R, Hall S, Sinclair JE, et al. Using technology to deliver cancer follow-up: a systematic review. BMC Cancer 2014;14:311. [Crossref] [PubMed]
- Basch E, Deal AM, Kris MG, et al. Symptom Monitoring With Patient-Reported Outcomes During Routine Cancer Treatment: A Randomized Controlled Trial. J Clin Oncol 2015;34:JCO.2015.63.0830.
- Morren M, van Dulmen S, Ouwerkerk J, et al. Compliance with momentary pain measurement using electronic diaries: A systematic review. Eur J Pain 2009;13:354-65. [Crossref] [PubMed]
- Stone AA, Shiffman S, Schwartz JE, et al. Patient compliance with paper and electronic diaries. Control Clin Trials 2003;24:182-99. [Crossref] [PubMed]
- Falchook AD, Tracton G, Stravers L, et al. Use of mobile device technology to continuously collect patient-reported symptoms during radiation therapy for head and neck cancer: A prospective feasibility study. Adv Radiat Oncol 2016;1:115-21. [Crossref] [PubMed]
- Duman-Lubberding S, Van Uden-Kraan CF, Peek N, et al. An eHealth application in head and neck cancer survivorship care: Health care professionals’ perspectives. J Med Internet Res 2015;17:e235 [Crossref] [PubMed]
- Gustafson DH, Hawkins R, Mctavish F, et al. Internet-based interactive support for cancer patients: Are integrated systems better? J Commun 2008;58:238-57. [Crossref] [PubMed]
- Kearney N, McCann L, Norrie J, et al. Evaluation of a mobile phone-based, advanced symptom management system (ASyMS??) in the management of chemotherapy-related toxicity. Support Care Cancer 2009;17:437-44. [Crossref] [PubMed]
- Sundberg K, Eklöf AL, Blomberg K, et al. Feasibility of an interactive ICT-platform for early assessment and management of patient-reported symptoms during radiotherapy for prostate cancer. Eur J Oncol Nurs 2015;19:523-8. [Crossref] [PubMed]
- Kenny R, Dooley B, Fitzgerald A. Feasibility of “CopeSmart”: A Telemental Heath App for Adolescents. JMIR Ment Health 2015;2:e22 [Crossref] [PubMed]
Cite this article as: Sogono PG, Bacorro WR, Sideno JE, Escuadra CJT, Que JC, Sy Ortin TT. Functionality and end-user acceptability of the Internet-based Computerized Patient Assessment System (iComPAsS), a mobile symptom monitoring system. J Hosp Manag Health Policy 2018;2:26.
