PRO-Angoff method for remote standard setting: establishing clinical thresholds for the upper digestive disease tool.

Journal: Journal of patient-reported outcomes

Volume: 8

Issue: 1

Year of Publication: 2024

Affiliated Institutions:  Department of Quantitative Health Sciences, Mayo Clinic, st Ave SW, , Rochester, MN, USA. lee.minji@mayo.edu. Department of Surgery, Division of Thoracic Surgery, Mayo Clinic, Rochester, MN, USA. Department of Quantitative Health Sciences, Mayo Clinic, st Ave SW, , Rochester, MN, USA.

Abstract summary 

The Upper Digestive Disease (UDD) Tool™ is used to monitor symptom frequency, intensity, and interference across nine symptom domains and includes two Patient-Reported Outcome Measurement Information System (PROMIS) domains assessing physical and mental health. This study aimed to establish cut scores for updated symptom domains through standard setting exercises and evaluate the effectiveness and acceptability of virtual standard setting.The extended Angoff method was employed to determine cut scores. Subject matter experts refined performance descriptions for symptom control categories and achieved consensus. Domains were categorized into good, moderate, and poor symptom control. Two cut scores were established, differentiating good vs. moderate and moderate vs. poor. Panelists estimated average scores for 100 borderline patients per item. Cut scores were computed based on the sum of the average ratings for individual questions, converted to 0-100 scale.Performance descriptions were refined. Panelists discussed that interpretation of the scores should take into account the timing of symptoms after surgery and patient populations, and the importance of items asking symptom frequency, severity, and interference with daily life. The good/moderate cut scores ranged from 21.3 to 35.0 (mean 28.6, SD 3.6) across domains, and moderate/poor ranged from 47.5 to 71.3 (mean 54.5, SD 7.0).Panelists were confident in the virtual standard setting process, expecting valid cut scores. Future studies can further validate the cut scores using patient perspectives and collect patient and physician preferences for displaying contextual items on patient- and physician-facing dashboard.

Authors & Co-authors:  Lee Abou Chaar Blackmon Yost

Study Outcome 

Source Link: Visit source

Statistics
Citations :  Basch E, et al. Symptom Monitoring with patient-reported outcomes during Routine Cancer treatment: a Randomized Controlled Trial. J Clin Oncology: Official J Am Soc Clin Oncol. 2016;34(6):557–565. doi: 10.1200/JCO.2015.63.0830.
Authors :  4
Identifiers
Doi : 30
SSN : 2509-8020
Study Population
Male,Female
Mesh Terms
Humans
Other Terms
Angoff method;Cut score;Remote monitoring of symptoms;Standard setting;Upper digestive disease
Study Design
Study Approach
Country of Study
Publication Country
Germany