Dataset Information

Health Systems and Their Assessment: A Methodological Proposal of the Synthetic Outcome Measure.

ABSTRACT: The effectiveness of health systems is an area of constant interest for public health researchers and practitioners. The varied approach to effectiveness itself has resulted in numerous methodological proposals related to its measurement. The limitations of the currently used methods lead to a constant search for better tools for the assessment of health systems. This article shows the possibilities of using the health system synthetic outcome measure (SOM) for this purpose. It is an original tool using 41 indicators referring to the epidemiological situation, health behaviors, and factors related to the health-care system, which allows a relatively quick and easy assessment of the health system in terms of its effectiveness. Construction of the measure of health system functioning in such a way allowed its presentation in dynamic perspective, i.e., assessing not only the health system itself in a given moment of time but also changes in the value of the effectiveness measures. In order to demonstrate the cognitive value of the SOM, the analysis of the effectiveness of health systems in 21 countries of Central and Eastern Europe during the transformation period was carried out. The mean SOM values calculated on the basis of the component measures allowed to differentiate countries in terms of the effectiveness of their health systems. Considering the whole period, a similar level of health system effects can be observed in Slovenia, Croatia, Czech Republic, Slovakia, Poland, Macedonia, and Albania. In the middle group, Hungary, Romania, Latvia, Lithuania, Georgia, Estonia, Bulgaria, Belarus, and Armenia were found. The third group, weakest in terms of achieved effects, was formed by health systems in countries like Ukraine, Moldova, and Russia. The presented method allows for the analysis of the health system outcomes from a comparative angle, eliminating arbitrariness of pinpointing a model solution as a potential reference point in the assessment of the systems. The measure, with the use of additional statistical tools to establish correlations with elements of the external and internal environment of a health system, allows for conducting analyses of conditions for differences in the effects of health system operation and circumstances for the effectiveness of reform processes.

SUBMITTER: Romaniuk P

PROVIDER: S-EPMC5949533 | biostudies-literature | 2018

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Health Systems and Their Assessment: A Methodological Proposal of the Synthetic Outcome Measure.

Romaniuk Piotr P Kaczmarek Krzysztof K Syrkiewicz-Świtała Magdalena M Holecki Tomasz T Szromek Adam R AR

Frontiers in public health 20180507

The effectiveness of health systems is an area of constant interest for public health researchers and practitioners. The varied approach to effectiveness itself has resulted in numerous methodological proposals related to its measurement. The limitations of the currently used methods lead to a constant search for better tools for the assessment of health systems. This article shows the possibilities of using the health system synthetic outcome measure (SOM) for this purpose. It is an original to ...[more]

PMID: 29868536

Similar Datasets

Project description:BACKGROUND:In the past years, a mobile health (mHealth) app called the Dutch Talking Touch Screen Questionnaire (DTTSQ) was developed in The Netherlands. The aim of development was to enable Dutch physical therapy patients to autonomously complete a health-related questionnaire regardless of their level of literacy and digital skills. OBJECTIVE:The aim of this study was to evaluate the usability (defined as the effectiveness, efficiency, and satisfaction) of the prototype of the DTTSQ for Dutch physical therapy patients with diverse levels of experience in using mobile technology. METHODS:The qualitative Three-Step Test-Interview method, including both think-aloud and retrospective probing techniques, was used to gain insight into the usability of the DTTSQ. A total of 24 physical therapy patients were included. The interview data were analyzed using a thematic content analysis approach aimed at analyzing the accuracy and completeness with which participants completed the questionnaire (effectiveness), the time it took the participants to complete the questionnaire (efficiency), and the extent to which the participants were satisfied with the ease of use of the questionnaire (satisfaction). The problems encountered by the participants in this study were given a severity rating that was used to provide a rough estimate of the need for additional usability efforts. RESULTS:All participants within this study were very satisfied with the ease of use of the DTTSQ. Overall, 9 participants stated that the usability of the app exceeded their expectations. The group of 4 average-/high-experienced participants encountered only 1 problem in total, whereas the 11 little-experienced participants encountered an average of 2 problems per person and the 9 inexperienced participants an average of 3 problems per person. A total of 13 different kind of problems were found during this study. Of these problems, 4 need to be addressed before the DTTSQ will be released because they have the potential to negatively influence future usage of the tool. The other 9 problems were less likely to influence future usage of the tool substantially. CONCLUSIONS:The usability of the DTTSQ needs to be improved before it can be released. No problems were found with satisfaction or efficiency during the usability test. The effectiveness needs to be improved by (1) making it easier to navigate through screens without the possibility of accidentally skipping one, (2) enabling the possibility to insert an answer by tapping on the text underneath a photograph instead of just touching the photograph itself, and (3) making it easier to correct wrong answers. This study shows the importance of including less skilled participants in a usability study when striving for inclusive design and the importance of measuring not just satisfaction but also efficiency and effectiveness during such studies.

Project description:ImportanceMany clinical trial outcomes are documented in free-text electronic health records (EHRs), making manual data collection costly and infeasible at scale. Natural language processing (NLP) is a promising approach for measuring such outcomes efficiently, but ignoring NLP-related misclassification may lead to underpowered studies.ObjectiveTo evaluate the performance, feasibility, and power implications of using NLP to measure the primary outcome of EHR-documented goals-of-care discussions in a pragmatic randomized clinical trial of a communication intervention.Design, setting, and participantsThis diagnostic study compared the performance, feasibility, and power implications of measuring EHR-documented goals-of-care discussions using 3 approaches: (1) deep-learning NLP, (2) NLP-screened human abstraction (manual verification of NLP-positive records), and (3) conventional manual abstraction. The study included hospitalized patients aged 55 years or older with serious illness enrolled between April 23, 2020, and March 26, 2021, in a pragmatic randomized clinical trial of a communication intervention in a multihospital US academic health system.Main outcomes and measuresMain outcomes were natural language processing performance characteristics, human abstractor-hours, and misclassification-adjusted statistical power of methods of measuring clinician-documented goals-of-care discussions. Performance of NLP was evaluated with receiver operating characteristic (ROC) curves and precision-recall (PR) analyses and examined the effects of misclassification on power using mathematical substitution and Monte Carlo simulation.ResultsA total of 2512 trial participants (mean [SD] age, 71.7 [10.8] years; 1456 [58%] female) amassed 44 324 clinical notes during 30-day follow-up. In a validation sample of 159 participants, deep-learning NLP trained on a separate training data set from identified patients with documented goals-of-care discussions with moderate accuracy (maximal F1 score, 0.82; area under the ROC curve, 0.924; area under the PR curve, 0.879). Manual abstraction of the outcome from the trial data set would require an estimated 2000 abstractor-hours and would power the trial to detect a risk difference of 5.4% (assuming 33.5% control-arm prevalence, 80% power, and 2-sided α = .05). Measuring the outcome by NLP alone would power the trial to detect a risk difference of 7.6%. Measuring the outcome by NLP-screened human abstraction would require 34.3 abstractor-hours to achieve estimated sensitivity of 92.6% and would power the trial to detect a risk difference of 5.7%. Monte Carlo simulations corroborated misclassification-adjusted power calculations.Conclusions and relevanceIn this diagnostic study, deep-learning NLP and NLP-screened human abstraction had favorable characteristics for measuring an EHR outcome at scale. Adjusted power calculations accurately quantified power loss from NLP-related misclassification, suggesting that incorporation of this approach into the design of studies using NLP would be beneficial.

Dataset Information

Health Systems and Their Assessment: A Methodological Proposal of the Synthetic Outcome Measure.

Publications

Health Systems and Their Assessment: A Methodological Proposal of the Synthetic Outcome Measure.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets