Dataset Information

Task-oriented evaluation of electronic medical records systems: development and validation of a questionnaire for physicians.

ABSTRACT:

Background

Evaluation is a challenging but necessary part of the development cycle of clinical information systems like the electronic medical records (EMR) system. It is believed that such evaluations should include multiple perspectives, be comparative and employ both qualitative and quantitative methods. Self-administered questionnaires are frequently used as a quantitative evaluation method in medical informatics, but very few validated questionnaires address clinical use of EMR systems.

Methods

We have developed a task-oriented questionnaire for evaluating EMR systems from the clinician's perspective. The key feature of the questionnaire is a list of 24 general clinical tasks. It is applicable to physicians of most specialties and covers essential parts of their information-oriented work. The task list appears in two separate sections, about EMR use and task performance using the EMR, respectively. By combining these sections, the evaluator may estimate the potential impact of the EMR system on health care delivery. The results may also be compared across time, site or vendor. This paper describes the development, performance and validation of the questionnaire. Its performance is shown in two demonstration studies (n = 219 and 80). Its content is validated in an interview study (n = 10), and its reliability is investigated in a test-retest study (n = 37) and a scaling study (n = 31).

Results

In the interviews, the physicians found the general clinical tasks in the questionnaire relevant and comprehensible. The tasks were interpreted concordant to their definitions. However, the physicians found questions about tasks not explicitly or only partially supported by the EMR systems difficult to answer. The two demonstration studies provided unambiguous results and low percentages of missing responses. In addition, criterion validity was demonstrated for a majority of task-oriented questions. Their test-retest reliability was generally high, and the non-standard scale was found symmetric and ordinal.

Conclusion

This questionnaire is relevant for clinical work and EMR systems, provides reliable and interpretable results, and may be used as part of any evaluation effort involving the clinician's perspective of an EMR system.

SUBMITTER: Laerum H

PROVIDER: S-EPMC356923 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundAs a result of demographic changes, physicians are required to deliver needed services with limited resources. Research suggests that tablet PCs with access to patient data may streamline clinical workflow. A recent study found tablets with mobile electronic medical records (EMRs) can facilitate data retrieval and produce time savings across the clinical routine within hospital settings. However, the reasons for these time savings, including details on how tablets were being used, remain unclear. The same applies to physicians' perceptions of this tool within an inpatient setting.ObjectiveThis study examined physicians' perception of tablets with EMRs in an inpatient setting. The rationale was to identify both subjective and objective factors that impacted the successful implementation and use of tablets running an EMR.MethodsWe developed a 57-item survey questionnaire designed to examine users' perception of and attitude toward tablets, which was administered to 14 participating physicians following 7 weeks of tablet use. Five participants volunteered to participate in a second study that investigated physicians' patterns of tablet use within the EMR environment by digitally tracking and storing usage behavior. Statistical analyses of questionnaire results included mean values with their bootstrapped 95% confidence intervals and multivariate analysis of variance to identify predictors of tablet use.ResultsPhysicians reported high degrees of satisfaction with the tablets. There was a general consensus among physicians that tablet use streamlined clinical workflow through optimized data retrieval (rated 0.69, 0.23-1.15 points better than control) and improved communication with patients and other physicians (rated 0.85, 0.54-1.15 and 0.77, 0.38-1.15 points better than control, respectively). Age (F3,11=3.54, P=.04), occupational group (F1,11=7.17, P=.04), and attitude toward novel technologies (F1,11=10.54, P=.02) predicted physicians' satisfaction with the devices and their motivation regarding their further use. Tracking data yielded that only a few of the available functions were used frequently.ConclusionsAlthough tablet PCs were consistently perceived as beneficial, several factors contributed to the fact that their full potential was not fully exploited. Training in functionality and providing a reliable infrastructure might foster successful tablet implementation.

Project description:BACKGROUND:Electronic medical records (EMRs) contain a wealth of information that can support data-driven decision making in health care policy design and service planning. Although research using EMRs has become increasingly prevalent, challenges such as coding inconsistency, data validity, and lack of suitable measures in important domains still hinder the progress. OBJECTIVE:The objective of this study was to design a structured way to process records in administrative EMR systems for health services research and assess validity in selected areas. METHODS:On the basis of a local hospital EMR system in Singapore, we developed a structured framework for EMR data processing, including standardization and phenotyping of diagnosis codes, construction of cohort with multilevel views, and generation of variables and proxy measures to supplement primary data. Disease complexity was estimated by Charlson Comorbidity Index (CCI) and Polypharmacy Score (PPS), whereas socioeconomic status (SES) was estimated by housing type. Validity of modified diagnosis codes and derived measures were investigated. RESULTS:Visit-level (N=7,778,761) and patient-level records (n=549,109) were generated. The International Classification of Diseases, Tenth Revision, Australian Modification (ICD-10-AM) codes were standardized to the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) with a mapping rate of 87.1%. In all, 97.4% of the ICD-9-CM codes were phenotyped successfully using Clinical Classification Software by Agency for Healthcare Research and Quality. Diagnosis codes that underwent modification (truncation or zero addition) in standardization and phenotyping procedures had the modification validated by physicians, with validity rates of more than 90%. Disease complexity measures (CCI and PPS) and SES were found to be valid and robust after a correlation analysis and a multivariate regression analysis. CCI and PPS were correlated with each other and positively correlated with health care utilization measures. Larger housing type was associated with lower government subsidies received, suggesting association with higher SES. Profile of constructed cohorts showed differences in disease prevalence, disease complexity, and health care utilization in those aged above 65 years and those aged 65 years or younger. CONCLUSIONS:The framework proposed in this study would be useful for other researchers working with EMR data for health services research. Further analyses would be needed to better understand differences observed in the cohorts.

Project description:BackgroundDespite a substantial increase in the adoption of electronic medical records (EMRs) in primary health care settings, the use of advanced EMR features is limited. Several studies have identified both barriers and facilitating factors that influence primary care physicians' (PCPs) use of advanced EMR features and the maturation of their EMR use. The purpose of this study is to explore and identify the factors that impact PCPs' mature use of EMRs.MethodsA systematic review was conducted in accordance with the Cochrane Handbook. The MEDLINE, Embase, and PsycINFO electronic databases were searched from 1946 to June 13, 2019. Two independent reviewers screened the studies for eligibility; to be included, studies had to address factors influencing PCPs' mature use of EMRs. A narrative synthesis was conducted to collate study findings and to report on patterns identified across studies. The quality of the studies was also appraised.ResultsOf the 1893 studies identified, 14 were included in this study. Reported factors that influenced PCPs' mature use of EMRs fell into one of the following 5 categories: technology, people, organization, resources, and policy. Concerns about the EMR system's functionality, lack of physician awareness of EMR functionality, limited physician availability to learn more about EMRs, the habitual use of successfully completing clinical tasks using only basic EMR features, business-oriented organizational objectives, lack of vendor training, limited resource availability, and lack of physician readiness were reported as barriers to PCPs' mature use of EMRs. The motivation of physicians, user satisfaction, coaching and peer mentoring, EMR experience, gender, physician perception, transition planning for changes in roles and work processes, team-based care, adequate technical support and training, sharing resources, practices affiliated with an integrated delivery system, financial incentives, and policies to increase EMR use all had a favorable impact on PCPs' use of advanced EMR features.ConclusionsBy using a narrative synthesis to synthesize the evidence, we identified interrelated factors influencing the mature use of EMRs by PCPs. The findings underline the need to provide adequate training and policies that facilitate the mature use of EMRs by PCPs.Trial registrationPROSPERO CRD42019137526.