Dataset Information

Clinical measurements obtained from point-of-care ultrasound images to assess acquisition skills.

ABSTRACT:

Background

Current methods of assessing competence in acquiring point-of-care ultrasound images are inadequate. They rely upon cumbersome rating systems that do not depend on the actual outcome measured and lack evidence of validity. We describe a new method that uses a rigorous statistical model to assess performance of individual trainees based on the actual task, image acquisition. Measurements obtained from the images acquired (the actual desired outcome) are themselves used to validate effective training and competence acquiring ultrasound images. We enrolled a convenience sample of 21 spontaneously breathing adults from a general medicine ward. In random order, two trainees (A and B) and an instructor contemporaneously acquired point-of-care ultrasound images of the inferior vena cava and the right internal jugular vein from the same patients. Blinded diameter measurements from each ultrasound were analyzed quantitatively using a multilevel model. Consistent mean differences between each trainee's and the instructor's images were ascribed to systematic acquisition errors, indicative of poor measurement technique and a need for further training. Wider variances were attributed to sporadic errors, indicative of inconsistent application of measurement technique across patients. In addition, the instructor recorded qualitative observations of each trainee's performance during image acquisition.

Results

For all four diameters, the means and variances of measurements from trainee A's images differed significantly from the instructor's, whereas those from trainee B's images were comparable. Techniques directly observed by the instructor supported these model-derived findings. For example, mean anteroposterior diameters of the internal jugular vein obtained from trainee A's images were 3.8 mm (90% CI 2.3-5.4) smaller than from the instructor's; this model-derived finding matched the instructor's observation that trainee A compressed the vein during acquisition. Instructor summative assessments agreed with model-derived findings, providing internal validation of the descriptive and quantitative assessments of competence acquiring ultrasound images.

Conclusions

Clinical measurements obtained from point-of-care ultrasound images acquired contemporaneously by trainees and an instructor can be used to quantitatively assess the image acquisition competence of specific trainees. This method may obviate resource-intensive qualitative rating systems that are based on ultrasound image quality and direct observation, while also helping instructors guide remediation.

SUBMITTER: Lucas BP

PROVIDER: S-EPMC6638581 | biostudies-literature | 2019 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Clinical measurements obtained from point-of-care ultrasound images to assess acquisition skills.

Lucas Brian P BP D'Addio Antonietta A Block Clay C Manning Harold L HL Remillard Brian B Leiter James C JC

The ultrasound journal 20190222 1

<h4>Background</h4>Current methods of assessing competence in acquiring point-of-care ultrasound images are inadequate. They rely upon cumbersome rating systems that do not depend on the actual outcome measured and lack evidence of validity. We describe a new method that uses a rigorous statistical model to assess performance of individual trainees based on the actual task, image acquisition. Measurements obtained from the images acquired (the actual desired outcome) are themselves used to valid ...[more]

PMID: 31359267

Similar Datasets

Project description:Few studies have compared the effectiveness of brief training courses on point-of-care ultrasound (POCUS) skill acquisition of novice attending physicians vs. trainees. The purpose of this study was to evaluate the change in POCUS image interpretation skills and confidence of novice attending physicians vs. trainees after a 1-day POCUS training course.A 1-day POCUS training course was held in March 2017 in Japan. A standardized training curriculum was developed that included online education, live lectures, and hands-on training. The pre-course assessment tools included a written examination to evaluate baseline knowledge and image interpretation skills, and a physician survey to assess confidence in performing specific ultrasound applications. The same assessment tools were administered post-course, along with a course evaluation. All learners were novices and were categorized as trainees or attending physicians. Data were analyzed using two-way analysis of variance.In total, 60 learners attended the course, and 51 learners (85%) completed all tests and surveys. The 51 novice learners included 29 trainees (4 medical students, 9 PGY 1-2 residents, 16 PGY 3-5 residents) and 22 attending physicians (6 PGY 6-10 physicians, and 16 physicians PGY 11 and higher). The mean pre- and post-course test scores of novice trainees improved from 65.5 to 83.9% while novice attending physicians improved from 66.7 to 81.5% (p?<?0.001). The post-course physician confidence scores in using ultrasound significantly increased in all skill categories for both groups. Both trainees and attending physicians demonstrated similar improvement in their post-course test scores and confidence with no statistically significant differences between the groups. The course evaluation scores for overall satisfaction and satisfaction with faculty members' teaching skills were 4.5 and 4.6 on a 5-point scale, respectively.Both novice trainees and attending physicians showed similar improvement in point-of-care ultrasound image interpretation skills and confidence after a brief training course. Although separate training courses have traditionally been developed for attending physicians and trainees, novice learners of point-of-care ultrasound may acquire skills at similar rates, regardless of their ranking as an attending physician or trainee. Future studies are needed to compare the effectiveness of short training courses on image acquisition skills and determine the ideal course design.

Project description:BackgroundPainful arthropathy is a long-term complication in patients with hemophilia (PWH), affecting mobility and quality of life. A major barrier for the appraisal of joint health is the absence of point-of-care (POC) imaging modalities to promptly identify and manage arthropathic changes. Accordingly, we developed the Joint tissue Activity and Damage Exam (JADE) POC musculoskeletal ultrasound (MSKUS) protocol. JADE is validated for haemophilic joint tissue recognition with high intra/inter-rater and inter-operator reliability.AimsEvaluate associations of JADE with clinical (Hemophilia Joint Health Score, [HJHS]) and functional (total arc [combined flexion and extension range of motion [ROM]]) parameters.MethodologyIn this multi-centre prospective study, we recruited PWH A or B with at least one arthropathic joint. We evaluated joint health (both elbows, knees, and ankles) by comparing JADE measurements (soft tissue and cartilage thickness, and osteochondral alterations) with HJHS and total arc.ResultsOf 44 PWH, most had hemophilia A (35/44), were severe (36/44) and had a median age of 36 years. Increasing HJHSs and declining total arc, indicating worsening arthropathy, were associated with JADE measurements in the expected direction, including (1) increasing length of osteochondral alterations, (2) diminished cartilage thickness, and (3) greater soft tissue expansion. The ankles had the highest proportion of joints without measurable (missing) cartilage. In multivariable models MSKUS measurements explained 68% and 71% of the variation in HJHS and total arc respectively for the elbow, 55% and 29% respectively for the knee, and 50% and 73% for the ankle.ConclusionsThis study highlights the associations of direct intra-articular ultrasonography measurements using the JADE protocol with clinical and functional parameters. Our findings underscore the clinical value of POC MSKUS using the JADE protocol as a complementary instrument for the diagnosis and management of haemophilic arthropathy.

Project description:ObjectivesAssessment of competence in technical skills, including point-of-care ultrasound (POCUS), is required before a novice can safely perform the skill independently. Ongoing assessment of competence is also required because technical skills degrade over time, especially when they are infrequently performed or complex. Hand-motion analysis (HMA) is an objective assessment tool that has been used to evaluate competency in many technical skills. The purpose of this study was to demonstrate the feasibility and validity of HMA as an assessment tool for competence in both simple and complex technical skills as well as skill degradation over time.MethodsThis prospective cohort study included 36 paramedics with no POCUS experience and six physicians who were fellowship trained in POCUS. The novices completed a 4-hour didactic and hands-on training program for cardiac and lung POCUS. HMA measurements, objective structured clinical examinations (OSCE), and written examinations were collected for novices immediately before and after training as well as 2 and 4 months after training. Expert HMA metrics were also recorded.ResultsExpert HMA metrics for cardiac and lung POCUS were significantly better than those of novices. After completion of the training program, the novices improved significantly in all HMA metrics, knowledge test scores, and OSCE scores. Novices showed skill degradation in cardiac POCUS based on HMA metrics and OSCE scores while lung POCUS image acquisition skills were preserved. Novices deemed competent by OSCE score performed significantly better in HMA metrics than those not deemed competent.ConclusionWe have demonstrated that HMA is a feasible and valid tool for assessment of competence in technical skills and can also evaluate skill degradation over time. Skill degradation appears more apparent in complex skills, such as cardiac POCUS. HMA may provide a more efficient and reliable assessment of technical skills, including POCUS, when compared to traditional assessment tools.

Dataset Information

Clinical measurements obtained from point-of-care ultrasound images to assess acquisition skills.

Background

Results

Conclusions

Publications

Clinical measurements obtained from point-of-care ultrasound images to assess acquisition skills.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure