Video-based fully automatic assessment of open surgery suturing skills
Ontology highlight
ABSTRACT: Purpose
The goal of this study was to develop a new reliable open surgery suturing simulation system for training medical students in situations where resources are limited or in the domestic setup. Namely, we developed an algorithm for tools and hands localization as well as identifying the interactions between them based on simple webcam video data, calculating motion metrics for assessment of surgical skill. Methods
Twenty-five participants performed multiple suturing tasks using our simulator. The YOLO network was modified to a multi-task network for the purpose of tool localization and tool–hand interaction detection. This was accomplished by splitting the YOLO detection heads so that they supported both tasks with minimal addition to computer run-time. Furthermore, based on the outcome of the system, motion metrics were calculated. These metrics included traditional metrics such as time and path length as well as new metrics assessing the technique participants use for holding the tools. Results
The dual-task network performance was similar to that of two networks, while computational load was only slightly bigger than one network. In addition, the motion metrics showed significant differences between experts and novices. Conclusion
While video capture is an essential part of minimal invasive surgery, it is not an integral component of open surgery. Thus, new algorithms, focusing on the unique challenges open surgery videos present, are required. In this study, a dual-task network was developed to solve both a localization task and a hand–tool interaction task. The dual network may be easily expanded to a multi-task network, which may be useful for images with multiple layers and for evaluating the interaction between these different layers. Supplementary Information
The online version contains supplementary material available at 10.1007/s11548-022-02559-6.
SUBMITTER: Goldbraikh A
PROVIDER: S-EPMC8805431 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BACKGROUND:Pre-recorded videotapes have become the standard approach when teaching clinical communication skills (CCS). Furthermore, video-based feedback (VF) has proven to be beneficial in formative assessments. However, VF in CCS with the use of pre-recorded videos from real-life settings is less commonly studied than the use of simulated patients. To explore: 1) perceptions about the potential benefits and challenges in this kind of VF; 2) differences in the CCC scores in first-year medical residents in primary care, before and after a communication program using VF in a curricular formative assessment. METHOD:We conducted a pre/post study with a control group. The intervention consisted of VF sessions regarding CCS, performed in a small group with peers and a facilitator. They reviewed clinical consultations pre-recorded in a primary care setting with real patients. Before and after the intervention, 54 medical residents performed two clinical examinations with simulated patients (SP), answered quantitative scales (Perception of Patient-Centeredness and Jefferson Empathy Scale), and semi-structured qualitative questionnaires. The performances were scored by SP (Perception of Patient-Centeredness and CARE scale) and by two blind raters (SPIKES protocol-based and CCOG-based scale). The quantitative data analysis employed repeated-measures ANOVA. The qualitative analysis used the Braun and Clarke framework for thematic analysis. RESULTS:The quantitative analyses did not reveal any significant differences in the sum scores of the questionnaires, except for the Jefferson Empathy Scale. In the qualitative questionnaires, the main potential benefits that emerged from the thematic analysis of the VF method were self-perception, peer-feedback, patient-centered approach, and incorporation of reflective practices. A challenging aspect that emerged from facilitators was the struggle to relate the VF with theoretical references and the resident's initial stress to record and watch oneself on video. CONCLUSION:VF taken from real-life settings seems to be associated with a significant increase in self-perceived empathy. The study of other quantitative outcomes related to this VF intervention needs larger sample sizes. VF with clinical patients from real healthcare settings appears to be an opportunity for a deeper level of self-assessment, peer-feedback, and reflective practices.
| S-EPMC7041283 | biostudies-literature
Project description:Aiming at these problems of image colorization algorithms based on deep learning, such as color bleeding and insufficient color, this paper converts the study of image colorization to the optimization of image semantic segmentation, and proposes a fully automatic image colorization model based on semantic segmentation technology. Firstly, we use the encoder as the local feature extraction network and use VGG-16 as the global feature extraction network. These two parts do not interfere with each other, but they share the low-level feature. Then, the first fusion module is constructed to merge local features and global features, and the fusion results are input into semantic segmentation network and color prediction network respectively. Finally, the color prediction network obtains the semantic segmentation information of the image through the second fusion module, and predicts the chrominance of the image based on it. Through several sets of experiments, it is proved that the performance of our model becomes stronger and stronger under the nourishment of the data. Even in some complex scenes, our model can predict reasonable colors and color correctly, and the output effect is very real and natural.
| S-EPMC8631650 | biostudies-literature
Project description:ObjectivesTo simulate clinical deployment, evaluate performance, and establish quality assurance of a deep learning algorithm (U-Net) for detection, localization, and segmentation of clinically significant prostate cancer (sPC), ISUP grade group ??2, using bi-parametric MRI.MethodsIn 2017, 284 consecutive men in active surveillance, biopsy-naïve or pre-biopsied, received targeted and extended systematic MRI/transrectal US-fusion biopsy, after examination on a single MRI scanner (3 T). A prospective adjustment scheme was evaluated comparing the performance of the Prostate Imaging Reporting and Data System (PI-RADS) and U-Net using sensitivity, specificity, predictive values, and the Dice coefficient.ResultsIn the 259 eligible men (median 64 [IQR 61-72] years), PI-RADS had a sensitivity of 98% [106/108]/84% [91/108] with a specificity of 17% [25/151]/58% [88/151], for thresholds at ? 3/? 4 respectively. U-Net using dynamic threshold adjustment had a sensitivity of 99% [107/108]/83% [90/108] (p >?0.99/>?0.99) with a specificity of 24% [36/151]/55% [83/151] (p >?0.99/>?0.99) for probability thresholds d3 and d4 emulating PI-RADS ? 3 and ? 4 decisions respectively, not statistically different from PI-RADS. Co-occurrence of a radiological PI-RADS ? 4 examination and U-Net ??d3 assessment significantly improved the positive predictive value from 59 to 63% (p =?0.03), on a per-patient basis.ConclusionsU-Net has similar performance to PI-RADS in simulated continued clinical use. Regular quality assurance should be implemented to ensure desired performance.Key points• U-Net maintained similar diagnostic performance compared to radiological assessment of PI-RADS ? 4 when applied in a simulated clinical deployment. • Application of our proposed prospective dynamic calibration method successfully adjusted U-Net performance within acceptable limits of the PI-RADS reference over time, while not being limited to PI-RADS as a reference. • Simultaneous detection by U-Net and radiological assessment significantly improved the positive predictive value on a per-patient and per-lesion basis, while the negative predictive value remained unchanged.
| S-EPMC7755653 | biostudies-literature
Project description:ImportanceA high level of surgical skill is essential to prevent intraoperative problems. One important aspect of surgical education is surgical skill assessment, with pertinent feedback facilitating efficient skill acquisition by novices.ObjectivesTo develop a 3-dimensional (3-D) convolutional neural network (CNN) model for automatic surgical skill assessment and to evaluate the performance of the model in classification tasks by using laparoscopic colorectal surgical videos.Design, setting, and participantsThis prognostic study used surgical videos acquired prior to 2017. In total, 650 laparoscopic colorectal surgical videos were provided for study purposes by the Japan Society for Endoscopic Surgery, and 74 were randomly extracted. Every video had highly reliable scores based on the Endoscopic Surgical Skill Qualification System (ESSQS, range 1-100, with higher scores indicating greater surgical skill) established by the society. Data were analyzed June to December 2020.Main outcomes and measuresFrom the groups with scores less than the difference between the mean and 2 SDs, within the range spanning the mean and 1 SD, and greater than the sum of the mean and 2 SDs, 17, 26, and 31 videos, respectively, were randomly extracted. In total, 1480 video clips with a length of 40 seconds each were extracted for each surgical step (medial mobilization, lateral mobilization, inferior mesenteric artery transection, and mesorectal transection) and separated into 1184 training sets and 296 test sets. Automatic surgical skill classification was performed based on spatiotemporal video analysis using the fully automated 3-D CNN model, and classification accuracies and screening accuracies for the groups with scores less than the mean minus 2 SDs and greater than the mean plus 2 SDs were calculated.ResultsThe mean (SD) ESSQS score of all 650 intraoperative videos was 66.2 (8.6) points and for the 74 videos used in the study, 67.6 (16.1) points. The proposed 3-D CNN model automatically classified video clips into groups with scores less than the mean minus 2 SDs, within 1 SD of the mean, and greater than the mean plus 2 SDs with a mean (SD) accuracy of 75.0% (6.3%). The highest accuracy was 83.8% for the inferior mesenteric artery transection. The model also screened for the group with scores less than the mean minus 2 SDs with 94.1% sensitivity and 96.5% specificity and for group with greater than the mean plus 2 SDs with 87.1% sensitivity and 86.0% specificity.Conclusions and relevanceThe results of this prognostic study showed that the proposed 3-D CNN model classified laparoscopic colorectal surgical videos with sufficient accuracy to be used for screening groups with scores greater than the mean plus 2 SDs and less than the mean minus 2 SDs. The proposed approach was fully automatic and easy to use for various types of surgery, and no special annotations or kinetics data extraction were required, indicating that this approach warrants further development for application to automatic surgical skill assessment.
| S-EPMC8363914 | biostudies-literature
Project description:BackgroundPandemic-induced restrictions forced curriculum transformation from on-site education to virtual learning options. This report describes this transition, the challenge of creating technology-enhanced learning for hands-on psychomotor skills teaching in physiotherapy, and students' evaluations of the new technology-enhanced learning approach in Complex Decongestive Physiotherapy.MethodsOn-site theoretical background lectures were replaced with e-learning sessions. Faculty hands-on skills demonstrations for the entire class were replaced with video-recorded demonstrations. Videos included verbal and written instructions and were complemented with checklists guiding the students, training in pairs, through their learning tasks. A cross-sectional observational survey for teaching quality evaluated this new technology-enhanced learning approach and assessed students' preference for traditional or video-based hands-on skills learning.ResultsSurvey return rate was > 50% (46 participating students). Teaching quality was rated between 1.5 ± 0.5 and 1.8 ± 0.4 (Likert scale from - 2 to + 2). Most students (66.7%) preferred the new approach. They appreciated for example that videos were available all the time, enabling self-paced learning, providing an equally good view on skills demonstrations, and the convenience to be able to rewind, re-view, and use speed adjustment options.ConclusionsStudents preferred the new video-based learning of skills for Complex Decongestive Physiotherapy. Because in-class live skills demonstrations were omitted, faculty had more time to provide individual feedback and answer questions. The shift from teacher- to student-centered learning enabled students to control their own learning pace. The innovative program was maintained after pandemic-induced restrictions were lifted. The success of this approach should be tested in other physiotherapy settings and different educational institutions.
| S-EPMC10474678 | biostudies-literature
Project description:BackgroundResidents may benefit from simulated practice with personalized feedback to prepare for high-stakes disclosure conversations with patients after harmful errors and to meet American Council on Graduate Medical Education mandates. Ideally, feedback would come from patients who have experienced communication after medical harm, but medical researchers and leaders have found it difficult to reach this community, which has made this approach impractical at scale. The Video-Based Communication Assessment app is designed to engage crowdsourced laypeople to rate physician communication skills but has not been evaluated for use with medical harm scenarios.ObjectiveWe aimed to compare the reliability of 2 assessment groups (crowdsourced laypeople and patient advocates) in rating physician error disclosure communication skills using the Video-Based Communication Assessment app.MethodsInternal medicine residents used the Video-Based Communication Assessment app; the case, which consisted of 3 sequential vignettes, depicted a delayed diagnosis of breast cancer. Panels of patient advocates who have experienced harmful medical error, either personally or through a family member, and crowdsourced laypeople used a 5-point scale to rate the residents' error disclosure communication skills (6 items) based on audiorecorded responses. Ratings were aggregated across items and vignettes to create a numerical communication score for each physician. We used analysis of variance, to compare stringency, and Pearson correlation between patient advocates and laypeople, to identify whether rank order would be preserved between groups. We used generalizability theory to examine the difference in assessment reliability between patient advocates and laypeople.ResultsInternal medicine residents (n=20) used the Video-Based Communication Assessment app. All patient advocates (n=8) and 42 of 59 crowdsourced laypeople who had been recruited provided complete, high-quality ratings. Patient advocates rated communication more stringently than crowdsourced laypeople (patient advocates: mean 3.19, SD 0.55; laypeople: mean 3.55, SD 0.40; P<.001), but patient advocates' and crowdsourced laypeople's ratings of physicians were highly correlated (r=0.82, P<.001). Reliability for 8 raters and 6 vignettes was acceptable (patient advocates: G coefficient 0.82; crowdsourced laypeople: G coefficient 0.65). Decision studies estimated that 12 crowdsourced layperson raters and 9 vignettes would yield an acceptable G coefficient of 0.75.ConclusionsCrowdsourced laypeople may represent a sustainable source of reliable assessments of physician error disclosure skills. For a simulated case involving delayed diagnosis of breast cancer, laypeople correctly identified high and low performers. However, at least 12 raters and 9 vignettes are required to ensure adequate reliability and future studies are warranted. Crowdsourced laypeople rate less stringently than raters who have experienced harm. Future research should examine the value of the Video-Based Communication Assessment app for formative assessment, summative assessment, and just-in-time coaching of error disclosure communication skills.
| S-EPMC9107044 | biostudies-literature
Project description:White matter hyperintensities (WMH) on T2 or FLAIR sequences have been commonly observed on MR images of elderly people. They have been associated with various disorders and have been shown to be a strong risk factor for stroke and dementia. WMH studies usually required visual evaluation of WMH load or time-consuming manual delineation. This paper introduced WHASA (White matter Hyperintensities Automated Segmentation Algorithm), a new method for automatically segmenting WMH from FLAIR and T1 images in multicentre studies. Contrary to previous approaches that were based on intensities, this method relied on contrast: non linear diffusion filtering alternated with watershed segmentation to obtain piecewise constant images with increased contrast between WMH and surroundings tissues. WMH were then selected based on subject dependant automatically computed threshold and anatomical information. WHASA was evaluated on 67 patients from two studies, acquired on six different MRI scanners and displaying a wide range of lesion load. Accuracy of the segmentation was assessed through volume and spatial agreement measures with respect to manual segmentation; an intraclass correlation coefficient (ICC) of 0.96 and a mean similarity index (SI) of 0.72 were obtained. WHASA was compared to four other approaches: Freesurfer and a thresholding approach as unsupervised methods; k-nearest neighbours (kNN) and support vector machines (SVM) as supervised ones. For these latter, influence of the training set was also investigated. WHASA clearly outperformed both unsupervised methods, while performing at least as good as supervised approaches (ICC range: 0.87-0.91 for kNN; 0.89-0.94 for SVM. Mean SI: 0.63-0.71 for kNN, 0.67-0.72 for SVM), and did not need any training set.
| S-EPMC3495958 | biostudies-literature
Project description:The study objective was to investigate the performance of a dedicated convolutional neural network (CNN) optimized for wrist cartilage segmentation from 2D MR images. CNN utilized a planar architecture and patch-based (PB) training approach that ensured optimal performance in the presence of a limited amount of training data. The CNN was trained and validated in 20 multi-slice MRI datasets acquired with two different coils in 11 subjects (healthy volunteers and patients). The validation included a comparison with the alternative state-of-the-art CNN methods for the segmentation of joints from MR images and the ground-truth manual segmentation. When trained on the limited training data, the CNN outperformed significantly image-based and PB-U-Net networks. Our PB-CNN also demonstrated a good agreement with manual segmentation (Sørensen-Dice similarity coefficient [DSC] = 0.81) in the representative (central coronal) slices with a large amount of cartilage tissue. Reduced performance of the network for slices with a very limited amount of cartilage tissue suggests the need for fully 3D convolutional networks to provide uniform performance across the joint. The study also assessed inter- and intra-observer variability of the manual wrist cartilage segmentation (DSC = 0.78-0.88 and 0.9, respectively). The proposed deep learning-based segmentation of the wrist cartilage from MRI could facilitate research of novel imaging markers of wrist osteoarthritis to characterize its progression and response to therapy.
| S-EPMC7784718 | biostudies-literature
Project description:Seizure detection is a routine process in epilepsy units requiring manual intervention of well-trained specialists. This process could be extensive, inefficient and time-consuming, especially for long term recordings. We proposed an automatic method to detect epileptic seizures using an imaged-EEG representation of brain signals. To accomplish this, we analyzed EEG signals from two different datasets: the CHB-MIT Scalp EEG database and the EPILEPSIAE project that includes scalp and intracranial recordings. We used fully convolutional neural networks to automatically detect seizures. For our best model, we reached average accuracy and specificity values of 99.3% and 99.6%, respectively, for the CHB-MIT dataset, and corresponding values of 98.0% and 98.3% for the EPILEPSIAE patients. For these patients, the inclusion of intracranial electrodes together with scalp ones increased the average accuracy and specificity values to 99.6% and 58.3%, respectively. Regarding the other metrics, our best model reached average precision of 62.7%, recall of 58.3%, F-measure of 59.0% and AP of 54.5% on the CHB-MIT recordings, and comparatively lowers performances for the EPILEPSIAE dataset. For both databases, the number of false alarms per hour reached values less than 0.5/h for 92% of the CHB-MIT patients and less than 1.0/h for 80% of the EPILEPSIAE patients. Compared to recent studies, our lightweight approach does not need any estimation of pre-selected features and demonstrates high performances with promising possibilities for the introduction of such automatic methods in the clinical practice.
| S-EPMC7732993 | biostudies-literature
Project description: Not available
| S-EPMC6560530 | biostudies-literature