Dataset Information

Arthroscopic proficiency: methods in evaluating competency.

ABSTRACT: BACKGROUND: The current paradigm of arthroscopic training lacks objective evaluation of technical ability and its adequacy is concerning given the accelerating complexity of the field. To combat insufficiencies, emphasis is shifting towards skill acquisition outside the operating room and sophisticated assessment tools. We reviewed (1) the validity of cadaver and surgical simulation in arthroscopic training, (2) the role of psychomotor analysis and arthroscopic technical ability, (3) what validated assessment tools are available to evaluate technical competency, and (4) the quantification of arthroscopic proficiency. METHODS: The Medline and Embase databases were searched for published articles in the English literature pertaining to arthroscopic competence, arthroscopic assessment and evaluation and objective measures of arthroscopic technical skill. Abstracts were independently evaluated and exclusion criteria included articles outside the scope of knee and shoulder arthroscopy as well as original articles about specific therapies, outcomes and diagnoses leaving 52 articles cited in this review. RESULTS: Simulated arthroscopic environments exhibit high levels of internal validity and consistency for simple arthroscopic tasks, however the ability to transfer complex skills to the operating room has not yet been established. Instrument and force trajectory data can discriminate between technical ability for basic arthroscopic parameters and may serve as useful adjuncts to more comprehensive techniques. There is a need for arthroscopic assessment tools for standardized evaluation and objective feedback of technical skills, yet few comprehensive instruments exist, especially for the shoulder. Opinion on the required arthroscopic experience to obtain proficiency remains guarded and few governing bodies specify absolute quantities. CONCLUSIONS: Further validation is required to demonstrate the transfer of complex arthroscopic skills from simulated environments to the operating room and provide objective parameters to base evaluation. There is a deficiency of validated assessment tools for technical competencies and little consensus of what constitutes a sufficient case volume within the arthroscopy community.

SUBMITTER: Hodgins JL

PROVIDER: S-EPMC3643847 | biostudies-other | 2013

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Arthroscopic proficiency: methods in evaluating competency.

Hodgins Justin L JL Veillette Christian C

BMC medical education 20130501

<h4>Background</h4>The current paradigm of arthroscopic training lacks objective evaluation of technical ability and its adequacy is concerning given the accelerating complexity of the field. To combat insufficiencies, emphasis is shifting towards skill acquisition outside the operating room and sophisticated assessment tools. We reviewed (1) the validity of cadaver and surgical simulation in arthroscopic training, (2) the role of psychomotor analysis and arthroscopic technical ability, (3) what ...[more]

PMID: 23631421

Similar Datasets

Project description:IntroductionLearning health systems (LHS) science is fundamentally a transdisciplinary field. To capture the breadth of the competencies of an LHS scientist, AHRQ and national experts defined a series of 42 competencies across seven domains that support success. Clinicians, researchers, and leaders who are new to the LHS field can identify and prioritize proficiency development among these domains. In addition, existing leaders and researchers will assemble teams of experts who together represent the LHS science domains. To serve LHS workforce development and proficiency assessment, the AHRQ-funded ACCELERAT K12 training program recruited domain experts and trainees to define and operationalize items to include in an LHS Competency Assessment to support emerging and existing LHS scientists in prioritizing and monitoring proficiency development.MethodsSequential interviews with 18 experts iteratively defined skills and tasks to illustrate stage in proficiency, and its progression, for each of 42 competencies in the seven LHS expertise domains: systems science; research questions and standards of scientific evidence; research methods; informatics; ethics of research and implementation in health systems; improvement and implementation science; and engagement, leadership, and research management. An educational assessment expert and LHS scientist refined the assessment criteria at each stage to use parallel language across domains. Last, current trainees reviewed and pilot tested the assessment and the LHS Competency Assessment was further refined using their feedback. The assessment framework was informed by Bloom's revised taxonomy of educational objectives (Anderson and Krathwohl, A taxonomy for learning, teaching, and assessing: A revision of Bloom's taxonomy of educational objectives, 2001) where learning progresses from recalling, defining, understanding, and awareness at the lower levels of the taxonomy, to applying and adopting and finally to creating, designing, and critiquing at the upper levels of the taxonomy. We also developed assessment criteria that could be used for longer term assessment of direct performance. Van der Vleuten et al. (Best Pract Res Clin Obstetr Gynaecol. 2010;24(6):703-719) define longer term direct assessment methods as assessment that occurs over a period ranging from weeks to even years and involves multiple assessment methods and exposure to the learner's work over an extended period.ResultsThis experience report describes the content of the LHS Competency Assessment. For each domain and competency, the assessment lists examples of evidence to support expertise at each level of proficiency: no exposure; foundational (awareness/understanding); emerging (early application); and proficient (application with a high level of skill). Trainees begin with baseline standard assessment tables, where they can indicate no exposure or mark the foundational and emerging skills with which they have competence. For domains where foundational and emerging skills have been achieved, users can move on to assessment tables that list evidence of proficiency.ConclusionThe LHS Competency Assessment offers consistent, graded criteria across the seven LHS domains to guide trainees and mentors to evaluate progress from no experience to foundational knowledge, emerging proficiency, and proficiency. The assessment can also be used to design training and mentoring for those newly exposed to LHS science and for those with key expertise who wish to expand LHS expertise.

Project description:Researchers have previously developed a multitude of methods designed to identify biological pathways associated with specific clinical or experimental conditions of interest, with the aim of facilitating biological interpretation of high-throughput data. Before practically applying such pathway analysis (PA) methods, we must first evaluate their performance and reliability, using datasets where the pathways perturbed by the conditions of interest have been well characterized in advance. However, such 'ground truths' (or gold standards) are often unavailable. Furthermore, previous evaluation strategies that have focused on defining 'true answers' are unable to systematically and objectively assess PA methods under a wide range of conditions.In this work, we propose a novel strategy for evaluating PA methods independently of any gold standard, either established or assumed. The strategy involves the use of two mutually complementary metrics, recall and discrimination. Recall measures the consistency of the perturbed pathways identified by applying a particular analysis method to an original large dataset and those identified by the same method to a sub-dataset of the original dataset. In contrast, discrimination measures specificity-the degree to which the perturbed pathways identified by a particular method to a dataset from one experiment differ from those identifying by the same method to a dataset from a different experiment. We used these metrics and 24 datasets to evaluate six widely used PA methods. The results highlighted the common challenge in reliably identifying significant pathways from small datasets. Importantly, we confirmed the effectiveness of our proposed dual-metric strategy by showing that previous comparative studies corroborate the performance evaluations of the six methods obtained by our strategy.Unlike any previously proposed strategy for evaluating the performance of PA methods, our dual-metric strategy does not rely on any ground truth, either established or assumed, of the pathways perturbed by a specific clinical or experimental condition. As such, our strategy allows researchers to systematically and objectively evaluate pathway analysis methods by employing any number of datasets for a variety of conditions.

Project description:BackgroundNLP models like ChatGPT promise to revolutionize text-based content delivery, particularly in medicine. Yet, doubts remain about ChatGPT's ability to reliably support evaluations of cognitive performance, warranting further investigation into its accuracy and comprehensiveness in this area.MethodA cohort of 60 cognitively normal individuals and 30 stroke survivors underwent a comprehensive evaluation, covering memory, numerical processing, verbal fluency, and abstract thinking. Healthcare professionals and NLP models GPT-3.5 and GPT-4 conducted evaluations following established standards. Scores were compared, and efforts were made to refine scoring protocols and interaction methods to enhance ChatGPT's potential in these evaluations.ResultWithin the cohort of healthy participants, the utilization of GPT-3.5 revealed significant disparities in memory evaluation compared to both physician-led assessments and those conducted utilizing GPT-4 (P < 0.001). Furthermore, within the domain of memory evaluation, GPT-3.5 exhibited discrepancies in 8 out of 21 specific measures when compared to assessments conducted by physicians (P < 0.05). Additionally, GPT-3.5 demonstrated statistically significant deviations from physician assessments in speech evaluation (P = 0.009). Among participants with a history of stroke, GPT-3.5 exhibited differences solely in verbal assessment compared to physician-led evaluations (P = 0.002). Notably, through the implementation of optimized scoring methodologies and refinement of interaction protocols, partial mitigation of these disparities was achieved.ConclusionChatGPT can produce evaluation outcomes comparable to traditional methods. Despite differences from physician evaluations, refinement of scoring algorithms and interaction protocols has improved alignment. ChatGPT performs well even in populations with specific conditions like stroke, suggesting its versatility. GPT-4 yields results closer to physician ratings, indicating potential for further enhancement. These findings highlight ChatGPT's importance as a supplementary tool, offering new avenues for information gathering in medical fields and guiding its ongoing development and application.

Dataset Information

Arthroscopic proficiency: methods in evaluating competency.

Publications

Arthroscopic proficiency: methods in evaluating competency.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets