Grading reflective essays: the reliability of a newly developed tool- GRE-9
Ontology highlight
ABSTRACT:
SUBMITTER: Makarem N
PROVIDER: S-EPMC7520967 | biostudies-literature | 2020 Jan
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundClinical examination of trachoma is used to justify intervention in trachoma-endemic regions. Currently, field graders are certified by determining their concordance with experienced graders using the kappa statistic. Unfortunately, trachoma grading can be highly variable and there are cases where even expert graders disagree (borderline/marginal cases). Prior work has shown that inclusion of borderline cases tends to reduce apparent agreement, as measured by kappa. Here, we confirm those results and assess performance of trainees on these borderline cases by calculating their reliability error, a measure derived from the decomposition of the Brier score.Methods and findingsWe trained 18 field graders using 200 conjunctival photographs from a community-randomized trial in Niger and assessed inter-grader agreement using kappa as well as reliability error. Three experienced graders scored each case for the presence or absence of trachomatous inflammation-follicular (TF) and trachomatous inflammation-intense (TI). A consensus grade for each case was defined as the one given by a majority of experienced graders. We classified cases into a unanimous subset if all 3 experienced graders gave the same grade. For both TF and TI grades, the mean kappa for trainees was higher on the unanimous subset; inclusion of borderline cases reduced apparent agreement by 15.7% for TF and 12.4% for TI. When we assessed the breakdown of the reliability error, we found that our trainees tended to over-call TF grades and under-call TI grades, especially in borderline cases.ConclusionsThe kappa statistic is widely used for certifying trachoma field graders. Exclusion of borderline cases, which even experienced graders disagree on, increases apparent agreement with the kappa statistic. Graders may agree less when exposed to the full spectrum of disease. Reliability error allows for the assessment of these borderline cases and can be used to refine an individual trainee's grading.
| S-EPMC4006735 | biostudies-literature
Project description:Reactive agility (RAG) and change of direction speed (CODS) are important determinants of success in football (soccer), but there is an evident lack of information on reliable and valid football-specific testing procedures which will be applicable in defining sport-specific RAG and CODS in youth players. This study evaluated reliability and construct validity of newly developed tests of football-specific RAG (FS_RAG) and CODS (FS_CODS), which involved the ball kicking football technique. Additionally, factors associated with FS_RAG and FS_CODS were evaluated. The participants were youth football players (n = 59; age: 13.40 ± 1.25 years) divided according to their age into U13 (11-12 years of age; n = 29), and U15 (13-14 years of age; n = 30) categories. Additionally, performance levels (starters [first-team] vs. non-starters [substitutes]) were observed in each age category. The dependent variables were newly developed FS_RAG and FS_CODS tests. The independent variables were sprinting capacities over 10 and 20 meters (S10M, S20M), countermovement jump (CMJ), the reactive strength index (RSI), and a generic CODS test of 20 yards (20Y). The newly developed FS_CODS and FS_RAG were observed as dependent variables. Results showed appropriate intra-testing and inter-testing reliability of the FS_RAG and FS_CODS, with somewhat better reliability of the FS_CODS (ICC=0.82 and 0.79, respectively). Additionally, better reliability was evidenced in U15 than in U13 (ICC: 0.82-0.85, and 0.78-0.80 for U15 and U13, respectively). Independent samples t-test indicated significant differences between U13 and U15 in S10 (t-test: 3.57, p < 0.001), S20M (t-test: 3.13, p < 0.001), 20Y (t-test: 4.89, p < 0.001), FS_RAG (t-test: 3.96, p < 0.001), and FS_CODS (t-test: 6.42, p < 0.001), with better performance in U15. Starters outperformed non-starters in most capacities among U13, but only in FS_RAG among U15 (t-test: 1.56, p < 0.05). Multiple regression calculations indicated nonsignificant association between independent and dependent variables in U13 (FS_CODS: 19%, FS_RAG: 21% of the explained variance, both p > 0.05), but independent variables explained significant proportion of both dependent variables in U15 (FS_CODS: 35%, FS_RAG: 33% explained variance, both p < 0.05). The study confirmed the applicability of newly developed tests in distinguishing studied age categories of players. Results indicate that superiority in all studied fitness capacities is translated into performance level in U13. Meanwhile, FS_RAG seems to be important determinant of quality in U15.
| S-EPMC6981745 | biostudies-literature
Project description:BackgroundFor the diagnosis of seasonal influenza, clinicians rely on point-of-care testing (POCT) using commercially available kits developed against seasonal influenza viruses. However, POCT has not yet been established for the diagnosis of pandemic influenza A virus (H1N1pdm) infection due to the low sensitivity of the existing kits for H1N1pdm.Methodology/principal findingsAn immunochromatography (IC) test kit was developed based on a monoclonal antibody against H1N1pdm, which does not cross-react with seasonal influenza A or B viruses. The efficacy of this kit (PDM-IC kit) for the diagnosis of H1N1pdm infection was compared with that of an existing kit for the detection of seasonal influenza viruses (SEA-IC kit). Nasal swabs (n?=?542) were obtained from patients with flu-like syndrome at 13 clinics in Osaka, Japan during the winter of 2010/2011. Among the 542 samples, randomly selected 332 were further evaluated for viral presence by reverse transcriptase polymerase chain reaction (RT-PCR). The PDM-IC kit versus the SEA-IC kit showed higher sensitivity to and specificity for H1N1pdm, despite several inconsistencies between the two kits or between the kits and RT-PCR. Consequently, greater numbers of false-negative and false-positive cases were documented when the SEA-IC kit was employed. Significant correlation coefficients for sensitivity, specificity, and negative prediction values between the two kits were observed at individual clinics, indicating that the results could be affected by clinic-related techniques for sampling and kit handling. Importantly, many patients (especially influenza-negative cases) were prescribed anti-influenza drugs that were incongruous with their condition, largely due to physician preference for patient responses to questionnaires and patient symptomology, as opposed to actual viral presence.Conclusions/significanceConcomitant use of SEA-IC and PDM-IC kits increased the likelihood of correct influenza diagnosis. Increasing the credibility of POCT is anticipated to decrease the inappropriate dispensing of anti-influenza drugs, thereby minimizing the emergence of drug-resistant H1N1pdm strains.
| S-EPMC3511324 | biostudies-literature
Project description:BACKGROUND:There is a need for a survey instrument to measure arthralgia (joint pain) that has been psychometrically validated in the context of existing reference instruments. We developed the 16-item Patient-Reported Arthralgia Inventory (PRAI) to measure arthralgia severity in 16 joints, in the context of a longitudinal cohort study to assess aromatase inhibitor-associated arthralgia in breast cancer survivors and arthralgia in postmenopausal women without breast cancer. We sought to evaluate the reliability and validity of the PRAI instrument in these populations, as well as to examine the relationship of patient-reported morning stiffness and arthralgia. METHODS:We administered the PRAI on paper in 294 women (94 initiating aromatase inhibitor therapy and 200 postmenopausal women without breast cancer) at weeks 0, 2, 4, 6, 8, 12, 16, and 52, as well as once in 36 women who had taken but were no longer taking aromatase inhibitor therapy. RESULTS:Cronbach's alpha was 0.9 for internal consistency of the PRAI. Intraclass correlation coefficients of test-retest reliability were in the range of 0.87-0.96 over repeated PRAI administrations; arthralgia severity was higher in the non-cancer group at baseline than at subsequent assessments. Women with joint comorbidities tended to have higher PRAI scores than those without (estimated difference in mean scores: -0.3, 95% confidence interval [CI] -0.5, -0.2; P<0.001). The PRAI was highly correlated with the Functional Assessment of Cancer Therapy-Endocrine Subscale item "I have pain in my joints" (reference instrument; Spearman r range: 0.76-0.82). Greater arthralgia severity on the PRAI was also related to decreased physical function (r=-0.47, 95% CI -0.55, -0.37; P<0.001), higher pain interference (r=0.65, 95% CI 0.57-0.72; P<0.001), less active performance status (estimated difference in location (-0.6, 95% CI -0.9, -0.4; P<0.001), and increased morning stiffness duration (r=0.62, 95% CI 0.54-0.69; P<0.0001). CONCLUSION:We conclude that the psychometric properties of the PRAI are satisfactory for measuring arthralgia severity.
| S-EPMC4524452 | biostudies-literature
Project description:Interventions: Total colonoscopy, FIT and Blood test will be provided once for all participants.
Primary outcome(s): Diagnostic sensitivity, specificity, PPV, NPV and overall accuracy of the FIT and newly developed biomarker for colorectal cancer and precancerous lesions (so called advanced adenoma, >10mm, HGD).
Study Design: Single arm Non-randomized
| 2635555 | ecrin-mdr-crc
Project description:The aim of this study is to develop a postlaryngectomy airway climate explorer (ACE) for assessment of intratracheal temperature and humidity and of influence of heat and moisture exchangers (HMEs). Engineering goals were within-device condensation prevention and fast response time characteristics. The ACE consists of a small diameter, heated air-sampling catheter connected to a heated sensor house, containing a humidity sensor. Air is sucked through the catheter by a controlled-flow pump. Validation was performed in a climate chamber using a calibrated reference sensor and in a two-flow system. Additionally, the analyser was tested in vivo. Over the clinically relevant range of humidity values (5-42 mg H2O/l air) the sensor output highly correlates with the reference sensor readings (R2 > 0.99). The 1-1/e response times are all <0.5 s. A first in vivo pilot measurement was successful. The newly developed, verified, fast-responding ACE is suitable for postlaryngectomy airway climate assessment.
| S-EPMC2039845 | biostudies-other
Project description:Objectives: Antimicrobial resistance (AMR) in clinically relevant bacteria is a growing threat to public health globally. In these bacteria, antimicrobial resistance genes are often associated with mobile genetic elements (MGEs), which promote their mobility, enabling them to rapidly spread throughout a bacterial community.Methods: The tool MobileElementFinder was developed to enable rapid detection of MGEs and their genetic context in assembled sequence data. MGEs are detected based on sequence similarity to a database of 4452 known elements augmented with annotation of resistance genes, virulence factors and detection of plasmids.Results: MobileElementFinder was applied to analyse the mobilome of 1725 sequenced Salmonella enterica isolates of animal origin from Denmark, Germany and the USA. We found that the MGEs were seemingly conserved according to multilocus ST and not restricted to either the host or the country of origin. Moreover, we identified putative translocatable units for specific aminoglycoside, sulphonamide and tetracycline genes. Several putative composite transposons were predicted that could mobilize, among others, AMR, metal resistance and phosphodiesterase genes associated with macrophage survivability. This is, to our knowledge, the first time the phosphodiesterase-like pdeL has been found to be potentially mobilized into S. enterica.Conclusions: MobileElementFinder is a powerful tool to study the epidemiology of MGEs in a large number of genome sequences and to determine the potential for genomic plasticity of bacteria. This web service provides a convenient method of detecting MGEs in assembled sequence data. MobileElementFinder can be accessed at https://cge.cbs.dtu.dk/services/MobileElementFinder/.
| S-EPMC7729385 | biostudies-literature
Project description:With a shift in moving towards the 4th industrial revolution, digital storytelling has been identified as a novel way of facilitating teaching and learning. This paper will be aimed at offering an understanding of the experience and perspective of occupational therapy students in using digital storytelling as a reflective tool as an assignment as part of their undergraduate and masters occupational therapy curriculum at a university in South Africa. A descriptive qualitative study was undertaken, and five participants were purposively recruited. Individual semistructured interviews were conducted as well as a focus group with participants. An inductive analysis revealed two themes: Reflections on relevance within the occupational therapy curriculum and Is technology the new direction? The findings conclude that digital storytelling as a medium to showcase reflections on identifying formation was an innovative and novel way of documenting the reflective experiences of occupational therapy students.
| S-EPMC8460380 | biostudies-literature
Project description:PurposeTo present and evaluate a remote, tool-based system and structured grading rubric for adjudicating image-based diabetic retinopathy (DR) grades.MethodsWe compared three different procedures for adjudicating DR severity assessments among retina specialist panels, including (1) in-person adjudication based on a previously described procedure (Baseline), (2) remote, tool-based adjudication for assessing DR severity alone (TA), and (3) remote, tool-based adjudication using a feature-based rubric (TA-F). We developed a system allowing graders to review images remotely and asynchronously. For both TA and TA-F approaches, images with disagreement were reviewed by all graders in a round-robin fashion until disagreements were resolved. Five panels of three retina specialists each adjudicated a set of 499 retinal fundus images (1 panel using Baseline, 2 using TA, and 2 using TA-F adjudication). Reliability was measured as grade agreement among the panels using Cohen's quadratically weighted kappa. Efficiency was measured as the number of rounds needed to reach a consensus for tool-based adjudication.ResultsThe grades from remote, tool-based adjudication showed high agreement with the Baseline procedure, with Cohen's kappa scores of 0.948 and 0.943 for the two TA panels, and 0.921 and 0.963 for the two TA-F panels. Cases adjudicated using TA-F were resolved in fewer rounds compared with TA (P < 0.001; standard permutation test).ConclusionsRemote, tool-based adjudication presents a flexible and reliable alternative to in-person adjudication for DR diagnosis. Feature-based rubrics can help accelerate consensus for tool-based adjudication of DR without compromising label quality.Translational relevanceThis approach can generate reference standards to validate automated methods, and resolve ambiguous diagnoses by integrating into existing telemedical workflows.
| S-EPMC6922270 | biostudies-literature
Project description:Objective: To develop a nutrient-rich food (NRF) score that captures dietary reference values for older adults and to validate this against a diet index that was specifically designed to assess adherence to dietary guidelines for the older population. Design: A cross-sectional study within the Dutch National Food Consumption Survey (DNFCS, n = 735 men and women aged 70-94 years, enrolled between October 2010 and February 2012) and within the NU-AGE study (n = 250 men and women aged 65-79 years, enrolled between April 2012 and March 2013). Dietary intake was assessed by means of two non-consecutive dietary record assisted 24-h recalls and 7-day food records, respectively. Structured questionnaires collected data on lifestyle and socio-economic information. Anthropometrics were measured by trained dieticians or research assistants. We evaluated Elderly NRF (E-NRF) scores against the NU-AGE index, a measure of adherence to European dietary guidelines for the aging population. The E-NRF scores were composed of nutrients that: (1) have been shown to be of inadequate intake in the aging population (>20%), (2) were defined as nutrients of public health relevance, and (3) were associated with relevant health outcomes. Results: The E-NRF score that best predicted the NU-AGE index included seven nutrients to encourage (protein, dietary fiber, folate, vitamin D, calcium, magnesium, potassium) and three nutrients to limit (saturated fat, sodium and mono- and disaccharides) on a 100-kcal basis, the E-NRF7.3 score (model R2 0.27 in DNFCS and 0.41 in NU-AGE). Food groups contributing the most to the individual E-NRF7.3 scores were vegetables, bread, potatoes and milk and milk products. Conclusion: The E-NRF7.3 score is a useful tool for assessing nutrient density of diets within the older population. No index has previously been developed with the aim of evaluating nutrient density of diets and foods specifically capturing dietary reference values for older adults.
| S-EPMC6688200 | biostudies-literature