Dataset Information

An evaluation tool for myofascial adhesions in patients after breast cancer (MAP-BC evaluation tool): Development and interrater reliability.

ABSTRACT:

Purpose

To develop a tool to evaluate myofascial adhesions objectively in patients with breast cancer and to investigate its interrater reliability.

Methods

1) Development of the evaluation tool. Literature was searched, experts in the field of myofascial therapy were consulted and pilot testing was performed. 2) Thirty patients (63% had a mastectomy, 37% breast-conserving surgery and 97% radiotherapy) with myofascial adhesions were evaluated using the developed tool by 2 independent raters. The Weighted Kappa (WK) and the intra-class correlation coefficient (ICC) were calculated.

Results

1) The evaluation tool for Myofascial Adhesions in Patients with Breast Cancer (MAP-BC evaluation tool) consisted of the assessment of myofascial adhesions at 7 locations: axillary and breast region scars, musculi pectorales region, axilla, frontal chest wall, lateral chest wall and the inframammary fold. At each location the degree of the myofascial adhesion was scored at three levels (skin, superficial and deep) on a 4-points scale (between no adhesions and very stiff adhesions). Additionally, a total score (0-9) was calculated, i.e. the sum of the different levels of each location. 2) Interrater agreement of the different levels separately was moderate for the axillary and mastectomy scar (WK 0.62-0.73) and good for the scar on the breast (WK >0.75). Moderate agreement was reached for almost all levels of the non-scar locations. Interrater reliability of the total scores was the highest for the scars (ICC 0.82-0.99). At non-scar locations good interrater reliability was reached, except for the inframammary fold (ICC = 0.71).

Conclusions

The total scores of all locations of the MAP-BC evaluation tool had good to excellent interrater reliability, except for the inframammary fold which only reached moderate reliability.

SUBMITTER: De Groef A

PROVIDER: S-EPMC5466317 | biostudies-literature | 2017

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

An evaluation tool for myofascial adhesions in patients after breast cancer (MAP-BC evaluation tool): Development and interrater reliability.

De Groef An A Van Kampen Marijke M Vervloesem Nele N De Geyter Sophie S Dieltjens Evi E Christiaens Marie-Rose MR Neven Patrick P Geraerts Inge I Devoogdt Nele N

PloS one 20170609 6

<h4>Purpose</h4>To develop a tool to evaluate myofascial adhesions objectively in patients with breast cancer and to investigate its interrater reliability.<h4>Methods</h4>1) Development of the evaluation tool. Literature was searched, experts in the field of myofascial therapy were consulted and pilot testing was performed. 2) Thirty patients (63% had a mastectomy, 37% breast-conserving surgery and 97% radiotherapy) with myofascial adhesions were evaluated using the developed tool by 2 independ ...[more]

PMID: 28598978

Similar Datasets

Project description:BackgroundInterrater reliability, aka intercoder reliability, is defined as true agreement between raters, aka coders, without chance agreement. It is used across many disciplines including medical and health research to measure the quality of ratings, coding, diagnoses, or other observations and judgements. While numerous indices of interrater reliability are available, experts disagree on which ones are legitimate or more appropriate. Almost all agree that percent agreement (ao), the oldest and the simplest index, is also the most flawed because it fails to estimate and remove chance agreement, which is produced by raters' random rating. The experts, however, disagree on which chance estimators are legitimate or better. The experts also disagree on which of the three factors, rating category, distribution skew, or task difficulty, an index should rely on to estimate chance agreement, or which factors the known indices in fact rely on. The most popular chance-adjusted indices, according to a functionalist view of mathematical statistics, assume that all raters conduct intentional and maximum random rating while typical raters conduct involuntary and reluctant random rating. The mismatches between the assumed and the actual rater behaviors cause the indices to rely on mistaken factors to estimate chance agreement, leading to the numerous paradoxes, abnormalities, and other misbehaviors of the indices identified by prior studies.MethodsWe conducted a 4 × 8 × 3 between-subject controlled experiment with 4 subjects per cell. Each subject was a rating session with 100 pairs of rating by two raters, totaling 384 rating sessions as the experimental subjects. The experiment tested seven best-known indices of interrater reliability against the observed reliabilities and chance agreements. Impacts of the three factors, i.e., rating category, distribution skew, and task difficulty, on the indices were tested.ResultsThe most criticized index, percent agreement (ao), showed as the most accurate predictor of reliability, reporting directional r2 = .84. It was also the third best approximator, overestimating observed reliability by 13 percentage points on average. The three most acclaimed and most popular indices, Scott's π, Cohen's κ and Krippendorff's α, underperformed all other indices, reporting directional r2 = .312 and underestimated reliability by 31.4 ~ 31.8 points. The newest index, Gwet's AC1, emerged as the second-best predictor and the most accurate approximator. Bennett et al's S ranked behind AC1, and Perreault and Leigh's Ir ranked the fourth both for prediction and approximation. The reliance on category and skew and failure to rely on difficulty explain why the six chance-adjusted indices often underperformed ao, which they were created to outperform. The evidence corroborated the notion that the chance-adjusted indices assume intentional and maximum random rating while the raters instead exhibited involuntary and reluctant random rating.ConclusionThe authors call for more empirical studies and especially more controlled experiments to falsify or qualify this study. If the main findings are replicated and the underlying theories supported, new thinking and new indices may be needed. Index designers may need to refrain from assuming intentional and maximum random rating, and instead assume involuntary and reluctant random rating. Accordingly, the new indices may need to rely on task difficulty, rather than distribution skew or rating category, to estimate chance agreement.

Project description:BackgroundLearning strategies are thinking tools that students can use to actively acquire information. Examples of learning strategies include mnemonics, charts, and maps. One strategy that may help students master the tsunami of information presented in medical school is the mind map learning strategy. Currently, there is no valid and reliable rubric to grade mind maps and this may contribute to their underutilization in medicine. Because concept maps and mind maps engage learners similarly at a metacognitive level, a valid and reliable concept map assessment scoring system was adapted to form the mind map assessment rubric (MMAR). The MMAR can assess mind map depth based upon concept-links, cross-links, hierarchies, examples, pictures, and colors. The purpose of this study was to examine interrater reliability of the MMAR.MethodsThis exploratory study was conducted at a US medical school as part of a larger investigation on learning strategies. Sixty-six (N = 66) first-year medical students were given a 394-word text passage followed by a 30-minute presentation on mind mapping. After the presentation, subjects were again given the text passage and instructed to create mind maps based upon the passage. The mind maps were collected and independently scored using the MMAR by 3 examiners. Interrater reliability was measured using the intraclass correlation coefficient (ICC) statistic. Statistics were calculated using SPSS version 12.0 (Chicago, IL).ResultsAnalysis of the mind maps revealed the following: concept-links ICC = .05 (95% CI, -.42 to .38), cross-links ICC = .58 (95% CI, .37 to .73), hierarchies ICC = .23 (95% CI, -.15 to .50), examples ICC = .53 (95% CI, .29 to .69), pictures ICC = .86 (95% CI, .79 to .91), colors ICC = .73 (95% CI, .59 to .82), and total score ICC = .86 (95% CI, .79 to .91).ConclusionThe high ICC value for total mind map score indicates strong MMAR interrater reliability. Pictures and colors demonstrated moderate to strong interrater reliability. We conclude that the MMAR may be a valid and reliable tool to assess mind maps in medicine. However, further research on the validity and reliability of the MMAR is necessary.

Project description:BackgroundThe extensive availability and increasing use of mobile apps for nutrition-based health interventions makes evaluation of the quality of these apps crucial for integration of apps into nutritional counseling.ObjectiveThe goal of this research was the development, validation, and reliability testing of the app quality evaluation (AQEL) tool, an instrument for evaluating apps' educational quality and technical functionality.MethodsItems for evaluating app quality were adapted from website evaluations, with additional items added to evaluate the specific characteristics of apps, resulting in 79 initial items. Expert panels of nutrition and technology professionals and app users reviewed items for face and content validation. After recommended revisions, nutrition experts completed a second AQEL review to ensure clarity. On the basis of 150 sets of responses using the revised AQEL, principal component analysis was completed, reducing AQEL into 5 factors that underwent reliability testing, including internal consistency, split-half reliability, test-retest reliability, and interrater reliability (IRR). Two additional modifiable constructs for evaluating apps based on the age and needs of the target audience as selected by the evaluator were also tested for construct reliability. IRR testing using intraclass correlations (ICC) with all 7 constructs was conducted, with 15 dietitians evaluating one app.ResultsDevelopment and validation resulted in the 51-item AQEL. These were reduced to 25 items in 5 factors after principal component analysis, plus 9 modifiable items in two constructs that were not included in principal component analysis. Internal consistency and split-half reliability of the following constructs derived from principal components analysis was good (Cronbach alpha >.80, Spearman-Brown coefficient >.80): behavior change potential, support of knowledge acquisition, app function, and skill development. App purpose split half-reliability was .65. Test-retest reliability showed no significant change over time (P>.05) for all but skill development (P=.001). Construct reliability was good for items assessing age appropriateness of apps for children, teens, and a general audience. In addition, construct reliability was acceptable for assessing app appropriateness for various target audiences (Cronbach alpha >.70). For the 5 main factors, ICC (1,k) was >.80, with a P value of <.05. When 15 nutrition professionals evaluated one app, ICC (2,15) was .98, with a P value of <.001 for all 7 constructs when the modifiable items were specified for adults seeking weight loss support.ConclusionsOur preliminary effort shows that AQEL is a valid, reliable instrument for evaluating nutrition apps' qualities for clinical interventions by nutrition clinicians, educators, and researchers. Further efforts in validating AQEL in various contexts are needed.

Dataset Information

An evaluation tool for myofascial adhesions in patients after breast cancer (MAP-BC evaluation tool): Development and interrater reliability.

Purpose

Methods

Results

Conclusions

Publications

An evaluation tool for myofascial adhesions in patients after breast cancer (MAP-BC evaluation tool): Development and interrater reliability.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets