A Systematic Review of the Reliability and Validity of Behavioural Tests Used to Assess Behavioural Characteristics Important in Working Dogs.
Ontology highlight
ABSTRACT: Background
Working dogs are selected based on predictions from tests that they will be able to perform specific tasks in often challenging environments. However, withdrawal from service in working dogs is still a big problem, bringing into question the reliability of the selection tests used to make these predictions.Methods
A systematic review was undertaken aimed at bringing together available information on the reliability and predictive validity of the assessment of behavioural characteristics used with working dogs to establish the quality of selection tests currently available for use to predict success in working dogs.Results
The search procedures resulted in 16 papers meeting the criteria for inclusion. A large range of behaviour tests and parameters were used in the identified papers, and so behaviour tests and their underpinning constructs were grouped on the basis of their relationship with positive core affect (willingness to work, human-directed social behaviour, object-directed play tendencies) and negative core affect (human-directed aggression, approach withdrawal tendencies, sensitivity to aversives). We then examined the papers for reports of inter-rater reliability, within-session intra-rater reliability, test-retest validity and predictive validity.Conclusions
The review revealed a widespread lack of information relating to the reliability and validity of measures to assess behaviour and inconsistencies in terminologies, study parameters and indices of success. There is a need to standardise the reporting of these aspects of behavioural tests in order to improve the knowledge base of what characteristics are predictive of optimal performance in working dog roles, improving selection processes and reducing working dog redundancy. We suggest the use of a framework based on explaining the direct or indirect relationship of the test with core affect.
SUBMITTER: Brady K
PROVIDER: S-EPMC5982092 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:IntroductionSpeed is an important prerequisite in soccer. Therefore, a large number of tests have been developed aiming to investigate several speed skills relevant to soccer. This systematic review aimed to examine the validity and reliability of speed tests used in adult soccer players.MethodsA systematic search was performed according to the PRISMA guidelines. Studies were included if they investigated speed tests in adult soccer players and reported validity (construct and criterion) or reliability (intraday and interday) data. The tests were categorized into linear-sprint, repeated-sprint, change-of-direction sprint, agility, and tests incorporating combinations of these skills.ResultsIn total, 90 studies covering 167 tests were included. Linear-sprint (n = 67) and change-of-direction sprint (n = 60) were studied most often, followed by combinations of the aforementioned (n = 21) and repeated-sprint tests (n = 15). Agility tests were examined fewest (n = 4). Mainly based on construct validity studies, acceptable validity was reported for the majority of the tests in all categories, except for agility tests, where no validity study was identified. Regarding intraday and interday reliability, ICCs>0.75 and CVs<3.0% were evident for most of the tests in all categories. These results applied for total and average times. In contrast, measures representing fatigue such as percent decrement scores indicated inconsistent validity findings. Regarding reliability, ICCs were 0.11-0.49 and CVs were 16.8-51.0%.ConclusionExcept for agility tests, several tests for all categories with acceptable levels of validity and high levels of reliability for adult soccer players are available. Caution should be given when interpreting fatigue measures, e.g., percent decrement scores. Given the lack of accepted gold-standard tests for each category, researchers and practitioners may base their test selection on the broad database provided in this systematic review. Future research should pay attention to the criterion validity examining the relationship between test results and match parameters as well as to the development and evaluation of soccer-specific agility tests.
| S-EPMC6693781 | biostudies-literature
Project description:UK Biobank is a health resource with data from over 500,000 adults. The cognitive assessment in UK Biobank is brief and bespoke, and is administered without supervision on a touchscreen computer. Psychometric information on the UK Biobank cognitive tests are limited. Despite the non-standard nature of these tests and the limited psychometric information, the UK Biobank cognitive data have been used in numerous scientific publications. The present study examined the validity and short-term test-retest reliability of the UK Biobank cognitive tests. A sample of 160 participants (mean age = 62.59, SD = 10.24) was recruited who completed the UK Biobank cognitive assessment and a range of well-validated cognitive tests ('reference tests'). Fifty-two participants returned 4 weeks later to repeat the UK Biobank tests. Correlations were calculated between UK Biobank tests and reference tests. Two measures of general cognitive ability were created by entering scores on the UK Biobank cognitive tests, and scores on the reference tests, respectively, into separate principal component analyses and saving scores on the first principal component. Four-week test-retest correlations were calculated for UK Biobank tests. UK Biobank cognitive tests showed a range of correlations with their respective reference tests, i.e. those tests that are thought to assess the same underlying cognitive ability (mean Pearson r = 0.53, range = 0.22 to 0.83, p≤.005). The measure of general cognitive ability based on the UK Biobank cognitive tests correlated at r = 0.83 (p < .001) with a measure of general cognitive ability created using the reference tests. Four-week test-retest reliability of the UK Biobank tests were moderate-to-high (mean Pearson r = 0.55, range = 0.40 to 0.89, p≤.003). Despite the brief, non-standard nature of the UK Biobank cognitive tests, some tests showed substantial concurrent validity and test-retest reliability. These psychometric results provide currently-lacking information on the validity of the UK Biobank cognitive tests.
| S-EPMC7170235 | biostudies-literature
Project description:Background: Sleep of pre-school aged children is important for their health and development, but there are currently no standards for measuring sleep in this age group. We aimed to examine the validity, reliability and feasibility of tools used to assess sleep of pre-school aged children. Methods: Studies were eligible for inclusion if they examined the validity and/or reliability and/or feasibility of a measurement tool used to examine sleep of pre-school aged children (aged 3-7 years). We systematically searched six electronic databases, grey literature and trial registries. We manually searched topic specific journals, reference and citations of included studies, and reference lists of existing reviews. We extracted data and conducted a risk of bias assessment on the included studies using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) risk of bias checklist. We used a narrative synthesis to present the results. Results: Sixteen studies met the inclusion criteria: these explored accelerometers (n = 3) and parental reported tools (n = 13; nine questionnaires, six diaries). Studies assessed construct validity (n = 3), criterion validity (n = 1), convergent validity (n = 13), test-retest reliability (n = 2), internal consistency (n = 4) and feasibility (n = 12). Most studies assessed the convergent validity of questionnaires and diaries compared with accelerometers, but the validity of accelerometers for sleep in this age group is unknown. Of studies with a low risk of bias, one sleep diary was shown to be valid for measuring sleep duration. No measurement tools were appropriate for determining sleep quality. Reporting of reliability and feasibility was minimal. Discussion: The evidence base in this field is limited, and most studies had high risk of bias. Future research on sleep in pre-school aged children should focus on assessing the validity, reliability and feasibility of accelerometers, which in turn will improve the quality of studies that assess questionnaires and diaries against accelerometers. Systematic Review Registration: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021230900; PROSPERO: CRD42021230900.
| S-EPMC8662360 | biostudies-literature
Project description:ObjectivesArtistic gymnastics is a sport where most athletes start at an early age and training volumes are high. Hence, overuse and acute injuries are frequent due to the load endured during landing tasks. During landing, the ground reaction force (GRF) is up to 15.8 times the body weight and therefore reliable GRF measurements are crucial. The gold standard for GRF measurements are force plates. As force plates are mostly used in a constrained laboratory environment, it is difficult to measure the GRF in representative training settings. Textile insoles (novel GmbH, Munich, Germany) exist, which can be used to measure dynamic GRF. Hence, the motivation of this study is to test the validity and reliability of these insoles during landing tasks. GRF was measured during four different exercises, in two test subjects and compared to concurrent force plate data.ResultsTwelve out of 16 statistical parametric mapping plots showed no significant difference between the measured force curves of insoles and force plates. Across conditions, the root mean square error of the maximal vertical GRF was 21 N/kg and an impulse 0.4 Ns/kg. The intraclass correlation coefficient (ICC 2,1) ranged from 0.02 to 0.76 for maximal vertical GRF and from - 0.34 to 0.76 for impulse. The insoles are a valid measurement tool for GRF curve progression and impulse during landing but underestimate the maximal vertical GRF.
| S-EPMC10122392 | biostudies-literature
Project description:IntroductionClinicians rely on certain physical examination tests to diagnose and potentially grade ankle sprains and ankle instability. Diagnostic error and inaccurate prognosis may have important repercussions for clinical decision-making and patient outcomes. Therefore, it is important to recognize the diagnostic value of orthopaedic tests through understanding the reliability and validity of these tests.ObjectiveTo systematically review and report evidence on the reliability and validity of orthopaedic tests for the diagnosis of ankle sprains and instability.MethodsPubMed, CINAHL, Scopus, and Cochrane databases were searched from inception to December 2021. In addition, the reference list of included studies, located systematic reviews, and orthopaedic textbooks were searched. All articles reporting reliability or validity of physical examination or orthopaedic tests to diagnose ankle instability or sprains were included. Methodological quality of the reliability and the validity studies was assessed with The Quality Appraisal for Reliability studies checklist and the Quality Assessment of Diagnostic Accuracy Studies-2 respectively. We identified the number of times the orthopaedic test was investigated and the validity and/or reliability of each test.ResultsOverall, sixteen studies were included. Three studies assessed reliability, eight assessed validity, and five evaluated both. Overall, fifteen tests were evaluated, none demonstrated robust reliability and validity scores. The anterolateral talar palpation test reported the highest diagnostic accuracy. Further, the anterior drawer test, the anterolateral talar palpation, the reverse anterior lateral drawer test, and palpation of the anterior talofibular ligament reported the highest sensitivity. The highest specificity was attributed to the anterior drawer test, the anterolateral drawer test, the reverse anterior lateral drawer test, tenderness on palpation of the proximal fibular, and the squeeze test.ConclusionOverall, the diagnostic accuracy, reliability, and validity of physical examination tests for the assessment of ankle instability were limited. Physical examination tests should not be used in isolation, but rather in combination with the clinical history to diagnose an ankle sprain. Preliminary evidence suggests that the overall validity of physical examination for the ankle may be better if conducted five days after the injury rather than within 48 h of injury.
| S-EPMC9764698 | biostudies-literature
Project description:ObjectiveChild and youth mental health problems are often assessed by parent self-completed checklists that produce dimensional scale scores. When converted to binary ratings of disorder, little is known about their psychometric properties in relation to classifications based on lay-administered structured diagnostic interviews. In addition to estimating agreement, our objective is to test for statistical equivalence in the test-retest reliability and construct validity of two instruments used to classify child emotional, behavioural, and attentional disorders: the 25-item, parent completed Ontario Child Health Study Emotional Behavioural Scales-Brief Version (OCHS-EBS-B) and the Mini International Neuropsychiatric Interview for Children and Adolescents-parent version (MINI-KID-P).MethodsThis study draws on independent samples (n = 452) and uses the confidence interval approach to test for statistical equivalence. Reliability is based on kappa (κ). Construct validity is based on standardized beta coefficients (β) estimated in structural equation models.ResultsThe average differences between the MINI-KID-P and OCHS-EBS-B in κ and β were -0.022 and -0.020, respectively. However, in both instances, criteria for statistical equivalence were met in only 5 of 12 comparisons. Based on κ, between-instrument agreement on the classifications of disorder went from 0.481 (attentional disorder) to 0.721 (emotional disorder) but were substantially higher (0.731 to 0.895, respectively) when corrected for attenuation due to measurement error.ConclusionsAlthough falling short of equivalence, the results suggest on balance that the reliability and validity of the two instruments for classifying child psychiatric disorder assessed by parents are highly comparable. This conclusion is supported by the high levels of agreement between the instruments after correcting for attenuation due to measurement error.
| S-EPMC9014671 | biostudies-literature
Project description:Background:The SARC-F questionnaire has been developed as a rapid diagnostic test that can be used to screen for sarcopenia. Aim:To investigate the reliability and validity of the Vietnamese version of SARC-F as a screening tool for sarcopenia in older patients in Vietnam. Methods:A cross-sectional study was conducted in older people attending the outpatient clinics of the National Geriatric Hospital in Vietnam. Muscle mass (using dual-energy X-ray absorptiometry), handgrip strength and gait speed were assessed. SARC-F was validated against the three standard criteria for sarcopenia: the Foundation for the National Institutes of Health (FNIH), Asia Working Group for Sarcopenia (AWGS 2019) and European Working Group on Sarcopenia in Older People (EWGSOP2). Results:There were 764 participants (mean age 71.5 ± 8.9 years). The Vietnamese SARC-F questionnaire had a good internal consistency (Cronbach's alpha 0.85). The prevalence of sarcopenia was 49.2% according to SARC-F and 48.3%, 61.1% and 52.6% according to FNIH, AWGS 2019 and EWGSOP2, respectively. The sensitivity and specificity of SARC-F in identifying sarcopenia were 67.1% and 66.7% (for FINH), 66.7% and 67.1% (for AWGS 2019), and 64.9% and 68.2% (for EWGSOP2). The AUCs of SARC-F were 0.71-0.72 against the three sarcopenia criteria. Conclusion:The Vietnamese version of SARC-F questionnaire has acceptable diagnostic value for sarcopenia. SARC-F could be used as an initial screening for sarcopenia in hospital clinics.
| S-EPMC7293385 | biostudies-literature
Project description:BackgroundMany treatments aim to slow down or reverse the visible signs of skin aging and thereby improve skin quality. Measurement devices are frequently employed to measure the effects of these treatments to improve skin quality, for example, skin elasticity, color, and texture. However, it remains unknown which of these devices is most reliable and valid.Materials and methodsMEDLINE, Embase, Cochrane Central, Web of Science, and Google Scholar databases were searched. Instruments were scored on reporting construct validity by means of convergent validity, interobserver, intraobserver, and interinstrument reliability.ResultsFor the evaluation of skin color, 11 studies were included describing 16 measurement devices, analyzing 3172 subjects. The most reliable device for skin color assessment is the Minolta Chromameter CR-300 due to good interobserver, intraobserver, and interinstrument reliability. For skin elasticity, seven studies assessed nine types of devices analyzing 290 subjects in total. No intra and interobserver reliability was reported. Skin texture was assessed in two studies evaluating 72 subjects using three different types of measurement devices. The PRIMOS device reported excellent intra and interobserver reliability. None of the included reviewed devices could be determined to be valid based on construct validity.ConclusionThe most reliable devices to evaluate skin color and texture in ordinary skin were, respectively, the Minolta Chromameter and PRIMOS. No reliable device is available to measure skin elasticity in ordinary skin and none of the included devices could be determined to be designated as valid.
| S-EPMC9299221 | biostudies-literature
Project description:Physical activity (PA) and sedentary behaviour (SB) of pre-school aged children are associated with important health and developmental outcomes. Accurate measurement of these behaviours in young children is critical for research and practice in this area. The aim of this review was to examine the validity, reliability, and feasibility of measurement tools used to assess PA and SB of pre-school aged children. Searches of electronic databases, and manual searching, were conducted to identify articles that examined the measurement properties (validity, reliability or feasibility) of measurement tools used to examine PA and/or SB of pre-school aged children (3–7 years old). Following screening, data were extracted and risk of bias assessment completed on all included articles. A total of 69 articles, describing 75 individual studies were included. Studies assessed measurement tools for PA (n = 27), SB (n = 5), and both PA and SB (n = 43). Outcome measures of PA and SB differed between studies (e.g. moderate to vigorous activity, step count, posture allocation). Most studies examined the measurement properties of one measurement tool only (n = 65). Measurement tools examined included: calorimetry, direct observation, combined heart rate and accelerometry, heart rate monitors, accelerometers, pedometers, and proxy report (parent, carer or teacher reported) measures (questionnaires or diaries). Studies most frequently assessed the validity (criterion and convergent) (n = 65), face and content validity (n = 2), test-retest reliability (n = 10) and intra-instrument reliability (n = 1) of the measurement tools. Feasibility data was abstracted from 41 studies. Multiple measurement tools used to measure PA and SB in pre-school aged children showed some degree of validity, reliability and feasibility, but often for different purposes. Accelerometers, including the Actigraph (in particular GT3X versions), Actical, ActivPAL and Fitbit (Flex and Zip), and proxy reported measurement tools used in combination may be useful for a range of outcome measures, to measure intensity alongside contextual information. Supplementary Information The online version contains supplementary material available at 10.1186/s12966-021-01132-9.
| S-EPMC8567581 | biostudies-literature
Project description:BackgroundThere is an increased risk of subsequent concussion and musculoskeletal injury upon return to play following a sports-related concussion. Whilst there are numerous assessments available for clinicians for diagnosis and during return to play following concussion, many may lack the ability to detect these subclinical changes in function. Currently, there is no consensus or collated sources on the reliability, validity and feasibility of these assessments, which makes it difficult for clinicians and practitioners to select the most appropriate assessment for their needs.ObjectivesThis systematic review aims to (1) consolidate the reliability and validity of motor function assessments across the time course of concussion management and (2) summarise their feasibility for clinicians and other end-users.MethodsA systematic search of five databases was conducted. Eligible studies were: (1) original research; (2) full-text English language; (3) peer-reviewed with level III evidence or higher; (4) assessed the validity of lower-limb motor assessments used to diagnose or determine readiness for athletes or military personnel who had sustained a concussion or; (5) assessed the test-retest reliability of lower-limb motor assessments used for concussion management amongst healthy athletes. Acceptable lower-limb motor assessments were dichotomised into instrumented and non-instrumented and then classified into static (stable around a fixed point), dynamic (movement around a fixed point), gait, and other categories. Each study was assessed using the COSMIN checklist to establish methodological and measurement quality.ResultsA total of 1270 records were identified, with 637 duplicates removed. Titles and abstracts of 633 records were analysed, with 158 being retained for full-text review. A total of 67 records were included in this review; 37 records assessed reliability, and 35 records assessed the validity of lower-limb motor assessments. There were 42 different assessments included in the review, with 43% being non-instrumented, subjective assessments. Consistent evidence supported the use of instrumented assessments over non-instrumented, with gait-based assessments demonstrating sufficient reliability and validity compared to static or dynamic assessments.ConclusionThese findings suggest that instrumented, gait-based assessments should be prioritised over static or dynamic balance assessments. The use of laboratory equipment (i.e. 3D motion capture, pressure sensitive walkways) on average exhibited sufficient reliability and validity, yet demonstrate poor feasibility. Further high-quality studies evaluating the reliability and validity of more readily available devices (i.e. inertial measurement units) are needed to fill the gap in current concussion management protocols. Practitioners can use this resource to understand the accuracy and precision of the assessments they have at their disposal to make informed decisions regarding the management of concussion.Trail registrationThis systematic review was registered on PROSPERO (reg no. CRD42021256298).
| S-EPMC10425315 | biostudies-literature