Dataset Information

Operator bias in software-aided bat call identification.

ABSTRACT: Software-aided identification facilitates the handling of large sets of bat call recordings, which is particularly useful in extensive acoustic surveys with several collaborators. Species lists are generated by "objective" automated classification. Subsequent validation consists of removing any species not believed to be present. So far, very little is known about the identification bias introduced by individual validation of operators with varying degrees of experience. Effects on the quality of the resulting data may be considerable, especially for bat species that are difficult to identify acoustically. Using the batcorder system as an example, we compared validation results from 21 volunteer operators with 1-26 years of experience of working on bats. All of them validated identical recordings of bats from eastern Austria. The final outcomes were individual validated lists of plausible species. A questionnaire was used to enquire about individual experience and validation procedures. In the course of species validation, the operators reduced the software's estimate of species richness. The most experienced operators accepted the smallest percentage of species from the software's output and validated conservatively with low interoperator variability. Operators with intermediate experience accepted the largest percentage, with larger variability. Sixty-six percent of the operators, mainly with intermediate and low levels of experience, reintroduced species to their validated lists which had been identified by the automated classification, but were finally excluded from the unvalidated lists. These were, in many cases, rare and infrequently recorded species. The average dissimilarity of the validated species lists dropped with increasing numbers of recordings, tending toward a level of ˜20%. Our results suggest that the operators succeeded in removing false positives and that they detected species that had been wrongly excluded during automated classification. Thus, manual validation of the software's unvalidated output is indispensable for reasonable results. However, although application seems easy, software-aided bat call identification requires an advanced level of operator experience. Identification bias during validation is a major issue, particularly in studies with more than one participant. Measures should be taken to standardize the validation process and harmonize the results of different operators.

SUBMITTER: Fritsch G

PROVIDER: S-EPMC4113294 | biostudies-literature | 2014 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Operator bias in software-aided bat call identification.

Fritsch Georg G Bruckner Alexander A

Ecology and evolution 20140530 13

Software-aided identification facilitates the handling of large sets of bat call recordings, which is particularly useful in extensive acoustic surveys with several collaborators. Species lists are generated by "objective" automated classification. Subsequent validation consists of removing any species not believed to be present. So far, very little is known about the identification bias introduced by individual validation of operators with varying degrees of experience. Effects on the quality o ...[more]

PMID: 25077021

Similar Datasets

Project description:Quantitative methods for species identification are commonly used in acoustic surveys for animals. While various identification models have been studied extensively, there has been little study of methods for selecting calls prior to modeling or methods for validating results after modeling. We obtained two call libraries with a combined 1556 pulse sequences from 11 North American bat species. We used four acoustic filters to automatically select and quantify bat calls from the combined library. For each filter, we trained a species identification model (a quadratic discriminant function analysis) and compared the classification ability of the models. In a separate analysis, we trained a classification model using just one call library. We then compared a conventional model assessment that used the training library against an alternative approach that used the second library. We found that filters differed in the share of known pulse sequences that were selected (68 to 96%), the share of non-bat noises that were excluded (37 to 100%), their measurement of various pulse parameters, and their overall correct classification rate (41% to 85%). Although the top two filters did not differ significantly in overall correct classification rate (85% and 83%), rates differed significantly for some bat species. In our assessment of call libraries, overall correct classification rates were significantly lower (15% to 23% lower) when tested on the second call library instead of the training library. Well-designed filters obviated the need for subjective and time-consuming manual selection of pulses. Accordingly, researchers should carefully design and test filters and include adequate descriptions in publications. Our results also indicate that it may not be possible to extend inferences about model accuracy beyond the training library. If so, the accuracy of acoustic-only surveys may be lower than commonly reported, which could affect ecological understanding or management decisions based on acoustic surveys.

Project description:Concerns associated with the assessment of aesthetic damage or injury raise critical difficulties, such as the scarcity of methodology and standardization that may result in fundamental precepts to establish impartial forms of compensation and aiming the total reparation of bodily injury. The complexity of the aesthetic damage evaluation is associated with the confluence of legal and technical perspectives and expert subjectivity while conducting examination and writing a report. Experts face additional difficulties associated with the objectivity while assessing aesthetic damage, independently on its location or expert skills, due to complex details observed in these lesions. Another situation in the clinical area, doctors (mainly plastic surgeons) and dentists could show the improvement or not, of the aesthetic condition to the patients. In health related areas, the use of information technology has contributed to increase the number of appropriate diagnoses, besides promoting quality, efficiency and satisfaction to health care providers. In order to make this assessment more objective, a technological tool was developed to aid experts in the evaluation of aesthetic damage and report elaboration. The objective was to develop computer-aided design software for aesthetic damage quantification/evaluation that is accessible via internet to be applied as a complementary report on body aesthetic damage. The software uses as a parameter the AIPE method, translated transculturally from Spanish to Portuguese and English. The present study allowed the construction of open access auxiliary software for the evaluation of corporal aesthetic damage. Its use is facilitated by intuitive and interactive filling, and the text may be customized by the user. It transforms the report into PDF and saves all evaluations already done in its own file. Information is encrypted for added security and confidentiality. The software is available on website at https://www.aestheticdamage.com.

Dataset Information

Operator bias in software-aided bat call identification.

Publications

Operator bias in software-aided bat call identification.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets