Dataset Information

Measuring agreement among experts in classifying camera images of similar species.

ABSTRACT: Camera trapping and solicitation of wildlife images through citizen science have become common tools in ecological research. Such studies collect many wildlife images for which correct species classification is crucial; even low misclassification rates can result in erroneous estimation of the geographic range or habitat use of a species, potentially hindering conservation or management efforts. However, some species are difficult to tell apart, making species classification challenging-but the literature on classification agreement rates among experts remains sparse. Here, we measure agreement among experts in distinguishing between images of two similar congeneric species, bobcats (Lynx rufus) and Canada lynx (Lynx canadensis). We asked experts to classify the species in selected images to test whether the season, background habitat, time of day, and the visible features of each animal (e.g., face, legs, tail) affected agreement among experts about the species in each image. Overall, experts had moderate agreement (Fleiss' kappa = 0.64), but experts had varying levels of agreement depending on these image characteristics. Most images (71%) had ?1 expert classification of "unknown," and many images (39%) had some experts classify the image as "bobcat" while others classified it as "lynx." Further, experts were inconsistent even with themselves, changing their classifications of numerous images when they were asked to reclassify the same images months later. These results suggest that classification of images by a single expert is unreliable for similar-looking species. Most of the images did obtain a clear majority classification from the experts, although we emphasize that even majority classifications may be incorrect. We recommend that researchers using wildlife images consult multiple species experts to increase confidence in their image classifications of similar sympatric species. Still, when the presence of a species with similar sympatrics must be conclusive, physical or genetic evidence should be required.

SUBMITTER: Gooliaff TJ

PROVIDER: S-EPMC6262731 | biostudies-literature | 2018 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Measuring agreement among experts in classifying camera images of similar species.

Gooliaff T J TJ Hodges Karen E KE

Ecology and evolution 20181030 22

Camera trapping and solicitation of wildlife images through citizen science have become common tools in ecological research. Such studies collect many wildlife images for which correct species classification is crucial; even low misclassification rates can result in erroneous estimation of the geographic range or habitat use of a species, potentially hindering conservation or management efforts. However, some species are difficult to tell apart, making species classification challenging-but the ...[more]

PMID: 30519423

Similar Datasets

Project description:Wild animals are essential for ecosystem structuring and stability, and thus they are important for ecological research. Since most wild animals have high athletic or concealable abilities or both, it is used to be relatively difficult to acquire evidence of animal appearances before applications of camera traps in ecological researches. However, a single camera trap may produce thousands of animal images in a short period of time and inevitably ends up with millions of images requiring classification. Although there have been many methods developed for classifying camera trap images, almost all of them follow the pattern of a very deep convolutional neural network processing all camera trap images. Consequently, the corresponding surveillance area may need to be delicately controlled to match the network capability, and it may be difficult to expand the area in the future. In this study, we consider a scenario in which camera traps are grouped into independent clusters, and images produced by a cluster are processed by an edge device installed with a customized network. Accordingly, edge devices in this scenario may be highly heterogeneous due to cluster scales. Resultantly, networks popular in the classification of camera trap images may not be deployable for edge devices without modifications requiring the expertise which may be hard to obtain. This motivates us to automatize network design via neural architecture search for edge devices. However, the search may be costly due to the evaluations of candidate networks, and its results may be infeasible without considering the resource limits of edge devices. Accordingly, we propose a search method using regression trees to evaluate candidate networks to lower search costs, and candidate networks are built based on a meta-architecture automatically adjusted regarding to the resource limits. In experiments, the search consumes 6.5 hours to find a network applicable to the edge device Jetson X2. The found network is then trained on camera trap images through a workstation and tested on Jetson X2. The network achieves competitive accuracies compared with the automatically and the manually designed networks.

Dataset Information

Measuring agreement among experts in classifying camera images of similar species.

Publications

Measuring agreement among experts in classifying camera images of similar species.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets