Dataset Information

Brain-inspired speech segmentation for automatic speech recognition using the speech envelope as a temporal reference.

ABSTRACT: Speech segmentation is a crucial step in automatic speech recognition because additional speech analyses are performed for each framed speech segment. Conventional segmentation techniques primarily segment speech using a fixed frame size for computational simplicity. However, this approach is insufficient for capturing the quasi-regular structure of speech, which causes substantial recognition failure in noisy environments. How does the brain handle quasi-regular structured speech and maintain high recognition performance under any circumstance? Recent neurophysiological studies have suggested that the phase of neuronal oscillations in the auditory cortex contributes to accurate speech recognition by guiding speech segmentation into smaller units at different timescales. A phase-locked relationship between neuronal oscillation and the speech envelope has recently been obtained, which suggests that the speech envelope provides a foundation for multi-timescale speech segmental information. In this study, we quantitatively investigated the role of the speech envelope as a potential temporal reference to segment speech using its instantaneous phase information. We evaluated the proposed approach by the achieved information gain and recognition performance in various noisy environments. The results indicate that the proposed segmentation scheme not only extracts more information from speech but also provides greater robustness in a recognition test.

SUBMITTER: Lee B

PROVIDER: S-EPMC5120313 | biostudies-literature | 2016 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Brain-inspired speech segmentation for automatic speech recognition using the speech envelope as a temporal reference.

Lee Byeongwook B Cho Kwang-Hyun KH

Scientific reports 20161123

Speech segmentation is a crucial step in automatic speech recognition because additional speech analyses are performed for each framed speech segment. Conventional segmentation techniques primarily segment speech using a fixed frame size for computational simplicity. However, this approach is insufficient for capturing the quasi-regular structure of speech, which causes substantial recognition failure in noisy environments. How does the brain handle quasi-regular structured speech and maintain h ...[more]

PMID: 27876875

Similar Datasets

Project description:The speech unit segmentation is an important pre-processing step in the analysis of cleft palate speech. In Mandarin, one syllable is composed of two parts: initial and final. In cleft palate speech, the resonance disorders occur at the finals and the voiced initials, while the articulation disorders occur at the unvoiced initials. Thus, the initials and finals are the minimum speech units, which could reflect the characteristics of cleft palate speech disorders. In this work, an automatic initial/final segmentation method is proposed. It is an important preprocessing step in cleft palate speech signal processing. The tested cleft palate speech utterances are collected from the Cleft Palate Speech Treatment Center in the Hospital of Stomatology, Sichuan University, which has the largest cleft palate patients in China. The cleft palate speech data includes 824 speech segments, and the control samples contain 228 speech segments. The syllables are extracted from the speech utterances firstly. The proposed syllable extraction method avoids the training stage, and achieves a good performance for both voiced and unvoiced speech. Then, the syllables are classified into with "quasi-unvoiced" or with "quasi-voiced" initials. Respective initial/final segmentation methods are proposed to these two types of syllables. Moreover, a two-step segmentation method is proposed. The rough locations of syllable and initial/final boundaries are refined in the second segmentation step, in order to improve the robustness of segmentation accuracy. The experiments show that the initial/final segmentation accuracies for syllables with quasi-unvoiced initials are higher than quasi-voiced initials. For the cleft palate speech, the mean time error is 4.4ms for syllables with quasi-unvoiced initials, and 25.7ms for syllables with quasi-voiced initials, and the correct segmentation accuracy P30 for all the syllables is 91.69%. For the control samples, P30 for all the syllables is 91.24%.

Project description:PurposeThis study aimed to determine if people living with HIV (PLWH) in preclinical human immunodeficiency virus (HIV)-associated neurocognitive disorder (HAND), with no clinical symptoms and without decreased daily functioning, suffer from brain volumetric alterations and its patterns.MethodFifty-nine male PLWH at the HAND preclinical stage were evaluated, including 19 subjects with asymptomatic neurocognitive impairment (ANI), 17 subjects with cognitive abnormality that does not reach ANI (Not reach ANI), and 23 subjects with cognitive integrity. Moreover, 23 healthy volunteers were set as the seronegative normal controls (NCs). These individuals underwent sagittal three-dimensional T1-weighted imaging (3D T1WI). Quantified data and volumetric measures of brain structures were automatically segmented and extracted using AccuBrain®. In addition, the multiple linear regression analysis was performed to analyze the relationship of volumes of brain structures and clinical variables in preclinical HAND, and the correlations of the brain volume parameters with different cognitive function states were assessed by Pearson's correlation analysis.ResultsThe significant difference was shown in the relative volumes of the ventricular system, bilateral lateral ventricle, thalamus, caudate, and left parietal lobe gray matter between the preclinical HAND and NCs. Furthermore, the relative volumes of the bilateral thalamus in preclinical HAND were negatively correlated with attention/working memory (left: r = -0.271, p = 0.042; right: r = -0.273, p = 0.040). Higher age was associated with increased relative volumes of the bilateral lateral ventricle and ventricular system and reduced relative volumes of the left thalamus and parietal lobe gray matter. The lower CD4+/CD8+ ratio was associated with increased relative volumes of the left lateral ventricle and ventricular system. Longer disease course was associated with increased relative volumes of the bilateral thalamus. No significant difference was found among preclinical HAND subgroups in all indices, and the difference between the individual groups (Not reach ANI and Cognitive integrity groups) and NCs was also insignificant. However, there was a significant difference between ANI and NCs in the relative volumes of the bilateral caudate and lateral ventricle.ConclusionMale PLWH at the HAND preclinical stage suffer from brain volumetric alterations. AccuBrain® provides potential value in evaluating HIV-related neurocognitive dysfunction.

Project description:Animals can perform complex and purposeful behaviors by executing simpler movements in flexible sequences. It is particularly challenging to analyze behavior sequences when they are highly variable, as is the case in language production, certain types of birdsong and, as in our experiments, flies grooming. High sequence variability necessitates rigorous quantification of large amounts of data to identify organizational principles and temporal structure of such behavior. To cope with large amounts of data, and minimize human effort and subjective bias, researchers often use automatic behavior recognition software. Our standard grooming assay involves coating flies in dust and videotaping them as they groom to remove it. The flies move freely and so perform the same movements in various orientations. As the dust is removed, their appearance changes. These conditions make it difficult to rely on precise body alignment and anatomical landmarks such as eyes or legs and thus present challenges to existing behavior classification software. Human observers use speed, location, and shape of the movements as the diagnostic features of particular grooming actions. We applied this intuition to design a new automatic behavior recognition system (ABRS) based on spatiotemporal features in the video data, heavily weighted for temporal dynamics and invariant to the animal's position and orientation in the scene. We use these spatiotemporal features in two steps of supervised classification that reflect two time-scales at which the behavior is structured. As a proof of principle, we show results from quantification and analysis of a large data set of stimulus-induced fly grooming behaviors that would have been difficult to assess in a smaller dataset of human-annotated ethograms. While we developed and validated this approach to analyze fly grooming behavior, we propose that the strategy of combining alignment-invariant features and multi-timescale analysis may be generally useful for movement-based classification of behavior from video data.

Dataset Information

Brain-inspired speech segmentation for automatic speech recognition using the speech envelope as a temporal reference.

Publications

Brain-inspired speech segmentation for automatic speech recognition using the speech envelope as a temporal reference.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets