Academic Journals Database
Disseminating quality controlled scientific knowledge

A Novel Algorithm for Acoustic and Visual Classifiers Decision Fusion in Audio-Visual Speech Recognition System

Author(s): P.S. Sathidevi | Rajavel

Journal: Signal Processing : An International Journal
ISSN 1985-2339

Volume: 4;
Issue: 1;
Start page: 23;
Date: 2010;
Original page

Keywords: Audio-visual speech recognition | Reliability-ratio based weight optimization | late integration

Audio-visual speech recognition (AVSR) using acoustic and visual signals of speech have received attention recently because of its robustness in noisy environments. Perceptual studies also support this approach by emphasizing the importance of visual information for speech recognition in humans. An important issue in decision fusion based AVSR system is how to obtain the appropriate integration weight for the speech modalities to integrate and ensure the combined AVSR system’s performances better than that of the audio-only and visual-only systems under various noise conditions. To solve this issue, we present a genetic algorithm (GA) based optimization scheme to obtain the appropriate integration weight from the relative reliability of each modality. The performance of the proposed GA optimized reliability-ratio based weight estimation scheme is demonstrated via single speaker, mobile functions isolated word recognition experiments. The results show that the proposed scheme improves robust recognition accuracy over the conventional unimodal systems and the baseline reliability ratio-based AVSR system under various signal to noise ratio conditions.

Tango Jona
Tangokurs Rapperswil-Jona

     Save time & money - Smart Internet Solutions