Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation

Andrew Abel; Amir Hussain; Quoc Dinh Nguyen; Fabien Ringeval; Mohamed Chetouani; Maurice Milgram

doi:10.1007/978-3-642-04391-8_9

Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation

Andrew Abel^*
, Amir Hussain
, Quoc Dinh Nguyen
, Fabien Ringeval
, Mohamed Chetouani
, Maurice Milgram

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

9 Scopus citations

Abstract

In recent years, the established link between the various human communication production domains has become more widely utilised in the field of speech processing. In this work, a state of the art Semi Adaptive Appearance Model (SAAM) approach developed by the authors is used for automatic lip tracking, and an adapted version of our vowel based speech segmentation system is employed to automatically segment speech. Canonical Correlation Analysis (CCA) on segmented and non segmented data in a range of noisy speech environments finds that segmented speech has a significantly better audiovisual correlation, demonstrating the feasibility of our techniques for further development as part of a proposed audiovisual speech enhancement system.

Original language	English
Title of host publication	Biometric ID Management and Multimodal Communication - Joint COST 2101 and 2102 International Conference, BioID_MultiComm 2009, Proceedings
Pages	65-72
Number of pages	8
DOIs	https://doi.org/10.1007/978-3-642-04391-8_9
State	Published - 2009
Externally published	Yes
Event	Joint COST 2101 and 2102 International Conference on Biometric ID Management and Multimodal Communication, BioID_MultiComm 2009 - Madrid, Spain Duration: 16 Sep 2009 → 18 Sep 2009

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	5707 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	Joint COST 2101 and 2102 International Conference on Biometric ID Management and Multimodal Communication, BioID_MultiComm 2009
Country/Territory	Spain
City	Madrid
Period	16/09/09 → 18/09/09

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-642-04391-8_9

Cite this

Abel, A., Hussain, A., Nguyen, Q. D., Ringeval, F., Chetouani, M., & Milgram, M. (2009). Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation. In Biometric ID Management and Multimodal Communication - Joint COST 2101 and 2102 International Conference, BioID_MultiComm 2009, Proceedings (pp. 65-72). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5707 LNCS). https://doi.org/10.1007/978-3-642-04391-8_9

Abel, Andrew ; Hussain, Amir ; Nguyen, Quoc Dinh et al. / Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation. Biometric ID Management and Multimodal Communication - Joint COST 2101 and 2102 International Conference, BioID_MultiComm 2009, Proceedings. 2009. pp. 65-72 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{a7c9967d9c21435a8fdb2a252e57f23d,

title = "Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation",

abstract = "In recent years, the established link between the various human communication production domains has become more widely utilised in the field of speech processing. In this work, a state of the art Semi Adaptive Appearance Model (SAAM) approach developed by the authors is used for automatic lip tracking, and an adapted version of our vowel based speech segmentation system is employed to automatically segment speech. Canonical Correlation Analysis (CCA) on segmented and non segmented data in a range of noisy speech environments finds that segmented speech has a significantly better audiovisual correlation, demonstrating the feasibility of our techniques for further development as part of a proposed audiovisual speech enhancement system.",

author = "Andrew Abel and Amir Hussain and Nguyen, \{Quoc Dinh\} and Fabien Ringeval and Mohamed Chetouani and Maurice Milgram",

year = "2009",

doi = "10.1007/978-3-642-04391-8\_9",

language = "English",

isbn = "3642043909",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

pages = "65--72",

booktitle = "Biometric ID Management and Multimodal Communication - Joint COST 2101 and 2102 International Conference, BioID\_MultiComm 2009, Proceedings",

note = "Joint COST 2101 and 2102 International Conference on Biometric ID Management and Multimodal Communication, BioID\_MultiComm 2009 ; Conference date: 16-09-2009 Through 18-09-2009",

}

Abel, A, Hussain, A, Nguyen, QD, Ringeval, F, Chetouani, M & Milgram, M 2009, Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation. in Biometric ID Management and Multimodal Communication - Joint COST 2101 and 2102 International Conference, BioID_MultiComm 2009, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 5707 LNCS, pp. 65-72, Joint COST 2101 and 2102 International Conference on Biometric ID Management and Multimodal Communication, BioID_MultiComm 2009, Madrid, Spain, 16/09/09. https://doi.org/10.1007/978-3-642-04391-8_9

Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation. / Abel, Andrew; Hussain, Amir; Nguyen, Quoc Dinh et al.
Biometric ID Management and Multimodal Communication - Joint COST 2101 and 2102 International Conference, BioID_MultiComm 2009, Proceedings. 2009. p. 65-72 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 5707 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation

AU - Abel, Andrew

AU - Hussain, Amir

AU - Nguyen, Quoc Dinh

AU - Ringeval, Fabien

AU - Chetouani, Mohamed

AU - Milgram, Maurice

PY - 2009

Y1 - 2009

N2 - In recent years, the established link between the various human communication production domains has become more widely utilised in the field of speech processing. In this work, a state of the art Semi Adaptive Appearance Model (SAAM) approach developed by the authors is used for automatic lip tracking, and an adapted version of our vowel based speech segmentation system is employed to automatically segment speech. Canonical Correlation Analysis (CCA) on segmented and non segmented data in a range of noisy speech environments finds that segmented speech has a significantly better audiovisual correlation, demonstrating the feasibility of our techniques for further development as part of a proposed audiovisual speech enhancement system.

AB - In recent years, the established link between the various human communication production domains has become more widely utilised in the field of speech processing. In this work, a state of the art Semi Adaptive Appearance Model (SAAM) approach developed by the authors is used for automatic lip tracking, and an adapted version of our vowel based speech segmentation system is employed to automatically segment speech. Canonical Correlation Analysis (CCA) on segmented and non segmented data in a range of noisy speech environments finds that segmented speech has a significantly better audiovisual correlation, demonstrating the feasibility of our techniques for further development as part of a proposed audiovisual speech enhancement system.

UR - https://www.scopus.com/pages/publications/77952056171

U2 - 10.1007/978-3-642-04391-8_9

DO - 10.1007/978-3-642-04391-8_9

M3 - Conference contribution

AN - SCOPUS:77952056171

SN - 3642043909

SN - 9783642043901

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 65

EP - 72

BT - Biometric ID Management and Multimodal Communication - Joint COST 2101 and 2102 International Conference, BioID_MultiComm 2009, Proceedings

T2 - Joint COST 2101 and 2102 International Conference on Biometric ID Management and Multimodal Communication, BioID_MultiComm 2009

Y2 - 16 September 2009 through 18 September 2009

ER -

Abel A, Hussain A, Nguyen QD, Ringeval F, Chetouani M, Milgram M. Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation. In Biometric ID Management and Multimodal Communication - Joint COST 2101 and 2102 International Conference, BioID_MultiComm 2009, Proceedings. 2009. p. 65-72. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-642-04391-8_9

Maximising audiovisual correlation with automatic lip tracking and vowel based segmentation

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this