AVSE Challenge: Audio-Visual Speech Enhancement Challenge

Andrea Lorena Aldana Blanco; Cassia Valentini-Botinhao; Ondrej Klejch; Mandar Gogate; Kia Dashtipour; Amir Hussain; Peter Bell

doi:10.1109/SLT54892.2023.10023284

AVSE Challenge: Audio-Visual Speech Enhancement Challenge

Andrea Lorena Aldana Blanco
, Cassia Valentini-Botinhao
, Ondrej Klejch
, Mandar Gogate
, Kia Dashtipour
, Amir Hussain
, Peter Bell

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

21 Scopus citations

Abstract

Audio-visual speech enhancement is the task of improving the quality of a speech signal when video of the speaker is available. It opens-up the opportunity of improving speech intelligibility in adverse listening scenarios that are currently too challenging for audio-only speech enhancement models. The Audio-Visual Speech Enhancement (AVSE) challenge aims to set the first benchmark in this area. We provide participants with datasets and scripts to test their audio-visual speech enhancement models under a common framework for both training and evaluation. The data is derived from real-world videos, and comprises noisy mixes, in which audio from target speaker is mixed with either a competing speaker or a noise signal. The submitted systems are evaluated by conducting AV intelligibility tests involving human participants. We expect this challenge to be a platform for advancing the field of audio-visual speech-enhancement and to provide further insight about the scope and limitations of current AV speech enhancement approaches.

Original language	English
Title of host publication	2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	465-471
Number of pages	7
ISBN (Electronic)	9798350396904
DOIs	https://doi.org/10.1109/SLT54892.2023.10023284
State	Published - 2023
Externally published	Yes
Event	2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Doha, Qatar Duration: 9 Jan 2023 → 12 Jan 2023

Publication series

Name	2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings

Conference

Conference	2022 IEEE Spoken Language Technology Workshop, SLT 2022
Country/Territory	Qatar
City	Doha
Period	9/01/23 → 12/01/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

Audio-visual speech enhancement
LRS3 dataset
subjective intelligibility

ASJC Scopus subject areas

Computer Vision and Pattern Recognition
Hardware and Architecture
Media Technology
Instrumentation
Linguistics and Language

Access to Document

10.1109/SLT54892.2023.10023284

Cite this

Blanco, A. L. A., Valentini-Botinhao, C., Klejch, O., Gogate, M., Dashtipour, K., Hussain, A., & Bell, P. (2023). AVSE Challenge: Audio-Visual Speech Enhancement Challenge. In 2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings (pp. 465-471). (2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SLT54892.2023.10023284

@inproceedings{44a501cc51d6407cae33feda5a0f79d1,

title = "AVSE Challenge: Audio-Visual Speech Enhancement Challenge",

abstract = "Audio-visual speech enhancement is the task of improving the quality of a speech signal when video of the speaker is available. It opens-up the opportunity of improving speech intelligibility in adverse listening scenarios that are currently too challenging for audio-only speech enhancement models. The Audio-Visual Speech Enhancement (AVSE) challenge aims to set the first benchmark in this area. We provide participants with datasets and scripts to test their audio-visual speech enhancement models under a common framework for both training and evaluation. The data is derived from real-world videos, and comprises noisy mixes, in which audio from target speaker is mixed with either a competing speaker or a noise signal. The submitted systems are evaluated by conducting AV intelligibility tests involving human participants. We expect this challenge to be a platform for advancing the field of audio-visual speech-enhancement and to provide further insight about the scope and limitations of current AV speech enhancement approaches.",

keywords = "Audio-visual speech enhancement, LRS3 dataset, subjective intelligibility",

author = "Blanco, \{Andrea Lorena Aldana\} and Cassia Valentini-Botinhao and Ondrej Klejch and Mandar Gogate and Kia Dashtipour and Amir Hussain and Peter Bell",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 2022 IEEE Spoken Language Technology Workshop, SLT 2022 ; Conference date: 09-01-2023 Through 12-01-2023",

year = "2023",

doi = "10.1109/SLT54892.2023.10023284",

language = "English",

series = "2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "465--471",

booktitle = "2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings",

address = "United States",

}

Blanco, ALA, Valentini-Botinhao, C, Klejch, O, Gogate, M, Dashtipour, K, Hussain, A & Bell, P 2023, AVSE Challenge: Audio-Visual Speech Enhancement Challenge. in 2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings. 2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings, Institute of Electrical and Electronics Engineers Inc., pp. 465-471, 2022 IEEE Spoken Language Technology Workshop, SLT 2022, Doha, Qatar, 9/01/23. https://doi.org/10.1109/SLT54892.2023.10023284

AVSE Challenge: Audio-Visual Speech Enhancement Challenge. / Blanco, Andrea Lorena Aldana; Valentini-Botinhao, Cassia; Klejch, Ondrej et al.
2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings. Institute of Electrical and Electronics Engineers Inc., 2023. p. 465-471 (2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - AVSE Challenge

T2 - 2022 IEEE Spoken Language Technology Workshop, SLT 2022

AU - Blanco, Andrea Lorena Aldana

AU - Valentini-Botinhao, Cassia

AU - Klejch, Ondrej

AU - Gogate, Mandar

AU - Dashtipour, Kia

AU - Hussain, Amir

AU - Bell, Peter

PY - 2023

Y1 - 2023

N2 - Audio-visual speech enhancement is the task of improving the quality of a speech signal when video of the speaker is available. It opens-up the opportunity of improving speech intelligibility in adverse listening scenarios that are currently too challenging for audio-only speech enhancement models. The Audio-Visual Speech Enhancement (AVSE) challenge aims to set the first benchmark in this area. We provide participants with datasets and scripts to test their audio-visual speech enhancement models under a common framework for both training and evaluation. The data is derived from real-world videos, and comprises noisy mixes, in which audio from target speaker is mixed with either a competing speaker or a noise signal. The submitted systems are evaluated by conducting AV intelligibility tests involving human participants. We expect this challenge to be a platform for advancing the field of audio-visual speech-enhancement and to provide further insight about the scope and limitations of current AV speech enhancement approaches.

AB - Audio-visual speech enhancement is the task of improving the quality of a speech signal when video of the speaker is available. It opens-up the opportunity of improving speech intelligibility in adverse listening scenarios that are currently too challenging for audio-only speech enhancement models. The Audio-Visual Speech Enhancement (AVSE) challenge aims to set the first benchmark in this area. We provide participants with datasets and scripts to test their audio-visual speech enhancement models under a common framework for both training and evaluation. The data is derived from real-world videos, and comprises noisy mixes, in which audio from target speaker is mixed with either a competing speaker or a noise signal. The submitted systems are evaluated by conducting AV intelligibility tests involving human participants. We expect this challenge to be a platform for advancing the field of audio-visual speech-enhancement and to provide further insight about the scope and limitations of current AV speech enhancement approaches.

KW - Audio-visual speech enhancement

KW - LRS3 dataset

KW - subjective intelligibility

UR - https://www.scopus.com/pages/publications/85147793192

U2 - 10.1109/SLT54892.2023.10023284

DO - 10.1109/SLT54892.2023.10023284

M3 - Conference contribution

AN - SCOPUS:85147793192

T3 - 2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings

SP - 465

EP - 471

BT - 2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings

PB - Institute of Electrical and Electronics Engineers Inc.

Y2 - 9 January 2023 through 12 January 2023

ER -

Blanco ALA, Valentini-Botinhao C, Klejch O, Gogate M, Dashtipour K, Hussain A et al. AVSE Challenge: Audio-Visual Speech Enhancement Challenge. In 2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings. Institute of Electrical and Electronics Engineers Inc. 2023. p. 465-471. (2022 IEEE Spoken Language Technology Workshop, SLT 2022 - Proceedings). doi: 10.1109/SLT54892.2023.10023284

AVSE Challenge: Audio-Visual Speech Enhancement Challenge

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this