OfGAN: Realistic Rendition of Synthetic Colonoscopy Videos

Jiabo Xu; Saeed Anwar; Nick Barnes; Florian Grimpen; Olivier Salvado; Stuart Anderson; Mohammad Ali Armin

doi:10.1007/978-3-030-59716-0_70

OfGAN: Realistic Rendition of Synthetic Colonoscopy Videos

Jiabo Xu^*, Saeed Anwar, Nick Barnes, Florian Grimpen, Olivier Salvado, Stuart Anderson, Mohammad Ali Armin

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

7 Scopus citations

Abstract

Data-driven methods usually require a large amount of labelled data for training and generalization, especially in medical imaging. Targeting the colonoscopy field, we develop the Optical Flow Generative Adversarial Network (OfGAN) to transform simulated colonoscopy videos into realistic ones while preserving annotation. The advantages of our method are three-fold: the transformed videos are visually much more realistic; the annotation, such as optical flow of the source video is preserved in the transformed video, and it is robust to noise. The model uses a cycle-consistent structure and optical flow for both spatial and temporal consistency via adversarial training. We demonstrate that the performance of our OfGAN overwhelms the baseline method in relative tasks through both qualitative and quantitative evaluation.

Original language	English
Title of host publication	Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings
Editors	Anne L. Martel, Purang Abolmaesumi, Danail Stoyanov, Diana Mateus, Maria A. Zuluaga, S. Kevin Zhou, Daniel Racoceanu, Leo Joskowicz
Publisher	Springer Science and Business Media Deutschland GmbH
Pages	732-741
Number of pages	10
ISBN (Print)	9783030597153
DOIs	https://doi.org/10.1007/978-3-030-59716-0_70
State	Published - 2020
Externally published	Yes
Event	23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020 - Lima, Peru Duration: 4 Oct 2020 → 8 Oct 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12263 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020
Country/Territory	Peru
City	Lima
Period	4/10/20 → 8/10/20

Bibliographical note

Publisher Copyright:
© 2020, Springer Nature Switzerland AG.

Keywords

Colonoscopy
Domain transformation
Generative adversarial network
Optical flow

ASJC Scopus subject areas

Theoretical Computer Science
General Computer Science

Access to Document

10.1007/978-3-030-59716-0_70

Cite this

Xu, J., Anwar, S., Barnes, N., Grimpen, F., Salvado, O., Anderson, S., & Armin, M. A. (2020). OfGAN: Realistic Rendition of Synthetic Colonoscopy Videos. In A. L. Martel, P. Abolmaesumi, D. Stoyanov, D. Mateus, M. A. Zuluaga, S. K. Zhou, D. Racoceanu, & L. Joskowicz (Eds.), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings (pp. 732-741). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12263 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-59716-0_70

Xu, Jiabo ; Anwar, Saeed ; Barnes, Nick et al. / OfGAN : Realistic Rendition of Synthetic Colonoscopy Videos. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings. editor / Anne L. Martel ; Purang Abolmaesumi ; Danail Stoyanov ; Diana Mateus ; Maria A. Zuluaga ; S. Kevin Zhou ; Daniel Racoceanu ; Leo Joskowicz. Springer Science and Business Media Deutschland GmbH, 2020. pp. 732-741 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{3f124f51aff74146b9e872f51a4c34eb,

title = "OfGAN: Realistic Rendition of Synthetic Colonoscopy Videos",

abstract = "Data-driven methods usually require a large amount of labelled data for training and generalization, especially in medical imaging. Targeting the colonoscopy field, we develop the Optical Flow Generative Adversarial Network (OfGAN) to transform simulated colonoscopy videos into realistic ones while preserving annotation. The advantages of our method are three-fold: the transformed videos are visually much more realistic; the annotation, such as optical flow of the source video is preserved in the transformed video, and it is robust to noise. The model uses a cycle-consistent structure and optical flow for both spatial and temporal consistency via adversarial training. We demonstrate that the performance of our OfGAN overwhelms the baseline method in relative tasks through both qualitative and quantitative evaluation.",

keywords = "Colonoscopy, Domain transformation, Generative adversarial network, Optical flow",

author = "Jiabo Xu and Saeed Anwar and Nick Barnes and Florian Grimpen and Olivier Salvado and Stuart Anderson and Armin, {Mohammad Ali}",

note = "Publisher Copyright: {\textcopyright} 2020, Springer Nature Switzerland AG.; 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020 ; Conference date: 04-10-2020 Through 08-10-2020",

year = "2020",

doi = "10.1007/978-3-030-59716-0_70",

language = "English",

isbn = "9783030597153",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Science and Business Media Deutschland GmbH",

pages = "732--741",

editor = "Martel, {Anne L.} and Purang Abolmaesumi and Danail Stoyanov and Diana Mateus and Zuluaga, {Maria A.} and Zhou, {S. Kevin} and Daniel Racoceanu and Leo Joskowicz",

booktitle = "Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings",

address = "Germany",

}

Xu, J, Anwar, S, Barnes, N, Grimpen, F, Salvado, O, Anderson, S & Armin, MA 2020, OfGAN: Realistic Rendition of Synthetic Colonoscopy Videos. in AL Martel, P Abolmaesumi, D Stoyanov, D Mateus, MA Zuluaga, SK Zhou, D Racoceanu & L Joskowicz (eds), Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12263 LNCS, Springer Science and Business Media Deutschland GmbH, pp. 732-741, 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020, Lima, Peru, 4/10/20. https://doi.org/10.1007/978-3-030-59716-0_70

OfGAN: Realistic Rendition of Synthetic Colonoscopy Videos. / Xu, Jiabo; Anwar, Saeed; Barnes, Nick et al.
Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings. ed. / Anne L. Martel; Purang Abolmaesumi; Danail Stoyanov; Diana Mateus; Maria A. Zuluaga; S. Kevin Zhou; Daniel Racoceanu; Leo Joskowicz. Springer Science and Business Media Deutschland GmbH, 2020. p. 732-741 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12263 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - OfGAN

T2 - 23rd International Conference on Medical Image Computing and Computer-Assisted Intervention, MICCAI 2020

AU - Xu, Jiabo

AU - Anwar, Saeed

AU - Barnes, Nick

AU - Grimpen, Florian

AU - Salvado, Olivier

AU - Anderson, Stuart

AU - Armin, Mohammad Ali

PY - 2020

Y1 - 2020

N2 - Data-driven methods usually require a large amount of labelled data for training and generalization, especially in medical imaging. Targeting the colonoscopy field, we develop the Optical Flow Generative Adversarial Network (OfGAN) to transform simulated colonoscopy videos into realistic ones while preserving annotation. The advantages of our method are three-fold: the transformed videos are visually much more realistic; the annotation, such as optical flow of the source video is preserved in the transformed video, and it is robust to noise. The model uses a cycle-consistent structure and optical flow for both spatial and temporal consistency via adversarial training. We demonstrate that the performance of our OfGAN overwhelms the baseline method in relative tasks through both qualitative and quantitative evaluation.

AB - Data-driven methods usually require a large amount of labelled data for training and generalization, especially in medical imaging. Targeting the colonoscopy field, we develop the Optical Flow Generative Adversarial Network (OfGAN) to transform simulated colonoscopy videos into realistic ones while preserving annotation. The advantages of our method are three-fold: the transformed videos are visually much more realistic; the annotation, such as optical flow of the source video is preserved in the transformed video, and it is robust to noise. The model uses a cycle-consistent structure and optical flow for both spatial and temporal consistency via adversarial training. We demonstrate that the performance of our OfGAN overwhelms the baseline method in relative tasks through both qualitative and quantitative evaluation.

KW - Colonoscopy

KW - Domain transformation

KW - Generative adversarial network

KW - Optical flow

UR - http://www.scopus.com/inward/record.url?scp=85092711022&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-59716-0_70

DO - 10.1007/978-3-030-59716-0_70

M3 - Conference contribution

AN - SCOPUS:85092711022

SN - 9783030597153

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 732

EP - 741

BT - Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings

A2 - Martel, Anne L.

A2 - Abolmaesumi, Purang

A2 - Stoyanov, Danail

A2 - Mateus, Diana

A2 - Zuluaga, Maria A.

A2 - Zhou, S. Kevin

A2 - Racoceanu, Daniel

A2 - Joskowicz, Leo

PB - Springer Science and Business Media Deutschland GmbH

Y2 - 4 October 2020 through 8 October 2020

ER -

Xu J, Anwar S, Barnes N, Grimpen F, Salvado O, Anderson S et al. OfGAN: Realistic Rendition of Synthetic Colonoscopy Videos. In Martel AL, Abolmaesumi P, Stoyanov D, Mateus D, Zuluaga MA, Zhou SK, Racoceanu D, Joskowicz L, editors, Medical Image Computing and Computer Assisted Intervention – MICCAI 2020 - 23rd International Conference, Proceedings. Springer Science and Business Media Deutschland GmbH. 2020. p. 732-741. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-59716-0_70

OfGAN: Realistic Rendition of Synthetic Colonoscopy Videos

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Fingerprint

Cite this