Skip to main navigation Skip to search Skip to main content

SegVLM: Deformable attentive visual enhancement for referring image segmentation using vision-language model

Research output: Contribution to journal › Article › peer-review

Abstract

Image segmentation is a fundamental task in computer vision that involves partitioning an image into semantically meaningful regions. Referring image segmentation extends this task by using natural language expressions to localize specific objects, requiring the effective integration of visual and linguistic information. In this work, we propose SegVLM, a vision-language model that incorporates architectural enhancements to improve both segmentation accuracy and cross-modal alignment. The model integrates Squeeze-and-Excitation (SE) module for dynamic feature recalibration, deformable convolutions for geometric adaptability, and residual connection for deep feature learning. We also introduce a novel Referring-Aware Fusion (RAF) loss, which balances region-level alignment, boundary precision, and class imbalance. Extensive experiments and ablation studies demonstrate that each component contributes to consistent performance improvements. SegVLM further exhibits strong generalization across diverse datasets and a variety of referring expression scenarios.

Original languageEnglish
Article number109721
JournalNeural Networks
Volume206
DOIs
StatePublished - Feb 2027

Bibliographical note

Publisher Copyright:
© 2026 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.

Keywords

  • Artificial intelligence
  • Computer vision
  • Deep Learning
  • Image segmentation
  • Referring image segmentation
  • Vision-language models

ASJC Scopus subject areas

  • Cognitive Neuroscience
  • Artificial Intelligence

Fingerprint

Dive into the research topics of 'SegVLM: Deformable attentive visual enhancement for referring image segmentation using vision-language model'. Together they form a unique fingerprint.

Cite this