RetailEye: Supervised Contrastive Learning with Compliance Matching for Retail Shelf Monitoring

By harnessing technological advancements in computer vision and artificial intelligence, retail entrepreneurs can not only meet their objectives but also position themselves for sustainable growth in a competitive marketplace. A critical area of focus is inventory management, particularly monitoring grocery products on shelves and identifying misplaced or out-of-stock items. However, automatically detecting and recognizing products in real-time retail environments presents significant challenges, including factors such as varied visual representations, unpredictable poses, partial or full occlusions, and variations of lighting reflections on glossy packaging, and a lack of unified resources. In this paper, we propose and evaluate a two-stage approach, termed RetailEye, which employs supervised contrastive learning with compliance matching and leverages the latest developments in deep learning. After evaluating different models for object detection and recognition, we designed our system based on YOLOv8s in the first stage and EfficientNetV2-S and ResNet18 in the second stage. The proposed model outperformed the one-stage approach with high detection and recognition accuracy. Additionally, we unveil a custom dataset specifically curated for this research, aimed at advancing the field of inventory management.