Tinysign: sign language recognition in low resolution settings


Huseyinoglu A., Bilge F. A., Bilge Y. C., İKİZLER CİNBİŞ N.

SIGNAL IMAGE AND VIDEO PROCESSING, sa.10, ss.6881-6890, 2024 (SCI-Expanded) identifier

  • Yayın Türü: Makale / Tam Makale
  • Basım Tarihi: 2024
  • Doi Numarası: 10.1007/s11760-024-03358-z
  • Dergi Adı: SIGNAL IMAGE AND VIDEO PROCESSING
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Compendex, INSPEC, zbMATH
  • Sayfa Sayıları: ss.6881-6890
  • Hacettepe Üniversitesi Adresli: Evet

Özet

Existing sign language recognition (SLR) methods mostly rely on high-quality videos that include clear hand and body movements. However, these approaches often fall short in addressing the challenges of real-world sign language communication scenarios. In this work, we tackle the problem of SLR in low resolution settings. To this end, we propose a novel approach by effectively encoding the spatial and temporal features. Our approach includes a sign-specific super-resolution module that improves discriminability between classes and a sign classifier that learns high-level spatio-temporal features. In order to evaluate the effectiveness of our approach, we introduce the first low-resolution sign language recognition benchmark and present the experimental results together with detailed analysis.