Top-down and bottom-up attentional multiple instance learning for still image action recognition

BAŞ Ç., İKİZLER CİNBİŞ N.

SIGNAL PROCESSING-IMAGE COMMUNICATION, cilt.104, 2022 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 104
  • Basım Tarihi: 2022
  • Doi Numarası: 10.1016/j.image.2022.116664
  • Dergi Adı: SIGNAL PROCESSING-IMAGE COMMUNICATION
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Academic Search Premier, Aerospace Database, Applied Science & Technology Source, Communication Abstracts, Computer & Applied Sciences, INSPEC, Metadex, Civil Engineering Abstracts
  • Anahtar Kelimeler: Still image action recognition, Multiple instance learning, Attention, RECOGNIZING ACTIONS, MODEL
  • Hacettepe Üniversitesi Adresli: Evet

Özet

We propose an end-to-end, top-down and bottom-up attentional deep multiple instance learning approach for still image action recognition. Our approach does not rely on neither any attribute nor object labels and/or explicitly trained object detectors. Instead, our approach simultaneously proposes image regions, chooses the action related regions with a top-down attention mechanism, weights each image region with a bottom-up attention layer to mask unrelated pixels to yield a fine-grained pixel-level action related map. Then, it embeds the selected regions into a single image-level feature descriptor and assigns an action label. We evaluate our approach on four benchmark still image action datasets: Stanford40 (Yao et al., 2011), Pascal VOC 2012 (Everingham et al., 2012) MPII (Andriluka et al., 2014) and VCOCO (Gupta and Malik, 2015). Our results demonstrate that, while increasing the overall recognition performance, our framework successfully selects action related image regions and creates pixel grade action masks.