3D Skeletal Volume Templates for Deep Learning-Based Activity Recognition

Creative Commons License

KEÇELİ A. S., Kaya A., CAN A. B.

ELECTRONICS, cilt.11, sa.21, 2022 (SCI-Expanded) identifier identifier

  • Yayın Türü: Makale / Tam Makale
  • Cilt numarası: 11 Sayı: 21
  • Basım Tarihi: 2022
  • Doi Numarası: 10.3390/electronics11213567
  • Dergi Adı: ELECTRONICS
  • Derginin Tarandığı İndeksler: Science Citation Index Expanded (SCI-EXPANDED), Scopus, Aerospace Database, Communication Abstracts, INSPEC, Metadex, Directory of Open Access Journals, Civil Engineering Abstracts
  • Anahtar Kelimeler: 3D CNN, activity recognition, transfer learning, deep learning, classification, POSE
  • Hacettepe Üniversitesi Adresli: Evet

Özet

Due to advances in depth sensor technologies, the use of these sensors has positively impacted studies of human-computer interaction and activity recognition. This study proposes a novel 3D action template generated from depth sequence data and two methods to classify single-person activities using this 3D template. Initially, joint skeleton-based three-dimensional volumetric templates are constructed from depth information. In the first method, images are obtained from various view angles of these three-dimensional templates and used for deep feature extraction using a pre-trained convolutional neural network. In our experiments, a pre-trained AlexNet model trained with the ImageNet dataset is used as a feature extractor. Activities are classified by combining deep features and Histogram of Oriented Gradient (HOG) features. The second approach proposes a three-dimensional convolutional neural network that uses volumetric templates as input for activity classification. Proposed methods have been tested with two publicly available datasets. Experiments provided promising results compared with the other studies presented in the literature.