End-to-End Learning of Semantic Grid Estimation Deep Neural Network with Occupancy Grids

Creative Commons License

ERKENT Ö., Wolf C., Laugier C.

UNMANNED SYSTEMS, vol.7, no.3, pp.171-181, 2019 (ESCI) identifier identifier

  • Publication Type: Article / Article
  • Volume: 7 Issue: 3
  • Publication Date: 2019
  • Doi Number: 10.1142/s2301385019410036
  • Journal Name: UNMANNED SYSTEMS
  • Journal Indexes: Emerging Sources Citation Index (ESCI), Scopus
  • Page Numbers: pp.171-181
  • Keywords: Semantic segmentation, occupancy grids, autonomous vehicles, perception
  • Hacettepe University Affiliated: No


We propose semantic grid, a spatial 2D map of the environment around an autonomous vehicle consisting of cells which represent the semantic information of the corresponding region such as car, road, vegetation, bikes, etc. It consists of an integration of an occupancy grid, which computes the grid states with a Bayesian filter approach, and semantic segmentation information from monocular RGB images, which is obtained with a deep neural network. The network fuses the information and can be trained in an end-to-end manner. The output of the neural network is refined with a conditional random field. The proposed method is tested in various datasets (KITTI dataset, Inria-Chroma dataset and SYNTHIA) and different deep neural network architectures are compared.