Cross-modality person re-identification between infrared (IR) and visible (VIS) domains is a challenging problem, which aims to identify persons in different spectrums, variety of camera specs, and broad illumination conditions. This paper proposes distance based training on an one-stream convolutional neural network architecture, in which network weights are shared between IR and VIS domains to learn discriminative features for person re-identification. The distance based score layer enables to train the network using distance metrics instead of the fully connected layer. Different distance metrics can be used for training and ranking stages. The proposed structure enables to extract discriminative features in the cross-modality data without using dedicated structures for each domain. Experimental results on a cross-modality person re-identification dataset indicate that the proposed approach outperforms the stateof-the-art methods.