Although action recognition is a widely studied field in computer vision, the recognitions of aggressive activities and crowd violence actions are comparatively less studied. Nowadays, so many surveillance cameras have been installed in the streets and there is a demand for intelligent crowd activity detection systems. A method for violence detection in videos is proposed. The primary contribution is a novel transfer learning-based violence detector that gives promising results compared with the existing detectors. First, the optical flows of the input videos are computed via Lucas-Kanade method. Then, several 2D templates are constructed with overlapping optical flow magnitudes and orientations. These templates are supplied to a pre-trained convolutional neural network as input and deep features of different layers are extracted. Cubic kernel support vector machine and subspace k-nearest neighbours classifiers are trained for prediction and the proposed method is tested with three different datasets that commonly used in violence detection studies.