There is a substantial increase in the utilization of web-based assessment procedures to assist teaching and learning processes in higher education institutions. Despite their benefits, institutions use them to a limited extent due to a number of factors influencing both instructors' and students' behaviors. This study examines students' acceptance of a web-based summative listening test administered in the 2014-2015 academic year within an 'English as a Foreign Language' course at a two-year post-secondary military school in Turkey. The participants consisted of 602 military students. As a model, Computer Based Assessment Acceptance Model based on Technology Acceptance Model was adopted in order to analyze the participants' perceptions on a web-based summative listening comprehension test. The data were collected via an online questionnaire. A structural equation modeling analysis was utilized to analyze the relationships among factors. The general results showed that perceived ease of use and perceived playfulness had a direct influence on the participants' behavioral intentions to use web-based tests. In other words, web-based tests are expected to be used if they are easy to use and playful enough. (C) 2016 Elsevier Ltd. All rights reserved.