One of the most important policies adopted in inventory control is the replenishment cycle policy. Such a policy provides an effective means of damping planning instability and coping with demand uncertainty. In this paper we develop a constraint programming approach able to compute optimal replenishment cycle policy parameters under non-stationary stochastic demand, ordering, holding and shortage costs. We show how in our model it is possible to exploit the convexity of the cost-function during the search to dynamically compute bounds and perform cost-based filtering. Our computational experience show the effectiveness of our approach. Furthermore, we use the optimal solutions to analyze the quality of the solutions provided by an existing approximate mixed integer programming approach that exploits a piecewise linear approximation for the cost function.