Resource allocation problems arise in application domains such as logistics, networking, manufacturing, and (CI)-I-4 systems. The discrete nature of resources to be allocated makes such problems combinatorially complex. In addition, uncertainties in the times when resources are requested and relinquished introduce additional complexities often necessitating the use of simulation for modeling and analysis purposes. In this paper, we present two approaches for solving such problems, the first based on ordinal optimization and the second on the idea of replacing the original discrete allocation problem by a "surrogate model" involving a continuous allocation problem. The latter is simpler to solve through gradient-based techniques and can be shown to recover the solutions of the original problem. Concurrent simulation is used to estimate the gradients required in this approach, leading to extremely fast solutions for many problems in practice.