Background Early neurological deterioration is encountered in up to a third of patients with isolated pontine infarcts. A limited number of clinical and imaging features have been suggested as predictors of neurological progression in this setting. In this study, we assessed whether quantitative apparent diffusion coefficient (ADC) measurements within the ischemic pontine region could be used as a radiomic feature to forecast clinical deterioration. Methods We calculated the mean ADC value of ischemic voxels within the ischemic region and normalized them to the contralateral non-ischemic tissue (relative ADC, rADC) in patients with isolated pontine infarcts. This imaging signature was then compared among patients with neurological progression (n = 21) and a propensity matched cohort of non-progressors (n = 42), together with other clinical and imaging features in bivariate and multivariate statistical models. Results The rADC(mean) was significantly lower among patients with progression (p = 0.008). Female gender and extension of the ischemic lesion to the ventral pontine surface were other features significantly associated with progression. The association between rADC(mean) and progression persisted in multivariate models with an odds ratio of 13.7 (95% CI 2.6-72.8; p = 0.002) for progression among patients with rADC(mean) <= 0.67 in their ischemic tissue. The probability for worsening was 80% among patients who had an ischemic lesion extending to the ventral pontine surface with a mean rADC <= 0.67. Conclusion The mean rADC value within the ischemic lesion is closely related with early neurological deterioration in patients with isolated pontine infarcts.