Aim of the study is to determine whether the auditory processing of temporal fine structure (TFS) is affected with normal aging, even in the presence of normal audiometric hearing and fine cognitive state; and, if it is, to see whether a comparable effect is also observed in the processing of a diotic change in sound envelope. The event-related potentials (ERPs) to binaural beats (BBs), which are the responses of the binaural mechanisms processing TFS of a sound, and the ERPs to diotic amplitude modulation (AM) stimuli, which are the responses of the monaural mechanisms processing the changes in its envelope, were recorded from thirteen young university students and ten senior but active university professors, all with normal hearing in low frequencies. To obtain directly the specific BB responses without confounding monaural frequency change-evoked responses, we used single-cycle BB stimuli with temporary sub-threshold frequency shifts. BBs of a 250-Hz tone and diotic AM of the same tone with similar perceptual salience were presented with 2-second stimulus onset asynchrony. The N1 components of the ERPs to both stimuli displayed notable age-dependent changes in their scalp topography and significant amplitude reduction and latency prolongation in the elderly. These amplitude and latency changes were at similar rates for the two stimulus types, implying that the auditory TFS and envelope processing mechanisms are proportionally affected by physiological aging. These results may serve as control data in future studies investigating the effect of aging-associated cognitive pathologies on auditory TFS processing.