In the areas of statistics and econometrics the analysis of binary and polychotomous response data is widely used. In classical statistics, the maximum likelihood method is used to model this data and inferences about the model are based on the associated asymptotic theory. However, the inferences based on the classical approach are not accurate if the sample size is small. J.H. Albert and S. Chib (Bayesian Analysis of Binary and Polychotomous Response Data, J. American Statistical Association 422, 669-679, 1993) proposed a Bayesian method to model categorical response data. In this method Gibbs sampling and the data augmentation algorithm are used together to model the data. In this article, Albert and Chib's approach is used to estimate the parameters in the logit and probit models. Furthermore, the maximum likelihood and ordinary least-squares methods axe discussed briefly, and a simple example is presented to compare these three methods.