Bayesian nonparametric inference

Bayesian nonparametric methods provide elegant solution to the problem of model selection and controlling model complexity by defining distributions on model structure which can adapt its complexity to data. Furthermore, it may refine and even complicate its structure as new data is being observed.

For many nonparametric models such as widely used Dirichlet process or Indian Buffet Process, it is common to assume exchangeability of data. While this assumption often holds, in many settings when data has temporal, spatial or any other internal dependencies, a proper non-exchangeable prior which takes such information into account could model data more adequately and thus fit it more accurately.

A number of such distributions were developed to date including recently proposed distance-dependent Chinese Restaurant Process (ddCRP) which generalizes widely adopted Chinese Restraunt Process (CRP) to the case when data points have some interdependencies which are represented in the form of prior “distances” such that points with small distances are more likely to share partition a priori.

For the important sequential case of ddCRP (seqddCRP) which is natural for modeling temporal and natural text data we could obtain interesting connections with Laplacian of the random graph modeled by the process. This allowed to derive the mean-field variational inference algorithm for seqddCRP mixture model providing deterministic alternative to MCMC methods.

Publication: S. Bartunov, D. Vetrov. Variational Inference for Sequential Distance Dependent Chinese Restaurant Process. Proceedings of the 31st International Conference on Machine Learning, Beijing, China, 2014. JMLR: W&CP volume 32. pdf