Institute of Information Theory and Automation

Publication details

Approximation of Unknown Multivariate Probability Distributions by Using Mixtures of Product Components: A Tutorial

Journal Article

Grim Jiří

serial: International Journal of Pattern Recognition and Artificial Intelligence vol.31,

project(s): GA17-18407S, GA ČR

keywords: multivariate statistics, product mixtures, naive Bayes models, EM algorithm, pattern recognition, neural networks, expert systems, image analysis

preview: Download

abstract (eng):

In literature the references to EM estimation of product mixtures are not very frequent. The simplifying assumption of product components, e.g. diagonal covariance matrices in case of Gaussian mixtures, is usually considered only as a compromise because of some computational constraints or limited data set. We have found that the product mixtures are rarely used intentionally as a preferable approximating tool. Probably, most practitioners do not "trust'' the product components because of their formal similarity to "naive Bayes models''. Another reason could be an unrecognized numerical instability of EM algorithm in multidimensional spaces. In this paper we recall that the product mixture model does not imply the assumption of independence of variables. It is even not restrictive if the number of components is large enough. In addition, the product components increase numerical stability of the standard EM algorithm, simplify the EM iterations and have some other important advantages. We discuss and explain the implementation details of EM algorithm and summarize our experience in estimating product mixtures. Finally we illustrate the wide applicability of product mixtures in pattern recognition and in other fields.


Responsible for information: admin
Last modification: 21.12.2012
Institute of Information Theory and Automation