Efficient Inference For Hybrid Bayesian Networks

Abstract

Uncertainty is everywhere in real life so we have to use stochastic model for most real-world problems. In general, both the systems mechanism and the observable measurements involve random noise. Therefore, probability theory and statistical estimation play important roles in decision making. First of all, we need a good knowledge representation to integrate information under uncertainty; then we need to conduct efficient reasoning about the state of the world given noisy observations. Bayesian networks (BNs) provide a compact, efficient and easy-to-interpret way to model the joint probability distribution of random variables over a problem domain. A Bayesian network encodes dependency relationship between random variables into a graphical probabilistic model. The structural properties and expressive power of Bayesian network make it an excellent knowledge base for effective probabilistic inference. Over the past several decades, a number of exact and approximate inference algorithms have been proposed and applied for inference in different types of Bayesian networks. However, in general, BN probabilistic inference is NP-hard. In particular, probabilistic reasoning for BNs with nonlinear non-Gaussian hybrid model is known to be one of the most difficult problems. First, no exact method is possible to compute the posterior distributions in such case. Second, relatively little research has been done for general hybrid models. Unfortunately, most real-world problems are naturally modeled with both categorical variables and continuous variables with typically nonlinear relationship. This dissertation focuses on the hybrid Bayesian networks containing both discrete and continuous random variables. The hybrid model may involve nonlinear functions in conditional probability distributions and the distributions could be arbitrary. I first give a thorough introduction to Bayesian networks and review of the state-of-the-art inference algorithms in the literature. Then a suite of efficient algorithms is proposed to compute the posterior distributions of hidden variables for arbitrary continuous and hybrid Bayesian networks. Moreover, in order to evaluate the performance of the algorithms with hybrid Bayesian networks, I present an approximate analytical method to estimate the performance bound. This method can help the decision maker to understand the prediction performance of a BN model without extensive simulation. It can also help the modeler to build and validate a model effectively. Solid theoretical derivations and promising numerical experimental results show that the research in this dissertation is fundamentally sound and can be applied in various decision support systems.