Abstract

A bivariate Markov process comprises a pair of random processes which are jointly Markov. One of the two processes in that pair is observable while the other plays the role of an underlying process. We are interested in three classes of bivariate Markov processes. In the first and major class of interest, the underlying and observable processes are continuous-time with finite alphabet; in the second class, they are discrete-time with finite alphabet; and in the third class, the underlying process is continuous-time with uncountably infinite alphabet, and the observable process is continuous-time with countably or uncountably infinite alphabet. We refer to processes in the first two classes as bivariate Markov chains. Important examples of continuoustime bivariate Markov chains include the Markov modulated Poisson process, and the batch Markovian arrival process. A hidden Markov model with finite alphabet is an example of a discrete-time bivariate Markov chain. In the third class we have diffusion processes observed in Brownian motion, and diffusion processes modulating the rate of a Poisson process. Bivariate Markov processes play central roles in the theory and applications of estimation, control, queuing, biomedical engineering, and reliability. We review properties of bivariate Markov processes, recursive estimation of their statistics, and recursive and iterative parameter estimation.

Bivariate Markov Processes and Their Estimation

Bivariate Markov processes play a central role in the theory and applications of estimation, control, queuing, biomedical engineering, and reliability. Bivariate Markov Processes and Their Estimation presents some of the fundamentals of the theory of bivariate Markov processes, and reviews the various parameters and signal estimation approaches that are associated with these Markov processes. It reviews both causal and non-causal estimation of some statistics of the bivariate Markov processes. In addition, it covers off-line as well as on-line recursive parameter estimation approaches.

Bivariate Markov Processes and Their Estimation is an ideal springboard for researchers and students who are interested in pursuing the study of this interesting family of processes. While proofs are generally omitted, an interested reader should be able to implement the estimation algorithms for bivariate Markov chains directly from the text. The material should be accessible to the signal processing community, although it requires some familiarity with Markov chains and the intricacies of the theory of hidden Markov models.