Abstract

Variational autoencoders (VAEs) are powerful deep generative models widely used to represent high-dimensional complex data through a low-dimensional latent space learned in an unsupervised manner. In the original VAE model, the input data vectors are processed independently. Recently, a series of papers have presented different extensions of the VAE to process sequential data, which model not only the latent space but also the temporal dependencies within a sequence of data vectors and corresponding latent vectors, relying on recurrent neural networks or state-space models. In this monograph, we perform a literature review of these models. We introduce and discuss a general class of models, called dynamical variational autoencoders (DVAEs), which encompasses a large subset of these temporal VAE extensions. Then, we present in detail seven recently proposed DVAE models, with an aim to homogenize the notations and presentation lines, as well as to relate these models with existing classical temporal models. We have reimplemented those seven DVAE models and present the results of an experimental benchmark conducted on the speech analysis-resynthesis task (the PyTorch code is made publicly available). The monograph concludes with a discussion on important issues concerning the DVAE class of models and future research guidelines.

Dynamical Variational Autoencoders: A Comprehensive Review

Variational autoencoders (VAEs) are powerful deep generative models widely used to represent high-dimensional complex data through a low-dimensional latent space learned in an unsupervised manner. In this monograph the authors introduce and discuss a general class of models, called dynamical variational autoencoders (DVAEs), which extend VAEs to model temporal vector sequences. In doing so the authors provide:

• a formal definition of the general class of DVAEs

• a detailed and complete technical description of seven DVAE models

• a rapid overview of other DVAE models presented in the recent literature

• discussion of the recent developments in DVAEs in relation to the history and technical background of the classical models DVAEs are built on

• a quantitative benchmark of the selected DVAE models

• a discussion to put the DVAE class of models into perspective

This monograph is a comprehensive review of the current state-of-the-art in DVAEs. It gives the reader an accessible summary of the technical aspects of the different DVAE models, their connections with classical models, their cross-connections, and their unification in the DVAE class in a concise, easy-to-read book.

The authors have put considerable effort into unifying the terminology and notation used across the various models which all students, researchers and practitioners working in machine learning will find an invaluable resource.