Learning and inferencing state-space models through GRU cells and Bayesian principles

Abstract

State-space models (SSMs) perform predictions by learning the underlying dynamics of observed sequence. We start with a throughout literature review on Gaussian Process (GP) models and time series models based on GPs. Then, we elaborate more on the Gaussian Process State-Space Model (GP-SSM): a Bayesian nonparametric generalisation of discrete time nonlinear state-space models. We provide a formulation of the GP-SSM that offers different insight into its properties. Then, we propose a new SSM approach in both high and low dimensional observation space, which utilizes Bayesian filtering-smoothing to model systems dynamics more accurately than RNN-based SSMs and can be learned in an end-to-end manner. The designed architecture, which we call the Gated Inference Network (GIN), is able to integrate the uncertainty estimates and learn the complicated dynamics of the system that enables us to perform estimation and imputation tasks in both data presence and absence. The proposed model uses the GRU cells into its structure to complete the data flow, while avoids expensive computations and potentially unstable matrix inversions. The GIN is able to deal with any time-series data and gives us a strong robustness to handle the observational noise. Finally, in the numerical experiments, we show that the GIN reduces the uncertainty of estimates and outperforms its counterparts , LSTMs, GRUs and variational approaches. Several SOTA approaches are taken into account for the sake of comparison in order to show the out-performance of the proposed algorithm.