Edit Page

Yuyang Shi

• Personal Webpage
• GitHub
• Twitter
• Google Scholar

Statistical Machine Learning, Deep Learning, Generative Models

Hi! I’m Yuyang, a 2nd year PhD student jointly supervised by Arnaud Doucet, Tom Rainforth and Yee Whye Teh.

I am interested in probabilistic and statistical methodologies in machine learning. My current research interests include probabilistic generative models, variational inference, and Monte Carlo methods.

Before starting my PhD, I completed a Master of Mathematics at University of Oxford.

Publications

2022

• Y. Shi , V. De Bortoli , G. Deligiannidis , A. Doucet , Conditional Simulation Using Diffusion Schr\backslash" odinger Bridges, arXiv preprint arXiv:2202.13460, 2022.
  

  @article{shi2022conditional,
    title = {Conditional Simulation Using Diffusion Schr$\backslash$" odinger Bridges},
    author = {Shi, Yuyang and De Bortoli, Valentin and Deligiannidis, George and Doucet, Arnaud},
    journal = {arXiv preprint arXiv:2202.13460},
    year = {2022}
  }
  

2021

• A. Campbell , Y. Shi , T. Rainforth , A. Doucet , Online Variational Filtering and Parameter Learning, in Advances in Neural Information Processing Systems, 2021.
  
  We present a variational method for online state estimation and parameter learning in state-space models (SSMs), a ubiquitous class of latent variable models for sequential data. As per standard batch variational techniques, we use stochastic gradients to simultaneously optimize a lower bound on the log evidence with respect to both model parameters and a variational approximation of the states’ posterior distribution. However, unlike existing approaches, our method is able to operate in an entirely online manner, such that historic observations do not require revisitation after being incorporated and the cost of updates at each time step remains constant, despite the growing dimensionality of the joint posterior distribution of the states. This is achieved by utilizing backward decompositions of this joint posterior distribution and of its variational approximation, combined with Bellman-type recursions for the evidence lower bound and its gradients. We demonstrate the performance of this methodology across several examples, including high-dimensional SSMs and sequential Variational Auto-Encoders.

  @inproceedings{campbell2021online,
    title = {Online Variational Filtering and Parameter Learning},
    author = {Campbell, Andrew and Shi, Yuyang and Rainforth, Tom and Doucet, Arnaud},
    booktitle = {Advances in Neural Information Processing Systems},
    year = {2021}
  }
  

• Y. Shi , R. Cornish , On Multilevel Monte Carlo Unbiased Gradient Estimation for Deep Latent Variable Models, in Proceedings of The 24th International Conference on Artificial Intelligence and Statistics, 2021.
  
  Standard variational schemes for training deep latent variable models rely on biased gradient estimates of the target objective. Techniques based on the Evidence Lower Bound (ELBO), and tighter variants obtained via importance sampling, produce biased gradient estimates of the true log-likelihood. The family of Reweighted Wake-Sleep (RWS) methods further relies on a biased estimator of the inference objective, which biases training of the encoder also. In this work, we show how Multilevel Monte Carlo (MLMC) can provide a natural framework for debiasing these methods with two different estimators. We prove rigorously that this approach yields unbiased gradient estimators with finite variance under reasonable conditions. Furthermore, we investigate methods that can reduce variance and ensure finite variance in practice. Finally, we show empirically that the proposed unbiased estimators outperform IWAE and other debiasing method on a variety of applications at the same expected cost.

  @inproceedings{shi2021multilevel,
    title = {On Multilevel Monte Carlo Unbiased Gradient Estimation for Deep Latent Variable Models},
    author = {Shi, Yuyang and Cornish, Rob},
    booktitle = {Proceedings of The 24th International Conference on Artificial Intelligence and Statistics},
    year = {2021}
  }