Publications
Variational Approximations for Robust Bayesian Inference via Rho-Posteriors
Preprint
The \(\rho\)-estimator is a robust alternative to MLE defined via a coordinate-wise contrast function. For a function \(\psi\) and observations \(X_i\), the contrast between \(\theta\) and \(\theta'\) is \[\ell_\psi(x_i;\theta,\theta') := \psi\!\left(\frac{p^i_{\theta'}(x_i)}{p^i_{\theta}(x_i)}\right),\] with empirical average \(\hat{R}_\psi(\theta,\theta') := \frac{1}{n}\sum_{i=1}^n \ell_\psi(X_i;\theta,\theta')\). The \(\rho\)-estimator \(\hat{\theta}_{n,\psi}\) minimizes the supremum contrast \(\hat{R}^*_\psi(\theta) := \sup_{\theta'\in\Theta}\hat{R}_\psi(\theta,\theta')\) over \(\Theta\). This paper introduces this framework and derives finite-sample PAC-Bayesian oracle inequalities for the resulting estimator, bridging robust estimation with Bayesian computation through tractable variational approximations that inherit the robustness properties of the exact \(\rho\)-posterior.
On importance sampling and independent Metropolis–Hastings with an unbounded weight function
Major revision at The Annals of Statistics
We study importance sampling (IS) and the particle independent Metropolis–Hastings (PIMH) algorithm when the weight function is unbounded but has finite moments of order \(p\). For PIMH with \(N\) particles, we establish the convergence rate \[\left|\bar{q}P^t - \bar{\pi}\right|_{\mathrm{TV}} \leq \frac{C}{\sqrt{N}}\,(1+t)^{p-1}.\] For the single-chain IMH, we prove that the common random numbers (CRN) coupling is maximal, yielding the exact identity \[\left|P^t(x,\cdot) - P^t(y,\cdot)\right|_{\mathrm{TV}} = P_{x,y}(\tau > t).\] This allows a formal comparison of the finite-time biases of IS and IMH, showing IMH to have strictly smaller bias.
Convergence of Statistical Estimators via Mutual Information Bounds
Submitted and under review for Journal of Machine Learning Research
We introduce a unified mutual information bound for general statistical models, bridging PAC-Bayesian theory, Bayesian nonparametrics, and classical estimation. The bound yields sharper contraction rates for fractional posteriors and applies to a wide family of estimators including variational inference and MLE. The central inequality is \[\mathbb{E}_{\theta\sim\hat{\rho}}\!\left[D_\alpha(P_\theta\|P_{\theta_0})\right] - \frac{\alpha}{n(1-\alpha)}\,\mathbb{E}_{\theta\sim\hat{\rho}}\!\left[r_n(\theta,\theta_0)\right] \leq \frac{I(\theta,S)}{n(1-\alpha)},\] where \(D_\alpha\) is the Rényi divergence of order \(\alpha\), \(r_n\) is the log-likelihood ratio evaluated on the sample \(S\), and \(I(\theta,S)\) is the mutual information between the estimator and the data.