Robust Inference for High-Dimensional Linear Models via Residual Randomization

Jul 1, 2021·
Y. Samuel Wang
Y. Samuel Wang
Si Kai Lee
Si Kai Lee
Panos Toulis
Panos Toulis
Mladen Kolar
Mladen Kolar
· 0 min read
PDF
Abstract
We propose a residual randomization procedure designed for robust inference using Lasso estimates in the high-dimensional setting. Compared to earlier work that focuses on sub-Gaussian errors, the proposed procedure is designed to work robustly in settings that also include heavy-tailed covariates and errors. Moreover, our procedure can be valid under clustered errors, which is important in practice, but has been largely overlooked by earlier work. Through extensive simulations, we illustrate our method’s wider range of applicability as suggested by theory. In particular, we show that our method outperforms state-of-art methods in challenging, yet more realistic, settings where the distribution of covariates is heavy-tailed or the sample size is small, while it remains competitive in standard, ‘well behaved’ settings previously studied in the literature.
Type
Publication
Proceedings of the 38th International Conference on Machine Learning (ICML) 2021
publication
Y. Samuel Wang
Authors
Post-doc (2018-2021)

Sam Wang is currently an assistant professor at Cornell University. He was a post-doc at the University of Chicago Booth School of Business advised by Mladen Kolar. His theoretical and methodological research interests are focused on causal discovery, graphical models, and high-dimensional statistical methods; his applied research interests lie in the social sciences. He completed his PhD in 2018 at the University of Washington advised by Mathias Drton and received his undergraduate degree from Rice University. Prior to embarking on his PhD studies, he worked in management consulting.

Personal Website

Si Kai Lee
Authors
RP (2019-2021)
Sky is a PhD student at Yale.
Panos Toulis
Authors

Panagiotis (Panos) Toulis studies causal inference in complex settings (e.g., networks) using methods of structured inference, such as permutation tests. He is also interested in the interface of statistics and optimization, particularly in inference problems on large data sets through stochastic gradient descent.

Personal website

Mladen Kolar
Authors
Professor of Data Sciences and Operations
Mladen Kolar is a Professor of Data Sciences and Operations at the University of Southern California Marshall School of Business and a Visiting Professor of Statistics and Data Science at Mohamed bin Zayed University of Artificial Intelligence. Before joining USC, he was on the faculty of the University of Chicago Booth School of Business. His research is focused on high-dimensional statistical methods, graphical models, varying-coefficient models and data mining, driven by the need to uncover interesting and scientifically meaningful structures from observational data. He is a Fellow of the Institute of Mathematical Statistics.