SMART: A Spectral Transfer Approach to Multi-Task Learning
Privacy from Symmetry: Orthogonally Equivariant Transformers for LLM Inference
alexander-yukhimchuk
Muon with Nesterov Momentum: Heavy-Tailed Noise and (Randomized) Inexact Polar Decomposition
Joint-Sparse Transfer Learning for High-Dimensional Multi-Output Regression
Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters
alexander-yukhimchuk
Trust-Region Sequential Quadratic Programming for Stochastic Optimization with Random Models
In this work, we consider solving optimization problems with a stochastic objective and deterministic equality constraints. We propose a Trust-Region Sequential Quadratic …
Personalized Binomial DAGs Learning with Network Structured Covariates
High-Dimensional Markov-switching Ordinary Differential Processes
katherine-tsai
Communication-Efficient Adaptive Batch Size Strategies for Distributed Local Gradient Methods
Modern deep neural networks often require distributed training with many workers due to their large size. As worker numbers increase, communication overheads become the main …
Personalized Federated Learning with Multiple Known Clusters
We consider the problem of personalized federated learning when there are known cluster structures within users. An intuitive approach would be to regularize the parameters so that …
