Publications

Attention with Markov: A Curious Case of Single-layer Transformers

Ashok Vardhan Makkuva*, Marco Bondaschi*, Adway Girish, Alliot Nagle, Martin Jaggi, Hyeji Kim, Michael Gastpar

[paper] [code]

Fundamental Limits of Prompt Compression: A Rate-Distortion Framework for Black-Box Language Models

Alliot Nagle*, Adway Girish*, Marco Bondaschi, Michael Gastpar, Ashok Vardhan Makkuva, Hyeji Kim

[paper] [code]

Local to Global: Learning Dynamics and Effect of Initialization for Transformers

Ashok Vardhan Makkuva*, Marco Bondaschi*, Chanakya Ekbote, Adway Girish, Alliot Nagle, Hyeji Kim, Michael Gastpar

[paper] [code]

Neural Distributed Source Coding

Jay Whang*, Alliot Nagle*, Anish Acharya, Hyeji Kim, Alexandros Dimakis

[paper] [code]

Improved high-dimensional multivariate autoregressive model estimation of human electrophysiological data using fMRI priors

Alliot Nagle, Josh P. Gerrelts, Bryan M. Krause, Aaron D. Boes, Joel E. Bruss, Kirill V. Nourski, Matthew I. Banks, Barry Van Veen

[paper]

Rare Gems: Finding Lottery Tickets at Initialization

Kartik Sreenivasan, Jy-yong Sohn, Liu Yang, Matthew Grinde, Alliot Nagle, Hongyi Wang, Eric Xing, Kangwook Lee, Dimitris Papailiopoulos

[paper] [code]

Optimal Lottery Tickets via SubsetSum: Logarithmic Over-Parameterization is Sufficient

Ankit Pensia, Shashank Rajput, Alliot Nagle, Harit Vishwakarma, Dimitris Papailiopoulos

[paper] [code]