Neil Band

I'm a fourth-year PhD student in the Computer Science Department at Stanford University. My advisors are Tatsunori Hashimoto and Tengyu Ma. My work is supported by the NSF GRFP, Quad, and Mercor Fellowships.

I was previously a Masters by Research student in the Oxford Applied and Theoretical Machine Learning Group (OATML) at the University of Oxford, where I was advised by Yarin Gal and supported by a Rhodes Scholarship. Before that, I did my A.B. in Computer Science and Economics at Harvard College. I worked with Stratos Idreos in the Harvard Data Systems Lab on systems machine learning, and Manolis Kellis in the MIT Computational Biology Group on machine learning in single-cell transcriptomics.

Google Scholar  /  Twitter  /  Github

I'm interested in data efficiency---learning more from limited data---in settings such as pretraining and continual learning. I work on data efficiency methods using synthetic data, alternative architectures and objectives such as latent variable modeling and meta-learning, and reinforcement learning.

For a full list, please see my Google Scholar.

We optimize synthetic data for pretraining end-to-end against downstream task performance, by using metagradients to obtain fine-grained per-document rewards and running RL on those rewards.

We show that LMs can bootstrap their capabilities on task-agnostic data by inferring the latent thoughts that plausibly underlie pretraining documents.

A simple approach to learn knowledge from corpora as small as ~1M tokens, using graph-based synthetic data generation followed by continued pretraining.

An alignment objective to train language models that express calibrated verbal statements of confidence in their long-form natural language generations.

An easy-to-use, expert-guided, open-source suite of diabetic retinopathy detection benchmarking tasks for Bayesian deep learning.

A novel deep learning architecture that takes the entire dataset as input and learns to reason about relationships between datapoints using self-attention.

Based on Jon Barron's website (source code here).