Publications

You can also find my articles on my Google Scholar profile.

Algorithmic Perspectives on LLMs/Transformers

Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regression

Deqing Fu, Tian-Qi Chen, Robin Jia, Vatsal Sharan
NeurIPS 2024
Previously titled: Transformers Learn Higher-Order Optimization Methods for In-Context Learning: A Study with Linear Models
SoCalNLP Symposium 2023 Best Paper Award
Paper | Code

Pre-trained Large Language Models Use Fourier Features to Compute Addition

Tianyi Zhou, Deqing Fu, Vatsal Sharan, Robin Jia
NeurIPS 2024
Paper

FoNE: Precise Single-Token Number Embeddings via Fourier Features

Tianyi Zhou, Deqing Fu, Mahdi Soltanolkotabi, Robin Jia, Vatsal Sharan
arxiv 2025 Paper, Website

Transformers Learn Low Sensitivity Functions: Investigations and Implications

Bhavya Vasudeva^‡, Deqing Fu^‡, Tianyi Zhou, Elliot Kau, You-Qi Huang, Vatsal Sharan
ICLR 2025
Paper
^‡Equal Contribution.

DeLLMa: Decision Making Under Uncertainty with Large Language Models

Ollie Liu^‡, Deqing Fu^‡, Dani Yogatama, Willie Neiswanger
ICLR 2025 (Spotlight, Top 5.1%)
^‡Equal Contribution.
Paper | Code | Website

Synthetic Data and Multimodal Learning

VisualLens: Personalization through Visual History

Wang Bill Zhu, Deqing Fu, Kai Sun, Yi Lu, Zhaojiang Lin, Seungwhan Moon, Kanika Narang, Mustafa Canim, Yue Liu, Anuj Kumar, Xin Luna Dong
arXiv 2024
Paper

TLDR: Token-Level Detective Reward Model for Large Vision Language Models

Deqing Fu, Tong Xiao, Rui Wang, Wang Zhu, Pengchuan Zhang, Guan Pang, Robin Jia, Lawrence Chen
ICLR 2025
Paper

IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representations

Deqing Fu^‡, Ruohao Guo^‡, Ghazal Khalighinejad^‡, Ollie Liu^‡, Bhuwan Dhingra, Dani Yogatama, Robin Jia, Willie Neiswanger
COLM 2024
‡Equal Contribution.
Paper | Website

DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback

Jiao Sun^‡, Deqing Fu^‡, Yushi Hu^‡, Su Wang, Royi Rassin, Da-Cheng Juan, Dana Alon, Charles Herrmann, Sjoerd van Steenkiste, Ranjay Krishna, Cyrus Rashtchian
NAACL 2025
^‡Equal Contribution.
Paper

SCENE: Self-Labeled Counterfactuals for Extrapolating to Negative Examples

Deqing Fu, Ameya Godbole, Robin Jia
EMNLP 2023
Paper | Code