Tianlong Chen

Incoming Assistant Professor
Computer Science, The University of North Carolina at Chapel Hill
Postdoctoral Researcher
CSAIL@MIT - BMI@Harvard - Broad Institute of MIT and Havard

tianlong@cs.unc.edu

Bio

Tianlong Chen received the Ph.D. degree in Electrical and Computer Engineering from University of Texas at Austin, TX, USA, in 2023. He will start as an Assistant Professor of Computer Science at The University of North Carolina at Chapel Hill in Fall 2024. Before that, he will be a Postdoctoral Researcher at Massachusetts Institute of Technology (CSAIL@MIT), Harvard (BMI@Harvard), and Borad Institute of MIT & Harvard in 2023-2024.

His research focuses on building accurate, trustworthy, and efficient machine learning systems. He devotes his most recent passion to various (A) important machine learning problems - sparsity, robustness, learning to optimize, graph learning, and diffusion models; (B) interdisciplinary scientific challenges - bioengineering and quantum comptuing. He received IBM Ph.D. Fellowship, Adobe Ph.D. Fellowship, Graduate Dean's Prestigious Fellowship, and the Best Paper Award from the inaugural Learning on Graphs (LoG) Conference 2022.

I am looking for highly motivated students, in terms of RA/TA/externship/internship/visiting students. Interested candidates are strongly encouraged to contact me by email, together with resume and transcripts.

News

• Apr., 2024    [PAPER] One Bioinformatics accepted - scRNA Imputation via Bi-level Feature Propagation.
• Apr., 2024    [Tutorial] Our tutorial of "Mixture-of-Experts in the Era of LLMs: A New Odyssey" has been accepted by ICML'24. See you at Vienna!
• Mar., 2024    [PAPER] One NAACL'24 accepted - Multiple LLM Agents Gaming.
• Mar., 2024    [STATUS] UNITES welcomes five new Ph.D. students: Sukwon Yun (KAIST), Xinyu Zhao (Peking University), Pingzhi Li (USTC), Mufan Qiu (USTC), and Huanzhi Qu (USTC).
• Mar., 2024    [PAPER] Two CVPR'24 accepted - Diffusion Quantization + Molecular Data Programming.
• Feb., 2024    [PAPER] Two ICLR'24 accepted - SMoE Merging (Spotlight) + Multilingual SMoE. Congratulations to Pingzhi Li and Xinyu Zhao for their first ICLR papers.
• Jan., 2024    [Tutorial] Our tutorial of "Zeroth-Order Machine Learning: Fundamental Principles and Emerging Applications in Foundation Models" has been accepted by AAAI'24 and ICASSP'24.
• Jan., 2024    [PAPER] One EACL'24 accepted - LLM + Knowledge Graph.

• Dec., 2023    [PAPER] One AAAI'24 accepted - Metacognitive Intervention for LLMs via Sparsity.
• Nov., 2023    [HONOR] Congratulations to Pingzhi Li and our amazing collaborators for wining the 1st place at 2023 Quantum Computing for Drug Discovery Challenge at ICCAD.
• Sep., 2023    [PAPER] Two NeurIPS'23 accepted - Essential Sparsity in LLM + LLM Heavy-Hitter Oracle.
• Jul., 2023    [STATUS] I started my PostDoc at CSAIL@MIT and BMI@Harvard.
• Jul., 2023    [PAPER] Three ICCV'23 accepted - Adaptive Multi-Task Vision MoE + Robust MoE (Oral) + Generalizable NeRF w. MoE.
• Jul., 2023    [PAPER] One QCE'23 accepted - Sparse Circuit Design for Quantum Computing.
• Jun., 2023    [HONOR] I received the 2023 AdvML Rising Star Award . Many thanks for the acknowledgment.
• May, 2023    [PAPER] One ACL'23 accepted - Sparse LLM Tuning.
• May, 2023    [PAPER] Three ICML'23 accepted - Instant Soup (Oral) + Graph Ladling + L2O Game.
• May, 2023    [STATUS] I received the Ph.D. degree from ECE@UT Austin. I deeply appreciate all the support and help from my family, advisor (Prof. Atlas Wang) , collaborators, and friends!

UNITES Lab

University of North Carolina, AI Trustworthiness, Efficiency, and for Science (UNITES) Group will be an active research lab at UNC Chapel Hill. Our research interests span the area of artificial intelligence (AI), machine learning (ML), optimization, computer vision, natural language processing, and data science, with two major focuses on (A) establishing robust and efficient AI systems; (B) bridging the gap between AI and societal & scientific challenges. Students' information is presented below.

Pingzhi Li, B.E., USTC, Aug. 2024 -
Mufan Qiu, B.S., USTC, Aug. 2024 -
Huaizhi Qu, B.E., USTC, Aug. 2024 -
Xinyu Zhao, M.S., Peking Uiversity, Aug. 2024 -
Sukwon Yun, M.S., KAIST, Aug. 2024 -

Mohan Zhang (USTC, Spring 2024) [Remote]
Haiyang Xu (USTC, Spring 2024) [Remote]
Daiwei Li (UCSD, Summer 2023, Spring 2024) [Remote]
Jialin Wu (USTC, Summer 2023, Spring 2024) [Remote]
Xinnan Zhang (USTC, Summer 2023, Spring 2024) [Remote]
Tiancheng He (SJTU+UMich, Summer 2023, Spring 2024) [Remote]
Yaoyao Xu (Columbia, Summer 2023, Spring 2024) [Remote]
Can Jin (USTC, Summer 2023) [Remote]
Kaixin Zheng (Summer 2023) [Remote]
Silin Cai (Summer 2023) [Remote]

Publications

Full publications on Google Scholar.
‡ indicates authors with equal contribution. ☆ indicates my students or interns.

MARSBench: Evaluating Multi-Agent Multi-Turn Strategic Reasoning of Large Language Models and Beyond

Jinhao Duan, Shiqi Wang, James Diffenderfer, Lichao Sun, Tianlong Chen, Bhavya Kailkhura, Kaidi Xu

NAACL'24: The North American Chapter of the Association for Computational Linguistics

Paper Code

Molecular Data Programming: Towards Molecule Pseudo-labeling with Systematic Weak Supervision

Xin Juan, Kaixiong Zhou, Ninghao Liu, Tianlong Chen, Xin Wang

CVPR'24: Conference on Computer Vision and Pattern Recognition

Paper Code

TFMQ-DM: Temporal Feature Maintenance Quantization for Diffusion Models

Yushi Huang, Ruihao Gong, Jing Liu, Tianlong Chen, Xianglong Liu

CVPR'24: Conference on Computer Vision and Pattern Recognition

Paper Code

Merge, Then Compress: Demystify Efficient SMoE with Hints from Its Routing Policy

Pingzhi Li☆, Zhenyu Zhang, Prateek Yadav, Yi-Lin Sung, Yu Cheng, Mohit Bansal, Tianlong Chen

ICLR'24: International Conference on Learning Representations

Paper Code

Sparse Moe with Language Guided Routing for Multilingual Machine Translation

Xinyu Zhao☆, Xuxi Chen, Yu Cheng, Tianlong Chen

ICLR'24: International Conference on Learning Representations

Paper Code

Contextualization Distillation from Large Language Model for Knowledge Graph Completion

Dawei Li☆, Zhen Tan, Tianlong Chen, Huan Liu

EACL'24: European chapter of the Association for Computational Linguistics

Paper Code

Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention

Zhen Tan, Tianlong Chen, Zhenyu Zhang, Huan Liu

AAAI'24: Association for the Advancement of Artificial Intelligence

Paper Code

The Emergence of Essential Sparsity in Large Pre-trained Models: The Weights that Matter

Ajay Jaiswal, Shiwei Liu, Tianlong Chen, Zhangyang Wang

NeurIPS'23: Advances in Neural Information Processing Systems

Paper Code

H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models

Zhenyu Zhang, Ying Sheng, Tianyi Zhou, Tianlong Chen, Lianmin Zheng, Ruisi Cai, Zhao Song, Yuandong Tian, Christopher Ré, Clark Barrett, Zhangyang Wang, Beidi Chen

NeurIPS'23: Advances in Neural Information Processing Systems

Paper Code

AdaMV-MoE: Adaptive Multi-Task Vision Mixture-of-Experts

Tianlong Chen‡, Xuxi Chen‡, Xianzhi Du, Abdullah Rashwan, Fan Yang, Huizhong Chen, Zhangyang Wang, Yeqing Li

ICCV'23: International Conference on Computer Vision

Paper Code

Robust Mixture-of-Expert Training for Convolutional Neural Networks

Yihua Zhang, Ruisi Cai, Tianlong Chen, Guanhua Zhang, Huan Zhang, Pin-Yu Chen, Shiyu Chang, Zhangyang Wang, Sijia Liu

ICCV'23 (Oral): International Conference on Computer Vision

Paper Code

GNT-MOVE: Generalizable NeRF Transformer with Mixture-of-View-Experts

Wenyan Cong, Hanxue Liang, Peihao Wang, Zhiwen Fan, Tianlong Chen, Mukund Varma, Yi Wang, Zhangyang Wang

ICCV'23: International Conference on Computer Vision

Paper Code

QuantumSEA: In-Time Sparse Exploration for Noise Adaptive Quantum Circuits

Tianlong Chen, Zhenyu Zhang, Hanrui Wang, Jiaqi Gu, Zirui Li, David Z. Pan, Frederic Chong, Song Han, Zhangyang Wang

QCE'23: International Conference on Quantum Computing and Engineering

Paper Code

DSEE: Dually Sparsity-embedded Efficient Tuning of Pre-trained Language Models

Xuxi Chen, Tianlong Chen, Weizhu Chen, Ahmed Hassan Awadallah, Zhangyang Wang, Yu Cheng

ACL'23: Annual Meeting of the Association for Computational Linguistics

Paper Code

Instant Soup: Cheap Pruning Ensemble in A Single Pass Can Draw Lottery Tickets from Large Models

Ajay Jaiswal, Shiwei Liu, Tianlong Chen, Ying Ding, Zhangyang Wang

ICML'23: International Conference on Machine Learning

Paper Code

Graph Ladling: Shockingly Simple Parallel GNN Training without Intermediate Communication

Ajay Jaiswal, Shiwei Liu, Tianlong Chen, Ying Ding, Zhangyang Wang

ICML'23: International Conference on Machine Learning

Paper Code

Learning to Optimize Differential Games

Xuxi Chen, Nelson Vadori, Tianlong Chen, Zhangyang Wang

ICML'23: International Conference on Machine Learning

Paper Code
Fundamental Sparse Model and Algorithms

The Lottery Ticket Hypothesis for Pre-trained BERT Networks

Tianlong Chen, Jonathan Frankle, Shiyu Chang, Sijia Liu, Yang Zhang, Zhangyang Wang, Michael Carbin

NeurIPS'20: Conference on Neural Information Processing Systems

Paper Code

Sparsity May Cry: Let Us Fail (Current) Sparse Neural Networks Together!

Shiwei Liu‡, Tianlong Chen‡, Zhenyu Zhang, Xuxi Chen, Tianjin Huang, Ajay Kumar Jaiswal, Zhangyang Wang

ICLR'23: International Conference on Learning Representations

Paper Code
Data-driven Discovery of Optimization Algorithms

Learning to Optimize: A Primer and A Benchmark

(α-β) Tianlong Chen, Xiaohan Chen, Wuyang Chen, Howard Heaton, Jialin Liu, Zhangyang Wang, Wotao Yin

JMLR'22: Journal of Machine Learning Research

Paper Code

Scalable Learning to Optimize: A Learned Optimizer Can Train Big Models

Xuxi Chen‡, Tianlong Chen‡, Yu Cheng, Weizhu Chen, Ahmed Awadallah, Zhangyang Wang

ECCV'22: European Conference on Computer Vision

Paper Code
Responsible and Reliable Machine Learning

Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning

Tianlong Chen, Sijia Liu, Shiyu Chang, Yu Cheng, Lisa Amini, Zhangyang Wang

CVPR'20: Conference on Computer Vision and Pattern Recognition

Paper Code

Linearity Grafting: How Neuron Pruning Helps Certifiable Robustness

Tianlong Chen‡, Huan Zhang‡, Zhenyu Zhang, Shiyu Chang, Sijia Liu, Pin-Yu Chen, Zhangyang Wang

ICML'22: International Conference on Machine Learning

Paper Code
Scalable Graph Neural Networks

Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Study

Tianlong Chen‡, Kaixiong Zhou‡, Keyu Duan, Wenqing Zheng, Peihao Wang, Xia Hu, Zhangyang Wang

TPAMI'22: IEEE Transactions on Pattern Analysis and Machine Intelligence

Paper Code

You Can Have Better Graph Neural Networks by Not Training Weights at All: Finding Untrained GNNs Tickets

Tianjin Huang, Tianlong Chen, Meng Fang, Vlado Menkovski, Jiaxu Zhao, Lu Yin, Yulong Pei, Decebal Constantin Mocanu, Zhangyang Wang, Mykola Pechenizkiy, Shiwei Liu

(Best Paper Award) LOG'22: Learning on Graphs Conference

Paper Code

Graph Contrastive Learning with Augmentations

Yuning You‡, Tianlong Chen‡, Yongduo Sui, Ting Chen, Zhangyang Wang, Yang Shen

NeurIPS'20: Conference on Neural Information Processing Systems

Paper Code
AI4Science

HotProtein: A Novel Framework for Protein Thermostability Prediction and Editing

Tianlong Chen‡, Chengyue Gong‡, Daniel Jesus Diaz, Xuxi Chen, Jordan Tyler Wells, qiang liu, Zhangyang Wang, Andrew Ellington, Alex Dimakis, Adam Klivans

ICLR'23: International Conference on Learning Representations

Paper Code

QuantumSEA: In-Time Sparse Exploration for Noise Adaptive Quantum Circuits

Tianlong Chen, Zhenyu Zhang, Hanrui Wang, Jiaqi Gu, Zirui Li, David Z. Pan, Frederic Chong, Song Han, Zhangyang Wang

QCE'23: International Conference on Quantum Computing and Engineering

Paper Code

Vitæ

Full Resume in PDF.

More About Me

• I am a big fan of Pokémon. Playing Pokémon Go is one of my daily activities. Join me, catch the Pokémon in North Carolina, and be a good Pokémon trainer!
• I also enjoy Hip-Hop and Country music. Air (艾热) is one of my favorite Chinese Hip-Hop stars.

This website was built with jekyll based on a template by Martin Saveski.
Tianlong Chen
Last updated on July 2023