Tianyi Chen

Tianyi Chen Assistant Professor Department of Electrical, Computer, and Systems Engineering Rensselaer Polytechnic Institute Office: Room 6036, Jonsson Engineering Center Address: 110 8th Street, Troy, New York, 12180-3590 Tel: 518-276-3173 E-mail: chent18 [@] rpi.edu;  chentianyi19 [@] gmail.com (preferred) Google Scholar ResearchGate

About me

Tianyi Chen has been with Rensselaer Polytechnic Institute (RPI) as an assistant professor since August 2019. In RPI, I am also an active member in the Rensselaer-IBM Artificial Intelligence Research Collaboration (AIRC). Prior to joining RPI, he received the doctoral degree from the University of Minnesota under the supervision of Prof. Georgios B. Giannakis.

Dr. Chen is the inaugural recipient of IEEE Signal Processing Society Best PhD Dissertation Award in 2020, a recipient of NSF CAREER Award in 2021 and a recipient of Amazon Research Award in 2022. He is also a co-author of the Best Student Paper Award at the NeurIPS Federated Learning Workshop in 2020 and at IEEE ICASSP in 2021.

Research interests: theoretical foundations of optimization and machine learning, with a focus on their applications to emerging data processing and computing paradigms.

Education

• University of Minnesota, Twin Cities, USA: Ph.D, Electrical and Computer Engineering, 2014.9 - 2019.6

• University of Minnesota, Twin Cities, USA: M.S., Electrical and Computer Engineering, 2014.9 - 2016.6

• Fudan University, China: B.S., Communication Science and Engineering, 2010.9 - 2014.7

Recent news

• Oct 2024: Two recent preprints on LLM post-training via bilevel/bi-objective optimization:

  SEAL: Safety-enhanced Aligned LLM Fine-tuning via Bilevel Data Selection.
  Mitigating Forgetting in LLM Supervised Fine-Tuning and Preference Learning.

• Oct 2024: I am invited to present our work at The FIND Seminar, Cornell University.

• Oct 2023: I am serving as an Area Chair for AISTATS 2025 and ICLR 2025.

• Sep 2024: Three papers have been accepted to NeurIPS 2024: preference-guided learning, analog training, and bilevel optimization.

• Sep 2024: I presented a tutorial on multi-objective learning at the MLSP 2024 and a seminar on analog training at King's College London.

• Aug 2024: Two interesting ideas (one on bilevel optimization landscape and another on analog training) our group is exploring:

  Towards Exact Gradient-based Training on Analog In-memory Computing.
  Unlocking Global Optimality in Bilevel Optimization: A Pilot Study.

• Aug 2024: We are excited to receive a new collaborative NSF project on bilevel optimization for graph representation learning:

  BLoG: A Bi-Level Optimization Framework for Learning Over Graphs.

• May 2024: Two papers have been accepted to ICML 2024 and one paper has been accepted to JMLR; see two preprints:

  Principled Penalty-based Methods for Bilevel Reinforcement Learning and RLHF.
  Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidance.

• Apr 2024: We presented a tutorial at ICASSP 2024 on Learning with Multiple Objectives and Applications to Speech Processing; see here.

• Mar 2024: My Ph.D. student Lisha Chen will join the Department of Electrical and Computer Engineering, University of Rochester in 2025 as a tenure-track Assistant Professor!

• Mar 2024: I am invited to present our work at the Department of Electrical and Computer Engineering, Rutgers University.

• Feb 2024: We presented a tutorial at AAAI 2024 on Learning with Multiple Objectives and Applications to Reinforcement Learning; see here.

• Feb 2024: Congrats to Quan Xiao on the Belsky Award for Computational Sciences and Engineering!

• Jan 2024: One paper has been accepted to AISTATS 2024:

  Enhancing In-context Learning via Linear Probe Calibration. Congrats to Momin!

• Dec 2023: Three of our papers were accepted in ICASSP 2024; see e.g., here.

• Oct 2023: I am invited to present our work at the Department of Electrical and Computer Engineering, Northeastern University.

• Oct 2023: I am invited to serve as an Area Chair for AISTATS 2024 and CPAL 2024.

• Sep 2023: Two papers have been accepted to NeurIPS 2023:

  Three-Way Trade-Off in Multi-Objective Learning: Optimization, Generalization and Conflict-Avoidance.
  A Generalized Alternating Method for Bilevel Learning under the Polyak-Lojasiewicz Condition.

• Sep 2023: I am invited to present our work on multi-objective learning at a departmental seminar from Department of Industrial and Systems Engineering, Lehigh University.

• Aug 2023: Our paper on lazy query for zeroth-order optimization has been accepted in TSP:

  Lazy Queries Can Reduce Variance in Zeroth-order Optimization.

• Jun 2023: I am invited to present our work on bilevel optimization at SIAM Conference on Optimization and MOPTA.

• Apr 2023: One paper on the inexact penalization method for bilevel optimization has been accepted in ICML 2023:

  On Penalty-based Bilevel Gradient Descent Method. Congrats to Han!

• Apr 2023: Our paper on linear speedup analysis of the popular A3C algorithm in RL has been accepted in TSP:

  Asynchronous Advantage Actor-Critic: Non-asymptotic Analysis and Linear Speedup.

• Feb 2023: Our paper on ensemble methods for bilevel learning was accepted in ICASSP 2023. Congrats to Lisha!

• Feb 2023: I am invited to present our work on bilevel optimization for distributed learning at EnCORE workshop at ITA 2023.

• Jan 2023: One paper has been selected as an oral paper (notable-top-5%) in ICLR 2023:

  Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach. Congrats to Heshan!

• Jan 2023: Two papers have been accepted to AISTATS 2023:

  Alternating Implicit Projected SGD and Its Efficient Variants for Equality-constrained Bilevel Optimization. Congrats to Quan!
  Distributed Offline Policy Optimization Over Batch Data. Congrats to Han!

• Dec 2022: With Atlas Wang, we will present a tutorial at ICASSP 2023 on Bilevel Optimization and Its Applications to Machine Learning.

• Nov 2022: We are honored to receive the Faculty Research Gifts from Cisco Research.

• Nov 2022: We are happy to receive several IBM-RPI AI research projects, part of the IBM AI Horizons Network.

• Oct 2022: Our monograph with Osvaldo's group on Meta Learning and Applications to Communications is published in Foundations and Trends in Signal Processing.

• Oct 2022: We are honored to receive the Amazon Research Award - AWS AI.

• Sep 2022: Two papers have been accepted to NeurIPS 2022 with one being selected as an oral presentation:

  Understanding Benign Overfitting in Gradient-based Meta Learning. Congrats to Lisha!
  A Single-Timescale Analysis For Stochastic Approximation With Multiple Coupled Sequences (oral). Congrats to Han!

• Sep 2022: I am invited to serve as an Area Chair for AISTATS 2023.

• Aug 2022: We have organized a Cross-Community Federated Learning (CrossFL) workshop at MLSys 2022; see the recording [link].

• May 2022: Our paper has been accepted to ICML 2022! Congrats to the teamwork among Momin, Quan and Lisha!

  Sharp-MAML: Sharpness-Aware Model-Agnostic Meta Learning.

• Mar 2022: Congrats to Lisha on the inaugural Belsky Award for Computational Sciences and Engineering!

• Jan 2022: Two papers have been accepted to AISTATS 2022 with one being selected as an oral presentation:

  A Single-Timescale Method for Stochastic Bilevel Optimization (oral).
  Is Bayesian Model-Agnostic Meta Learning Better than Model-Agnostic Meta Learning, Provably? (poster).

• Jan 2022: Our paper with Prof. Ali Tajer's group was accepted in ICASSP 2022 on federated bandit learning with uncoordinated exploration.

• Dec 2021: I will co-organize a workshop at MLSys 2022 with Pin-Yu Chen, Carlee Joe-Wong, and Nathalie Baracaldo on

  Cross-Community Federated Learning: Algorithms, Systems and Co-designs. [link]

• Nov 2021: I will co-present a tutorial at ICASSP 2022 with Prof. Osvaldo Simeone on

  Meta Learning and Applications to Communications.

• Sep 2021: Two papers have been accepted to NeurIPS 2021 with one being selected as a spotlight presentation:

  Closing the Gap: Tighter Analysis of Alternating Stochastic Gradient Methods for Bilevel Problems (spotlight).
  CAFE: Catastrophic Data Leakage in Vertical Federated Learning (poster).

• Sep 2021: Our paper was accepted in IEEE Trans on Pattern Analysis and Machine Intelligence:

  Adaptive Temporal Difference Learning with Linear Function Approximation.

• Sep 2021: I am invited to the Panel Session of the "Distributed ML workshop'' co-located with ACM CoNEXT 2021; see the program.

• Sep 2021: With Derya Malak, we are organizing a workshop on "Wireless Distributed Computing and Learning'' at WiOPT 2021.

• Aug 2021: I am excited to collaborate with Rongjie Lai (RPI) and Jie Chen (MIT-IBM AI Lab) for the NSF SCALE MoDL Program on:

  Representation Learning via Variational Mean Field Theory.

• Jul 2021: Our paper was accepted in IEEE Transactions on Signal Processing:

  LASG: Lazily Aggregated Stochastic Gradients for Communication-Efficient Distributed Learning.

• Jun 2021: Our paper was recognized as the Outstanding Student Paper Award for ICASSP 2021:

  A Stochastic Compositional Optimization Method with Applications to Meta Learning. [UCLA Math News]

• Jun 2021: Two papers were accepted in IEEE Transactions on Signal Processing:

  Solving Stochastic Compositional Optimization is Nearly As Easy As Solving Stochastic Optimization.
  Byzantine-Resilient Decentralized TD Learning with Linear Function Approximation.

• May 2021: I am honored to receive the NSF CAREER Award on:

  The Co-design of Distributed Machine Learning Algorithms and Wireless Systems. [RPI News]

• Apr 2021: Our paper was accepted in IEEE Transactions on Control of Network systems:

  Communication-Efficient Policy Gradient Methods for Distributed Reinforcement Learning.

• Feb 2021: I am organizing the special session ``Reinforcement learning over networks'' at Asilomar 2021.

• Feb 2021: New paper on the first single-timescale SGD algorithms for stochastic bilevel optimization:

  A Single-Timescale Stochastic Bilevel Optimization Method.

• Jan 2021: Our papers were accepted in ICASSP 2021 on meta learning and reinforcement learning.

• Jan 2021: Our paper was accepted in AISTATS 2021:

  CADA: Communication-Adaptive Distributed Adam.

• Jan 2021: My Ph.D dissertation was recognized as the inaugural IEEE SPS Best PhD Dissertation Award:

  Efficient Methods for Distributed Machine Learning and Resource Management in IoT. [UMN News] [RPI ECSE News] [RPI News]

• Dec 2020: New paper on linear speedup analysis of the popular A3C algorithm in RL:

  Asynchronous Advantage Actor Critic: Non-asymptotic Analysis and Linear Speedup.

• Dec 2020: Our paper was accepted in AAAI 2021:

  Decentralized policy gradient descent ascent for safe multi-agent reinforcement learning.

• Dec 2020: Our paper was recognized as the Best Student Paper Award at NeurIPS 2020 workshop on Federated Learning:

  Hybrid Federated Learning: Algorithms and Implementation. [UCLA Math News]

• Sep 2020: Our paper was accepted in IEEE Trans on Pattern Analysis and Machine Intelligence:

  Lazily Aggregated Quantized Gradient Innovation for Communication-Efficient Federated Learning.

• Aug 2020: New paper on simple stochastic algorithm for learning with compositional structures (e.g., meta learning and RL):

  Solving Stochastic Compositional Optimization is Nearly as Easy as Solving Stochastic Optimization.

• May 2020: I am invited to serve as a program committee member for ICML 2020, NeurIPS 2020 and ICLR 2021.

• May 2020: New paper on Byzantine-robust federated learning has been accepted in IEEE Transactions on Signal Processing:

  Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attacks.

• Mar 2020: I am organizing the special session ``Reinforcement learning for communication systems'' at Asilomar 2020.

• Feb 2020: New paper on adaptive temporal-difference learning and [Python code]:

  Adaptive Temporal Difference Learning with Linear Function Approximation.

• Jan 2020: Our paper on variance-reduced robust stochastic gradient descent has been accepted in ICASSP 2020.

• Jan 2020: I am invited to serve as a program committee member for ACM MobiHoc 2020.

• Dec 2019: Presented our work on Lazily Aggregated Quantized Gradients at NeurIPS 2019: [Paper] and [Poster]

• Oct 2019: Our paper has been accepted in IEEE Transactions on Signal Processing:

  Secure Mobile Edge Computing in IoT via Collaborative Online Learning.

• Sept 2019: Our paper was accepted at NeurIPS 2019:

  Communication-Efficient Distributed Learning via Lazily Aggregated Quantized Gradients. [Python code]

• Aug 2019: I am excited to join Department of Electrical, Computer, and Systems Engineering at Rensselaer Polytechnic Institute.

Sponsors