Hi, I am a PhD student at LANCO group, supervised by Prof. Xu Sun. My research interests lie in interpreting LLMs’ working mechanisms under various scenes. Previously, I completed my bachelor’s degree at Peking University.

Education

• PhD Student, Peking University, 2023 - Now
• B.S. in Intelligence Science and Technology (Summa Cum Laude), Peking University, 2019 - 2023.

Internship

• Wechat AI, Research Intern, Dec. 2022 - Now

Preprints

(# denotes Equal Contribution)

• Towards Codable Text Watermarking for Large Language Models
  Lean Wang#, Wenkai Yang#, Deli Chen#, Hao Zhou, Yankai Lin, Fandong Meng, Jie Zhou, Xu Sun
  [arxiv, code]

Publication

• Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning
  Lean Wang, Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng, Jie Zhou, Xu Sun EMNLP 2023 Best Long Paper Award
  [arxiv, code]

• Gradient Knowledge Distillation for Pre-trained Language Models
  Lean Wang, Lei Li, Xu Sun
  NeurIPS ENLSP 2022 workshop (spotlight)
  [arxiv, code]

Academic Service

• Teaching Assistant:
  • Probability Theory and Statistics (A) (2023 Spring, PKU)

Awards

• Best Long Paper Award, EMNLP, 2023