Home

My name is Dayiheng Liu (刘大一恒). I am currently an Algorithm Expert in the Alibaba DAMO. I received my Ph.D. at DICALab, Sichuan University, advised by Prof. Jiancheng Lv. I led an NLP group of DICALab which focuses on text generation. I joined the Alibaba DAMO Academy in January 2021.

I have published over 50 papers in leading NLP/AI journals and conferences such as ACL/ICML/NeurIPS/EMNLP/AAAI/CIKM/NAACL. I have participated in some Github projects (over 5K Github stars) as a core contributor, such as Qwen (通义千问), PolyLM, ProphetNet, and GLGE. We also released some LLM models (over 1 million downloads to date) on Huggingface and Modelscope, such as Qwen-7B, Qwen-14B, and PolyLM-13B.

Interests: Large Language Models (LLM), Natural Language Generation (NLG)

Honors and Awards

Ranked 1st in C-Eval Leaderboard, Sep 11, 2023
Winner of WMT 2022 Metrics shared Tasks, 2022
Winner of WMT 2022 QE Multilingual Tasks, 2022
Winner of WMT 2021 Metrics shared Tasks, 2021
Ranked 1st in CommonGen Leaderboard, May 18, 2021
Alibaba Star (Top 1%), Alibaba Group, 2021
List of Top 100 Chinese Rising Stars in AI (AI华人新星百强), 2021
Excellent Doctoral Thesis of Sichuan University (10 PhDs/year), Sichuan University, 2020
Ranked 1st in Google’s Natural Questions Leaderboard, Nov 19, 2019
ACM-ICPC Asian Regional Competition Silver Medal, Asian, 2015
CCF CSP (Certified Software Professional）C/C++ 430 scores (Top 0.1%), China, 2015
Computer Science and Technology Star (1/370+), Sichuan University, 2015
Winner of Chengdu International Software Design Competition (Top1), China, 2015

Publications

Unifying Discrete and Continuous Representations for Unsupervised Paraphrase Generation
Mingfeng Xue, Dayiheng Liu* (Corresponding author), Wenqiang Lei, Jie Fu, Jian Lan, Mei Li, Baosong Yang, Jun Xie, Yidan Zhang, Dezhong Peng, Jiancheng Lv
EMNLP 2023

Dynamic Voting for Efficient Reasoning in Large Language Models
Mingfeng Xue, Dayiheng Liu* (Corresponding author), Wenqiang Lei, Xingzhang Ren, Baosong Yang, Jun Xie, Yidan Zhang, Dezhong Peng, Jiancheng Lv
Findings of EMNLP 2023

Noisy Pair Corrector for Dense Retrieval
Hang Zhang, Yeyun Gong, Xingwei He, Dayiheng Liu, Daya Guo, Jiancheng Lv, Jian Guo
Findings of EMNLP 2023

MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization
Yuyan Chen, Zhihao Wen, Ge Fan, Zhengyu Chen, Wei Wu, Dayiheng Liu, Zhixu Li, Bang Liu, Yanghua Xiao
Findings of EMNLP 2023

Qwen Technical Report
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang, Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang, Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, Tianhang Zhu
arXiv 2023

EMMA-X: An EM-like Multilingual Pre-training Algorithm for Cross-lingual Representation Learning
Ping Guo, Xiangpeng Wei, Yue Hu, Baosong Yang, Dayiheng Liu, Fei Huang, Jun Xie
NeurIPS 2023

Polylm: An Open Source Polyglot Large Language Model
Xiangpeng Wei, Haoran Wei, Huan Lin, Tianhao Li, Pei Zhang, Xingzhang Ren, Mei Li, Yu Wan, Zhiwei Cao, Binbin Xie, Tianxiang Hu, Shangjie Li, Binyuan Hui, Bowen Yu, Dayiheng Liu* (Corresponding author), Baosong Yang, Fei Huang, Jun Xie
arXiv 2023

Hallucination Detection: Discerning Reliable Responses in Large Language Models
Yuyan Chen, Qiang Fu, Yichen Yuan, Zhihao Wen, Ge Fan, Zhixu Li, Yanghua Xiao, Dongmei Zhang and Dayiheng Liu
CIKM 2023

Fantastic Expressions and Where to Find Them: Chinese Simile Generation with Multiple Constraints
Kexin Yang, Dayiheng Liu* (Corresponding author), Wenqiang Lei, Baosong Yang, Xiangpeng Wei, Zhengyuan Liu, Jun Xie
ACL 2023

Bridging the Domain Gaps in Context Representations for k-Nearest Neighbor Neural Machine Translation-Nearest Neighbor Neural Machine Translation
Zhiwei Cao, Baosong Yang, Huan Lin, Suhang Wu, Xiangpeng Wei, Dayiheng Liu, Jun Xie, Min Zhang, Jinsong Su
ACL 2023

Tailor: A Soft-Prompt-Based Approach to Attribute-Based Controlled Text Generation
Kexin Yang, Dayiheng Liu* (Corresponding author), Wenqiang Lei, Baosong Yang, Mingfeng Xue, Boxing Chen, Jun Xie
ACL 2023

Interactive Natural Language Processing
Zekun Wang, Ge Zhang, Kexin Yang, Ning Shi, Wangchunshu Zhou, Shaochun Hao, Guangzheng Xiong, Yizhi Li, Mong Yuan Sim, Xiuying Chen, Qingqing Zhu, Zhenzhu Yang, Adam Nik, Qi Liu, Chenghua Lin, Shi Wang, Ruibo Liu, Wenhu Chen, Ke Xu, Dayiheng Liu, Yike Guo, Jie Fu
arXiv 2023

Towards Fine-grained Information: Identifying the Type and Location of Translation Errors
Keqin Bao, Yu Wan, Dayiheng Liu, Baosong Yang, Wenqiang Lei, Xiangnan He, Derek F Wong, Jun Xie
arXiv 2023

Effective Approaches to Neural Query Language Identification
Xingzhang Ren, Baosong Yang, Dayiheng Liu, Haibo Zhang, Xiaoyu Lv, Liang Yao, Jun Xie
Computational Linguistics 2022

Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality?
Pei Zhang, Baosong Yang, Haoran Wei, Dayiheng Liu, Kai Fan, Luo Si, Jun Xie
EMNLP 2022

Alibaba-Translate China’s Submission for WMT 2022 Metrics Shared Task
Yu Wan, Keqin Bao, Dayiheng Liu, Baosong Yang, Derek F Wong, Lidia S Chao, Wenqiang Lei, Jun Xie
WMT 2022

Alibaba-Translate China’s Submission for WMT 2022 Quality Estimation Shared Task
Yu Wan, Keqin Bao, Dayiheng Liu, Baosong Yang, Derek F Wong, Lidia S Chao, Wenqiang Lei, Jun Xie
WMT 2022

Draft, Command, and Edit: Controllable Text Editing in E-Commerce
Kexin Yang, Dayiheng Liu* (Corresponding author), Wenqiang Lei, Baosong Yang, Qian Qu, Jiancheng Lv
arXiv 2022

CoupGAN: Chinese Couplet Generation via Encoder–decoder Model and Adversarial Training under Global Control
Qian Qu, Jiancheng Lv, Dayiheng Liu, Kexin Yang
Soft Computing 2022

Should We Rely on Entity Mentions for Relation Extraction? Debiasing Relation Extraction with Counterfactual Analysis
Yiwei Wang, Muhao Chen, Wenxuan Zhou, Yujun Cai, Yuxuan Liang, Dayiheng Liu, Baosong Yang, Juncheng Liu, Bryan Hooi
NAACL 2022

Dangling-Aware Entity Alignment with Mixed High-Order Proximities
Juncheng Liu, Zequn Sun, Bryan Hooi, Yiwei Wang, Dayiheng Liu, Baosong Yang, Xiaokui Xiao, Muhao Chen
Findings of NAACL 2022

Bridging the Gap between Training and Inference: Multi-Candidate Optimization for Diverse Neural Machine Translation
Huan Lin, Baosong Yang, Liang Yao, Dayiheng Liu, Haibo Zhang, Jun Xie, Min Zhang, Jinsong Su
Findings of NAACL 2022

Self-supervised Product Title Rewrite for Product Listing Ads
Xue Zhao, Dayiheng Liu, Mahone Yan, Liang Yao, Wang Huibo Gary, Junwei Ding, Wenqing Yao
NAACL 2022, Track Industry

UniTE: Unified Translation Evaluation
Yu Wan, Dayiheng Liu, Baosong Yang, Haibo Zhang, Boxing Chen, Derek F. Wong, Lidia S. Chao
Association for Computational Linguistics. ACL 2022

GCPG: A General Framework for Controllable Paraphrase Generation
Kexin Yang, Dayiheng Liu* (Corresponding author), Wenqiang Lei, Baosong Yang, Haibo Zhang, Xue Zhao, Wenqing Yao, Boxing Chen
Findings of ACL 2022

Attention Mechanism with Energy-Friendly Operations
Yu Wan, Baosong Yang, Dayiheng Liu, Rong Xiao, Derek F. Wong, Haibo Zhang, Boxing Chen, Lidia S. Chao
Findings of ACL 2022

Unsupervised Preference-Aware Language Identification
Xingzhang Ren, Baosong Yang, Dayiheng Liu, Haibo Zhang, Xiaoyu Lv, Liang Yao, Jun Xie
Findings of ACL 2022

RMBR: A Regularized Minimum Bayes Risk Reranking Framework for Machine Translation
Yidan Zhang, Yu Wan, Dayiheng Liu, Baosong Yang, Zhenan He
arXiv 2022

KGR4: Retrieval, Retrospect, Refine and Rethink for Commonsense Generation
Xin Liu, Dayiheng Liu, Baosong Yang, Haibo Zhang, Junwei Ding, Wenqing Yao, Weihua Luo, Jinsong Su
AAAI 2022

Frequency-Aware Contrastive Learning for Neural Machine Translation
Tong Zhang, Wei Ye, Baosong Yang, Long Zhang, Xingzhang Ren, Dayiheng Liu, Jinan Sun, Shikun Zhang, Haibo Zhang, Wen Zhao
AAAI 2022

Prediction, Selection, and Generation: A Knowledge-driven Conversation System
Luo Cheng, Dayiheng Liu, Chanjuan Li, Jiancheng Lv, Li Lu
Neural Computing and Applications. NCAA 2022

GLGE: A New General Language Generation Evaluation Benchmark
Dayiheng Liu, Yu Yan, Yeyun Gong, Weizhen Qi, Hang Zhang, Jian Jiao, Weizhu Chen, Jie Fu, Linjun Shou, Ming Gong, Pengcheng Wang, Jiusheng Chen, Daxin Jiang, Jiancheng Lv, Ruofei Zhang, Winnie Wu, Ming Zhou, Nan Duan
Findings of ACL 2021

POS-constrained Parallel Decoding for Non-autoregressive Generation
Kexin Yang, Wenqiang Lei, Dayiheng Liu, Weizhen Qi and Jiancheng Lv
ACL 2021

Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generation
Xin Liu, Baosong Yang, Dayiheng Liu, Haibo Zhang, Weihua Luo, Min Zhang, Haiying Zhang and Jinsong Su
ACL 2021

Towards User-Driven Neural Machine Translation
Huan Lin, Liang Yao, Baosong Yang, Dayiheng Liu, Haibo Zhang, Weihua Luo, Degen Huang and Jinsong Su
ACL 2021

BANG: Bridging Autoregressive and Non-autoregressive Generation with Large Scale Pretraining
Weizhen Qi, Yeyun Gong*, Jian Jiao*, Yu Yan*, Dayiheng Liu* (equal contribution), Weizhu Chen*, Kewen Tang, Houqiang Li, Jiusheng Chen, Ruofei Zhang, Ming Zhou, Nan Duan
ICML 2021

Mask Attention Networks: Rethinking and Strengthen Transformer
Zhihao Fan, Yeyun Gong, Dayiheng Liu, Zhongyu Wei, Siyuan Wang, Jian Jiao, Nan Duan, Ruofei Zhang, Xuanjing Huang
NAACL 2021

RoBLEURT Submission for WMT2021 Metrics Task
Yu Wan, Dayiheng Liu, Baosong Yang, Haibo Zhang, Boxing Chen, Weihua Luo, Derek F. Wong and Lidia S. Chao
WMT 2021

AnchiBERT: A Pre-Trained Model for Ancient ChineseLanguage Understanding and Generation
Huishuang Tian, Kexin Yang, Dayiheng Liu, Jiancheng Lv
IJCNN 2021

Evolving Transformer Architecture for Neural Machine Translation
Ben Feng, Dayiheng Liu, Yanan Sun
GECCO 2021

Tell Me How to Ask Again: Question Data Augmentation with Controllable Rewriting in Continuous Space
Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Jiancheng Lv, Nan Duan and Ming Zhou
EMNLP 2020

Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation
Dayiheng Liu, Yeyun Gong, Yu Yan, Jie Fu, Bo Shao, Daxin Jiang, Jiancheng Lv, Nan Duan
EMNLP 2020

ProphetNet: Predicting Future N-gram for Sequence-to-Sequence Pre-training
Yu Yan*, Weizhen Qi*, Yeyun Gong*, Dayiheng Liu* (equal contribution), Nan Duan, Jiusheng Chen, Ruofei Zhang, Ming Zhou
Findings of EMNLP 2020

RikiNet: Reading Wikipedia Pages for Natural Question Answering
Dayiheng Liu, Yeyun Gong, Jie Fu, Yu Yan, Jiusheng Chen, Daxin Jiang, Jiancheng Lv and Nan Duan
ACL 2020

Revision in Continuous Space: Unsupervised Text Style Transfer without Adversarial Learning
Dayiheng Liu, Jie Fu, Yidan Zhang, Chris Pal, Jiancheng Lv
AAAI 2020

Deep Poetry: A Chinese Classical Poetry Generation System
Yusen Liu*, Dayiheng Liu* (equal contribution), Jiancheng Lv
AAAI 2020, Demo Paper

Herb-Know: Knowledge Enhanced Prescription Generation for Traditional Chinese Medicine
Chanjuan Li, Dayiheng Liu, Kexin Yang, Xiaoming Huang, Jiancheng Lv
BIBM 2020

Generating Chinese Poetry from Images via Concrete and Abstract Information
Yusen Liu, Dayiheng Liu, Jiancheng Lv, Yongsheng Sang
IJCNN 2020

An Automatic Evaluation Metric for Ancient-Modern Chinese Translation
Kexin Yang, Dayiheng Liu, Qian Qu, Jiancheng Lv, Yongsheng Sang
NCAA 2020

Exploration on the Generation of Chinese Palindrome Poetry
Liao Chen, Zhichen Lai, Dayiheng Liu, Jiancheng Lv and Yongsheng Sang
International Conference on Neural Information Processing. ICONIP 2020

Let’s be Humorous: Knowledge Enhanced Humor Generation
Hang Zhang, Dayiheng Liu, Jiancheng Lv, Cheng Luo
arXiv 2020

TIGS: An Inference Algorithm for Text Inﬁlling with Gradient Search
Dayiheng Liu, Jie Fu, Pengfei Liu, Jiancheng Lv
ACL 2019

BFGAN: Backward and Forward Generative Adversarial Networks for Lexically Constrained Sentence Generation
Dayiheng Liu, Jie Fu, Qian Qu, Jiancheng Lv
TASLP 2019

µ-Forcing: Training Variational Recurrent Autoencoders for Text Generation
Dayiheng Liu, Xue Yang*, Feng He, Yuanyuan Chen, Jiancheng Lv
TALLIP 2019

Ancient-Modern Chinese Translation with a New Large Training Dataset
Dayiheng Liu, Kexin Yang, Qian Qu, Jiancheng Lv
TALLIP 2019

A Multi-Modal Chinese Poetry Generation Model
Dayiheng Liu, Quan Guo, Wubo Li, Jiancheng Lv
IJCNN 2018

Deep learning-based automatic downbeat tracking: a brief review
Bijue Jia, Jiancheng Lv, Dayiheng Liu
Multimedia Systems 2019

Generating Style-specific Chinese Tang Poetry with a Simple Actor-Critic Model
Dayiheng Liu, Jiancheng Lv, Yunxia Li
TETCI 2018

Experience

2021.1 - Present, Researcher, Alibaba DAMO Academy.
2019.6 - 2020.7, Intern, NLC group, Microsoft Research Lab – Asia (MSRA).
- Team leader: Nan Duan
- Mentor: Yeyun Gong
2015.10 - 2020.12, Ph.D., DICALab, Sichuan University (SCU).
- 3+2+3 Program (Master-Doctor combind program)
- Advisor: Prof. Jiancheng Lv
2017.10 - 2020.12, Team Leader, DICALab-NLP, Sichuan University (SCU).
2017.3 - 2017.8, Intern, NExT++, National University of Singapore (NUS).
- Advisor: Prof. Hanwang Zhang
2015.6 - 2015.8, Visitor, School of Computing, National University of Singapore (NUS).
2012.9 - 2015.9, B.E., College of Computer Science, Sichuan University (SCU).
- Ranking: 1/370+
- Advisor: Prof. Jiancheng Lv