Zili Wang 

INF Technology

Shang Hai, China

Email: ziliwang.do@gmail.com
• CV • Google Scholar • GitHub

Zili Wang is currently an Algorithm Expert at INF Technology, focusing on large language model pretraining. Previously, he worked as an Algorithm Engineer at Xiaohongshu Inc. (March 2022 - September 2023) with Prof. Shusen Wang. He was a Research Intern at Microsoft Asia (October 2020 - February 2021) under Xueting Han, and a Research Assistant at Hong Kong Polytechnic University since February 2020 with Prof. Wenjie Li. He also interned at Meituan-Dianping's AI Lab (June 2019 - November 2019) with Dr. Jingang Wang and Dr. Fuzheng Zhang, and at Peking University (October 2018 - May 2019) with Prof. Xu Sun.

Recent Projects

INF-34B: INF’s Open-Source Large Language Models

INF-34B has 34 billion parameters with a context window length of 32K, and is trained on about 3.5T well-processed tokens from English and Chinese bilingual corpus. Compared with open source models of comparable size, INF-34B not only provides competitive performance in the OpenCompass evaluation, but also has impressive potential in both finance and healthcare domains. Besides, the quantized INF-34B runs on graphics cards of 24GB VRAM with negligible accuracy loss, which facilitates commercial applications, especially low-resource scenarios.

Links: GitHub GitHub | HuggingFace HuggingFace | Tech Report Tech Report

Large Language Model Pretraining Related Publications



pdf 		INF’s Open-Source Large Language Models
Jiaran Hao, Zili Wang, LiuYihan Song, Ansheng You, Zhipeng Zhou, Xiaoyu Tan, Dakuan Lu, Xiaoming Shi, Chao Qu, Haozhe Wang, Yinghui Xu, Wei Chu, Yuan Qi
• Github    • Huggingface   

pdf 		Map-neo: Highly capable and transparent bilingual large language model series
Ge Zhang, Scott Qu, Jiaheng Liu, Chenchen Zhang, Chenghua Lin, Chou Leuang Yu, Danny Pan, Esther Cheng, Jie Liu, Qunshu Lin, Raven Yuan, Tuney Zheng, Wei Pang, Xinrun Du, Yiming Liang, Yinghao Ma, Yizhi Li, Ziyang Ma, Bill Lin, Emmanouil Benetos, Huan Yang, Junting Zhou, Kaijing Ma, Minghao Liu, Morry Niu, Noah Wang, Quehry Que, Ruibo Liu, Sine Liu, Shawn Guo, Soren Gao, Wangchunshu Zhou, Xinyue Zhang, Yizhi Zhou, Yubo Wang, Yuelin Bai, Yuhan Zhang, Yuxiang Zhang, Zenith Wang, Zhenzhu Yang, Zijian Zhao, Jiajun Zhang, Wanli Ouyang, Wenhao Huang, Wenhu Chen
• Github   

pdf 		HyperMoE: Towards Better Mixture of Experts via Transferring Among Experts
Hao Zhao, Zihan Qiu, Huijia Wu, Zili Wang, Zhaofeng He, Jie Fu
ACL 2024
• Github   

pdf 		RefGPT: Reference-> Truthful & Customized Dialogues Generation by GPTs and for GPTs
Dongjie Yang, Ruifeng Yuan, YuanTao Fan, YiFei Yang, Zili Wang, Shushen Wang, Hai Zhao
EMNLP 2023
• Github   

pdf 		Xiezhi: An Ever-Updating Benchmark for Holistic Domain Knowledge Evaluation
Zhouhong Gu, Xiaoxuan Zhu, Haoning Ye, Lin Zhang, Jianchen Wang, Sihang Jiang, Zhuozhi Xiong, Zihan Li, Qianyu He, Rui Xu, Wenhao Huang, Zili Wang, Shusen Wang, Weiguo Zheng, Hongwei Feng, Yanghua Xiao
AAAI 2024
• Github   

pdf 		GW-MoE: Resolving Uncertainty in MoE Router with Global Workspace Theory
Haoze Wu, Zihan Qiu, Zili Wang, Hang Zhao, Jie Fu
• Github   

pdf 		A Closer Look into Mixture-of-Experts in Large Language Models
Ka Man Lo, Zeyu Huang, Zihan Qiu, Zili Wang, Jie Fu
• Github   

pdf 		Evolving Large Language Model Assistant with Long-Term Conditional Memory
Ruifeng Yuan, Shichao Sun, Zili Wang, Ziqiang Cao, Wenjie Li
• Github   

Multimodal model Related Publications



pdf		 Beyond Language Models: Byte Models are Digital World Simulators
Shangda Wu, Xu Tan, Zili Wang, Rui Wang, Xiaobing Li, Maosong Sun
• Github   

pdf		 AIM: Let Any Multi-modal Large Language Models Embrace Efficient In-Context Learning
Jun Gao, Qian Qiao, Ziqiang Cao, Zili Wang, Wenjie Li
• Github   

LLM-based Agent Related Publications


pdf		 Metagpt: Meta programming for multi-agent collaborative framework
Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu
ICLR 2024
• Github   

Text Summarization Related Publications


pdf		 QuerySum: A Multi-Document Query-Focused Summarization Dataset
Yushan Liu, Zili Wang, Ruifeng Yuan
AAAI 2024
• Github   

pdf		 Preserve Context Information through Interpretability for Extract-Generate Long-Input Summarization Framework
Ruifeng Yuan, Zili Wang, Ziqiang Cao, Wenjie Li
AAAI 2023
• Github   

pdf		 Few-shot Query-Focused Summarization with Prefix-Merging
Ruifeng Yuan, Zili Wang, Wenjie Li
EMNLP 2022
• Github   

pdf		 Event Graph based Sentence Fusion
Ruifeng Yuan*, Zili Wang*(Equal Contribution), Wenjie Li
EMNLP 2021
• Github   

pdf		 Fact level Extractive Summarization with Hierarchical Graph Mask on BERT
Ruifeng Yuan, Zili Wang, Wenjie Li
COLING 2020
• Github   

pdf		 Query-aware Tip Generation for Vertical Search
Yang Yang*, Junmei Hao*, Canjia Li*, Zili Wang*(Equal Contribution), Jingang Wang, Fuzheng Zhang, Rao Fu, Peixu Hou, Gong Zhang, Zhongyuan Wang
CIKM 2020
• Github    • Slides   
Look for the full publication list? Please see my CV or visit Google Scholar.

Professional Services


Program Committee Member of ACM CIKM (2023)
Program Committee Member of ACM CIKM (2022)
Program Committee Member of ACM CIKM (2021)