News
Jan 2025
One full paper is accepted by ICLR, about video moment retrieval. One full paper is accepted by ICRA, about cross-modal generation.
Jan 2025
I will serve as an Area Chair for IJCAI and IJCNN.
Dec 2024
We are hosting the 1st Workshop on Navigating the Future: Ensuring Trustworthiness in Multi-Modal Open-World Intelligence (CVPR 2025, Nashville TN). Please visit here for details.
Dec 2024
One full paper is accepted by AAAI, about visual-audio segmentation.
Nov 2024
Two full papers are accepted by ACM TOMM, about referring image segmentation, and cloud-device collaboration.
July 2024
Five full papers are accepted by ACM MM 2024, about image editing, speech event extraction, multi-modal LLM, visual programming, and video tube retrieval.
July 2024
One full paper is accepted by ECCV 2024, about text-guided geolocalization.
May 2024
We are hosting the 2nd International Workshop on Deep Multimodal Generation and Retrieval (ACM MM2024, Melbourne). Please visit here for details.
May 2024
Three full papers are accepted by ICML 2024, about multi-modal pretrained models (NExT-GPT & NExT-Chat), and multi-modal reasoning.
April 2024
One full paper is accepted by SIGIR 2024, about dense retrieval.
March 2024
One full paper is accepted by TIP 2024, about video relation detection.
Feb 2024
One full paper is accepted by CVPR 2024, about text-video generation.
Feb 2024
We are organizing a Special Issue on ACM TOMM about multi-modal retrieval and generation. Please visit here for details.
Jan 2024
Three full papers are accepted by ICLR, about LLM fine-tuning, image retrieval, and model selection.
Dec 2023
One full paper is accepted by AAAI, about scene graph generation. Two full papers are accepted by ICASSP, about scene graph generation and video moment retrieval.
Nov 2023
I will serve as an Area Chair for ACM Multimedia 2024.
Oct 2023
One full paper is accepted by EMNLP, about video reasoning. One full paper is accepted by TPAMI, about video question answering.
Sep 2023
One full paper is accepted by NeurIPS, about visual prompt generator.
July 2023
Four full papers are accepted by ACM MM, about Video Moment Retrieval, Scene Graph Generation, Visual Instance Retrieval, and Multi-modal Recommendation.
July 2023
Two full papers are accepted by ICCV, about Scene Graph Generation and Prompt Learning.
May 2023
Two full papers are accepted by ACL, about Image Captioning and Visual Spatial Description. One paper is accepted by ACL Findings, about video-based fake news detection.
April 2023
We are hosting the 1st Deep Multimodal Learning for Information Retrieval (ACM MM2023, Ottawa). Please visit here for details.
February 2023
Two full papers are accepted by CVPR, about video moment retrieval and spatial-temporal video grounding.
November 2022
Two full papers are accepted by AAAI, about video-audio domain generalization and video-based fake news detection.
October 2022
Two full papers are accepted by EMNLP, about video question answering and pretrained vision-language model.
July 2022
One full paper is accepted by ECCV, about scene graph generation.
May 2022
One full paper is accepted by TIP, about image super-resolution.
April 2022
One full paper is accepted by SIGIR'22, about conversational search.
March 2022
One full paper is accepted by CVPR'22, about video question answering.
Dec 2021
Three full papers are accepted by AAAI'22, about video question answering, grounded situation recognition, and image quality assessment.
August 2021
One full paper is accepted by ACM MM'21, about video relation detection.
July 2021
One full paper is accepted by SIGIR'21, about natural language video localization.
May 2021
We are hosting the 3rd Video Relation Understanding Grand Challenge (ACM MM2021, Chengdu). Please visit here for details.
December 2020
One full paper is accepted by AAAI'21, about natural language video localization.
August 2020
I have successfully defended my thesis and got the PhD degree! My thesis title is "Research on pixel-level semantic understanding based on deep learning".
|
Wei Ji
Tenure-track Associate Professor
School of Intelligence Science and Technology
No. 1520 Taihu Road, Suzhou City, Jiangsu Province, China, 215163
Email: weiji0523 AT gmail.com
Email: weiji AT nju.edu.cn
|
I have joined Nanjing University as a tenure-track associate professor in Fall 2024. I was a senior research fellow in the School of Computing, National University of Singapore, working with Prof. Tat-seng Chua and Prof. Roger Zimmermann. I obtained my Ph.D degree from DCD Lab@Zhejiang University, under the supervision of Prof. Yueting Zhuang and Prof. Xi Li. I have published 60+ papers in top conferences such as ICML, NeurIPS, ICLR, CVPR, ECCV, ACM MM, ACL, EMNLP, SIGIR,
and journals including TPAMI, TIP and TCYB. Moreover, I have served as Area Chair for ACM MM, Guest Editor for ACM TOMM, and the PC member for top-tier conferences/journals, including NeurIPS, ICLR, ICML, SIGIR, CVPR, ICCV, ECCV, AAAI, EMNLP, TPAMI, IJCV, TIP, TMM, etc.
My research interests include but are not limited to:
•Multi-modal Understanding and Generation;
•Multi-modal Retrieval and Recommendation;
•Multi-modal Pretrained Models;
•Multi-modal Security and Privacy.
My research interests include but are not limited to:
•Multi-modal Understanding and Generation;
•Multi-modal Retrieval and Recommendation;
•Multi-modal Pretrained Models;
•Multi-modal Security and Privacy.
I am actively seeking highly motivated Ph.D/Master candidates who share my research interests. Kindly reach out to me at weiji0523@gmail.com with your resume!
Professional Services
|
Publicity co-chair of ACM MM'25 Guest Editor of ACM TOMM Area Chair of ACM MM, IJCAI, IJCNN Program Committee Member of NeurIPS, ICML, ICLR, WWW, SIGIR, CVPR, ICCV, ECCV, AAAI, ACM MM, ACL, EMNLP, ACM Multimedia Asia Invited Reviewer for TPAMI, IJCV, TIP, TMM, TCSVT, ACM TOMM |
Workshops & Grand Challenges
 |
(MM-Open'25) Navigating the Future: Ensuring Trustworthiness in Multi-Modal Open-World Intelligence
Wei Ji, Hong Liu, Zhun Zhong, Zhe Zeng, Elisa Ricci, Andrew Gordon Wilson, Shin’ichi Satoh, Nicu Sebe CVPR 2025 Workshop |
|
(MMGR'24) The 2nd International Workshop on Deep Multimodal Generation and Retrieval
Wei Ji, Hao Fei, Yinwei Wei, Zhedong Zheng, Juncheng Li, Zhiqi Ge, Long Chen, Lizi Liao, Yueting Zhuang, Roger Zimmermann ACM MM 2024 Workshop |
|
(MMIR'23) Deep Multimodal Learning for Information Retrieval
Wei Ji, Yinwei Wei, Zhedong Zheng, Hao Fei, Tat-seng Chua ACM MM 2023 Workshop |
|
(VRU'21) Video Relation Understanding Grand Challenge
Wei Ji, Yicong Li, Xindi Shang, Xiaoyu Du, Tongwei Ren, Tat-Seng Chua ACM MM 2021 Grand Challenge |
Selected Publications
|
NExT-GPT: Any-to-any multimodal llm
Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji , Tat-Seng Chua ICML 2024 (Oral) |
|
NExT-Chat: An lmm for chat, detection and segmentation
Ao Zhang, Yuan Yao, Wei Ji , Zhiyuan Liu, Tat-Seng Chua ICML 2024 |
|
Backpropogation-Free Multi-modal On-Device Model Adaptation via Cloud-Device Collaboration
Wei Ji , Li Li, Zheqi Lv, Wenqiao Zhang, Mengze Li, Zhen Wan, Wenqiang Lei, Roger Zimmermann ACM TOMM 2024 |
|
Towards Complex-query Referring Image Segmentation: A Novel Benchmark
Wei Ji , Li Li, Hao Fei, Xiangyan Liu, Xun Yang, Juncheng Li, Roger Zimmermann ACM TOMM 2024 |
|
Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs
Hao Fei, Shengqiong Wu, Wei Ji , Hanwang Zhang, Tat-Seng Chua CVPR 2024 |
|
Video-of-thought: Step-by-step video reasoning from perception to cognition
Hao Fei, Shengqiong Wu, Wei Ji , Hanwang Zhang, Meishan Zhang, Mong-Li Lee, Wynne Hsu ICML 2024 (Oral) |
|
Hierarchical Debiasing and Noisy Correction for Cross-domain Video Tube Retrieval
Jingqiao Xiu, Mengze Li, Wei Ji , Jingyuan Chen, Hanbin Zhao, Shin'ichi Satoh, Roger Zimmermann ACM MM 2024 |
|
Transformer-empowered invariant grounding for video question answering
Yicong Li, Xiang Wang, Junbin Xiao, Wei Ji , Tat-Seng Chua TPAMI 2023 |
|
Transfer visual prompt generator across llms
Ao Zhang, Hao Fei, Yuan Yao, Wei Ji , Li Li, Zhiyuan Liu, Tat-Seng Chua NeurIPS 2023 |
|
Learning Style-Invariant Robust Representation for Generalizable Visual Instance Retrieval
Tianyu Chang, Xun Yang, Xin Luo, Wei Ji , Meng Wang ACM MM 2023 |
|
Partial Annotation-based Video Moment Retrieval via Iterative Learning
Wei Ji , Renjie Liang, Lizi Liao, Hao Fei, Fuli Feng ACM MM 2023 |
|
Online Distillation-enhanced Multi-modal Transformer for Sequential Recommendation
Wei Ji , Xiangyan Liu, An Zhang, Yinwei Wei, Yongxin Ni, Xiang Wang ACM MM 2023 |
|
Visually-Prompted Language Model for Fine-Grained Scene Graph Generation in an Open World.
Qifan Yu, Juncheng Li, Yu Wu, Siliang Tang, Wei Ji , Yueting Zhuang ICCV 2023 |
|
Two Heads Are Better Than One: Improving Fake News Video Detection by Correlating.
Peng Qi, Yuyang Zhao, Yufeng Shen, Wei Ji , Juan Cao, Tat-Seng Chua. ACL 2023 Findings |
|
Generating Visual Spatial Description via Holistic 3D Scene Understanding.
Yu Zhao, Hao Fei,Wei Ji , Jianguo Wei, Meishan Zhang, Min Zhang, Tat-Seng Chua. ACL 2023 |
|
Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment.
Shengqiong Wu, Hao Fei, Wei Ji, Tat-Seng Chua. ACL 2023 |
|
WINNER: Weakly-supervised hIerarchical decompositioN and aligNment for spatio-tEmporal video gRounding.
Mengze Li, Han Wang, Wenqiao Zhang, Jiaxu Miao,Wei Ji, Zhou Zhao, Shengyu Zhang, Fei Wu. CVPR 2023 |
|
Are Binary Annotations Sufficient? Video Moment Retrieval via Hierarchical Uncertainty-based Active Learning.
Wei Ji, Renjie Liang, Zhedong Zheng, Wenqiao Zhang, Shengyu Zhang, Juncheng, Li and Mengze Li, Tat-Seng Chua. CVPR 2023 |
|
FakeSV: A Multimodal Benchmark with Rich Social Context for Fake News Detection on Short Video Platforms
Peng Qi, Yuyan Bu, Juan Cao, Wei Ji, Ruihao Shui, Junbin Xiao, Danding Wang, Tat-Seng Chua AAAI 2023 |
|
PEVL: Position-enhanced Pre-training and Prompt Tuning for Vision-language Models
Yuan Yao, Qianyu Chen, Ao Zhang, Wei Ji, Zhiyuan Liu, Tat-Seng Chua, Maosong Sun EMNLP 2022 |
|
Video Question Answering: Datasets, Algorithms and Challenges
Yaoyao Zhong, Junbin Xiao, Wei Ji, Yicong Li, Weihong Deng, Tat-Seng Chua EMNLP 2022 |
|
Structured and Natural Responses Co-generation for Conversational Search
Chenchen Ye, Lizi Liao, Fuli Feng, Wei Ji, Tat-Seng Chua SIGIR 2022 |
|
Fine-Grained Scene Graph Generation with Data Transfer
Ao Zhang, Yuan Yao, Qianyu Chen, Wei Ji, Zhiyuan Liu, Maosong Sun, Tat-Seng Chua ECCV 2022 (oral) |
|
Conditional Meta-Network for Blind Super-Resolution with Multiple Degradations
Guanghao Yin, Wei Wang, Zehuan Yuan, Wei Ji, Dongdong Yu, Shouqian Sun, Tat-Seng Chua, Changhu Wang TIP 2022 |
|
Invariant grounding for video question answering
Yicong Li, Xiang Wang, Junbin Xiao, Wei Ji, Tat-Seng Chua CVPR 2022 (Oral, Best Paper Finalist) |
|
Content-Variant Reference Image Quality Assessment via Knowledge Distillation
Guanghao Yin, Wei Wang, Zehuan Yuan, Chuchu Han, Wei Ji, Shouqian Sun, Changhu Wang AAAI 2022 |
|
Rethinking the Two-Stage Framework for Grounded Situation Recognition
Meng Wei, Long Chen, Wei Ji, Xiaoyu Yue, Tat-Seng Chua AAAI 2022 |
|
Video as Conditional Graph Hierarchy for Multi-Granular Question Answering
Junbin Xiao, Angela Yao, Zhiyuan Liu, Yicong Li, Wei Ji, Tat-Seng Chua AAAI 2022 |
|
Deep Learning for Weakly-Supervised Object Detection and Localization: A Survey
Feifei Shao, Long Chen, Jian Shao, Wei Ji, Shaoning Xiao, Lu Ye, Yueting Zhuang, Jun Xiao Neurocomputing |
|
Deconfounded Video Moment Retrieval with Causal Intervention
Xun Yang, Fuli Feng, Wei Ji, Meng Wang, Tat-Seng Chua SIGIR 2021 |
|
Video Visual Relation Detection via Iterative Inference
Xindi Shang, Yicong Li, Junbin Xiao, Wei Ji, Tat-Seng Chua ACM MM 2021 |
|
Boundary Proposal Network for Two-Stage Natural Language Video Localization
Shaoning Xiao, Long Chen, Songyang Zhang, Wei Ji, Jian Shao, Lu Ye, Jun Xiao AAAI 2021 |
Honors
| Outstanding Graduate, Zhejiang University 2020 |
| Distinguished Doctoral Scholarship, Zhejiang University 2018-2019 |