Shengqiong Wu

I am currently a second-year Ph.D. student at NExT++ Research Center, advised by Prof. Tat-Seng Chua in School of Computing at National University of Singapore, after I obtained my M.S. and Bachelor degrees from Wuhan University.

My research interest mainly lies in Scene Graph-based Vision-Language Understanding, with the contexts of Multimodal Large Language Models and Diffusion Model, etc. Previously, I focused on syntax-aided NLP, i.e., employing external syntactic and linguistic representation to aid NLP tasks.

Email  /  Github  /  Google Scholar

  • [NEW!-2024/05] Congration πŸ₯³πŸ₯³πŸ₯³, two full papers are accpted in ICML-24, NExT-GPT and Video-of-Thought .
  • [NEW!-2024/04] We release Vitron (Demo, Paper , Code), a universal pixel-level vision LLM designed for understanding, generating, segmenting, editing of both image and video. 🌟🌟
  • [NEW!-2024/02] One full paper is accepted by CVPR-24 about Text-to-Video Generation. Congrats to all my co-authors.
  • [NEW!-2024/01] I honor Baidu Scholarship (10 people worldwide). πŸ₯³πŸ₯³πŸ₯³
  • [NEW!-2023/12] I have successfully passed my Rearcsh-based QE, I am now a Ph.D. Candidate. πŸ₯³πŸ₯³πŸ₯³
  • [NEW!-2023/11] I'll join Kunlun 2050 Research as a research intern, advised by Prof. Yan. πŸ˜†
  • [NEW!-2023/10] We build NExT-GPT, a general-purpose any-to-any MLLM. 🌟
  • [NEW!-2023/09] One full paper is accepted in NeurIPS-23, about Intricate Text-to-image Generation based on Scene Graph. πŸ₯³
  • [NEW!-2023/07] One full paper is accepted in ACM MM-23, about High-faithfull Text-to-image Generation enhanced with layout.
  • [NEW!-2023/05] Two full papers is accepted in ACL-23, about Multimodal Relation Extraction and Image Captioning.
  • [NEW!-2022/07] I'm heading to SoC NUS to pursue my PhDβ€”a new journey and fresh challenges await! πŸ™ƒ

  • Publications
    • Shengqiong Wu, Hao Fei, Leigang Qu, Wei Ji, Tat-Seng Chua.  NExT-GPT: Any-to-Any Multimodal Large Language Model. In Proceedings of ICML. 2023.  [Code | 2.9k 🌟][pdf]
    • Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Meishan Zhang, Mong-Li Lee, Wynne Hsu.  Video-of-Thought: Step-by-Step Video Reasoning from Perception to Cognition. In Proceedings of ICML. 2023.
    • Hao Fei, Shengqiong Wu, Wei Ji, Hanwang Zhang, Tat-Seng Chua.  Dysen-VDM: Empowering Dynamics-aware Text-to-Video Diffusion with LLMs. In Proceedings of CVPR. 2024.  [Code][pdf]
    • Hao Fei, Shengqiong Wu, Hanwang Zhang, Tat-Seng Chua, Shuicheng Yan.  VITRON: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing. arXiv. 2024.  [Code][pdf]
    • Shengqiong Wu, Hao Fei, Hanwang Zhang, Tat-Seng Chua.  Imagine That! Abstract-to-Intricate Text-to-Image Synthesis with Scene Graph Hallucination Diffusion. In Proceedings of NeurIPS. 2023.  (long, poster)  [Code][pdf]
    • Leigang Qu*, Shengqiong Wu*, Hao Fei, Liqiang Nie, Tat-Seng Chua.  LayoutLLM-T2I: Eliciting Layout Guidance from LLM for Text-to-Image Generation. In Proceedings of ACM MM. 2023.  (*: equal contribution, long)  [Code][pdf]
    • Bobo Li, Hao Fei, Yuhan Wu, Jinsong Zhang, Shengqiong Wu, Jingye Li, Yijiang Liu, Lizi Liao, Tat-Seng Chua, Fei Li, Donghong Ji.  DiaASQ: A benchmark of conversational aspect-based sentiment quadruple analysis.In Proceedings of ACL. 2023.  (long, poster)  [Code][pdf]
    • Shengqiong Wu, Hao Fei, Yixin Cao, Lidong Bing, Tat-Seng Chua.  Information Screening whilst Exploiting! Multimodal Relation Extraction with Feature Denoising and Multimodal Topic Modeling. In Proceedings of ACL. 2023.  (long, poster, paper award nomination, 1.6%)  [Code][pdf]
    • Shengqiong Wu, Hao Fei, Wei Ji, Tat-Seng Chua.  Cross2StrA: Unpaired Cross-lingual Image Captioning with Cross-lingual Cross-modal Structure-pivoted Alignment. In Proceedings of ACL. 2023.  (long, oral)  [pdf]
    • Hao Fei, Shengqiong Wu, Jingye Li, Bobo Li, Fei Li, Libo Qin, Meishan Zhang, Min Zhang, Tat-Seng Chua.  LasUIE: Unifying information extraction with latent adaptive structure-aware generative language model. In Proceedings of NeurIPS. 2022.  (long, poster) [Code][pdf]
    • Shunjie Chen, Xiaochuan Shi, Jingye Li, Shengqiong Wu, Hao Fei, Fei Li, Donghong Ji.  Joint Alignment of Multi-Task Feature and Label Spaces for Emotion Cause Pair Extraction. In Proceedings of COLING. 2022.  (long, poster) [Code][pdf]
    • Hu Cao, Jingye Li, Fangfang Su, Fei Li, Hao Fei, Shengqiong Wu, Bobo Li, Liang Zhao and Donghong Ji.  OneEE: A One-Stage Framework for Fast Overlapping and Nested Event Extraction. In Proceedings of COLING. 2022.  (long, oral) [Code][pdf]
    • Shengqiong Wu, Hao Fei, Fei Li, Meishan Zhang, Yijiang Liu, Chong Teng, Donghong Ji.  Mastering the Explicit Opinion-Role Interaction: Syntax-Aided Neural Transition System for Unified Opinion Role Labeling. In Proceedings of AAAI. 2022.  (long, online) [Code][pdf]
    • Jingye Li, Hao Fei, Jiang Liu, Shengqiong Wu, Meishan Zhang, Chong Teng, Donghong Ji, Fei Li.  Unified named entity recognition as word-word relation classification. In Proceedings of AAAI. 2022.  (long, online) [Code][pdf]
    • Shengqiong Wu, Hao Fei, Yafeng Ren, Donghong Ji, Jingye Li.  Learn from Syntax: Improving Pair-wise Aspect and Opinion Terms Extraction with Rich Syntactic Knowledge. In Proceedings of IJCAI. 2021.  (long, online) [Code][pdf]
    • Shengqiong Wu, Hao Fei, Yafeng Ren, Bobo Li, Fei Li and Donghong Ji.   High-order Pair-wise Aspectand Opinion Terms Extraction with Edge-enhanced Syntactic Graph Convolution. IEEE/ACM TASLP. 2021. [Code][pdf]
    • Shengqiong Wu, Bobo Li, Dongdong Xie, Chong Teng and Donghong Ji.  Neural Transition Model for Aspect-based Sentiment Triplet Extraction with Triplet Memory. Neurocomputing. 2021.   [pdf]
    • ShengqiongWu, Hao Fei and Donghong Ji.  Aggressive Language Detection with Joint Text Normalization via Adversarial Multi-task Learning. In Proceedings of NLPCC. 2020.  (long, oral) [Code][pdf]

    Academic Services

    Conference Reviewer
    WSDM-23, NeurIPS-23, ICLR-24, ICML-24, ACM MM-23/24, IJCAI-23/24, AAAI-24, ACL-23/24
    Journal Reviewer
    TOMM, IEEE/ACM TALLIP, IPM, KBS, Neurocomputing

    Invited Talks
    1. [2024/01]       The Path to AGI: Achieving Modality Unification with NExT-GPT. Qingyuan Talk.
    2. [2023/12]       NExT-GPT: Any-to-Any Multimodal LLM. AI New Youth.
    3. [2022/12]       Deep Learning based Natural Language Processing: A Survey and Outlook. Jianghan University, China.
    4. [2021/10]       Comparison of Aspect-based Sentiment Analysis based on span model and transition model. Wuhan University, China.
    5. [2021/05]       Learn from Syntax: Improving Pair-wise Aspect and Opinion Terms Extraction with Rich Syntactic Knowledge. CIPS, Youth Working Committee.

    Work Experience

    Nov. 2023 - Now                     Kunlun Skywork AI, Singapore
                                                              2050 Research Intern
                                                              Advisor: Shuicheng Yan, Director

    Jul. 2019 - Aug. 2019          China Merchants Bank, Wuhan, China
                                                              Information Technology Department Intern
                                                              Advisor: Hua Pan, General Manager

    Jul. 2018 - Nov. 2018          YITU, ShangHai, China
                                                              Solution Engineer Intern
                                                              Advisor: Ze Deng

    Honors & Awards

  • [2024]    Scholarship (Top 10 Global Recipients), Baidu.
  • [2023]    Research Achievement Award of SoC, NUS.
  • [2022]    President's Graduate Fellowship of NUS.
  • [2022]    Award of Excellent Master Thesis of Hubei Cyberspace Security Society.
  • [2022]    Excellent Graduated Graduate Student.
  • [2021]    Huawei Scholarship for graduate students.
  • [2021]    National Scholarship for graduate students.
  • [2021]    1st prize of Excellent Scholarship of Academy for M.S. Fellowship.
  • [2021]    1st prize of Extraordinary scholarship of academy.
  • [2020]    Outstanding graduate student award.
  • [2020]    2st prize of academic scholarship of Wuhan University for graduate.
  • [2017]    Excellent student cadre.
  • [2016-2019]    2st prize of academic scholarship of Wuhan University for undergraduate.
  • [2017]    National Encouragement Scholarship.

  • Education

    Aug. 2022 - Now                        School of Computing,   National University of Singapore,   Singapore
                                                                  Ph.D. in Computer Science

    Sep. 2019 - Jun. 2022             School of Cyber Science and Engineering,   Wuhan University,   China
                                                                  M.S. in Computer Science

    Sep. 2015 - Jun. 2019             School of Computer Science,   Wuhan University,   China
                                                                  B.S. in Computer Science and Technology

    Skills and MISC

  • Coding
  • Deep Learning
  • Hobby