About
I am a third year undergraduate student at the School of Instrument of Science and Engineering, Southeast University, advised by Prof. Yan Wang from AIR Tsinghua University. I am actively seeking MPhil/PhD positions in autonomous driving, multimodal large language models (MLLMs), embodied ai, and world models.
My research focuses on multimodal large language models (MLLMs) for autonomous driving, where I view vision–language–action (VLA) models as a natural extension of MLLMs from perception and reasoning to embodied action. My work centers on enhancing the spatial understanding capabilities of MLLMs in real-world environments. I have explored efficient 3D scene understanding (e.g., BEV-based detection and occupancy modeling) as a foundation for grounding multimodal representations in geometry and dynamics. Building on this, I investigate how to equip MLLMs with structured spatial representations, enabling them to reason about 3D scenes, predict interactions, and support decision-making. Ultimately, I aim to push MLLMs beyond passive understanding toward spatially grounded, action-oriented intelligence for closed-loop autonomous systems.
Looking forward, my research focus on tightly integrating world models with MLLMs and vision–language–action (VLA) models to enhance spatial understanding and interaction in embodied environments. In particular, I investigate how structured world representations can improve a model’s ability to reason about 3D space, dynamics, and agent–environment interactions. At the same time, I explore the fundamental role of semantic information across language and spatial modalities, aiming to understand how such signals can be effectively grounded and leveraged within VLA systems to support more robust perception, reasoning, and decision-making.
In parallel, I explore safety and knowledge management in MLLMs, including efficient fine-tuning, controllability, and reversible knowledge editing. I am particularly interested in how these techniques can improve reliability and generalization in safety-critical embodied systems.
My long-term goal is to advance generalizable embodied AI systems that deeply understand the physical world—systems that can perceive, reason, predict, and act in complex, open-world environments with strong robustness and safety guarantees.
News
Our work Knowledge Externalization: Reversible Unlearning and Modular Retrieval in Multimodal Large Language Models has been accepted by a ICLR2026 🎉
Publications
A collection of my research work.
Knowledge Externalization: Reversible Unlearning and Modular Retrieval in Multimodal Large Language Models
Jiaqi Li, Zihan You, Ruoyan Shen, Shenyu Zhang, Songlin Zhai, Yongrui Chen, Chuanyi Zhang, Jiahui Geng, Fakhri Karray, Sheng Bi, others
The Fourteenth International Conference on Learning Representations 2026
Co-first Author. This paper proposes Knowledge Externalization, a framework that moves sensitive knowledge from MLLM parameters into modular external memory tokens, enabling reversible, auditable, and composable unlearning and knowledge management without permanently altering the base model.
SAMoE-VLA: A Scene Adaptive Mixture-of-Experts Vision-Language-Action Model for Autonomous Driving
You Zihan, Liu Hongwei, Dang Chenxu, Wang Zhe, Ang Sining, Wang Aoqi, Wang Yan†
ECCV Under Reivew 2026
First Author; SAMoE-VLA is a scene-adaptive Vision-Language-Action model that replaces token-level MoE with scene-adaptive MoE routing with BEV to achieve more stable, safety-critical decision-making and state-of-the-art autonomous driving performance.
Mambabev: An efficient 3d detection model with mamba2
You Zihan, Wang Ni, Wang Hao, Zhao Qichao, Wang Jinxiang
ICPR under review 2026
First Author. This paper proposes MambaBEV, a BEV-based 3D detection framework that leverages Mamba2 state space models for global temporal fusion, significantly improving large-object perception and overall autonomous driving performance.
SparseOccVLA: Bridging Occupancy and Vision-Language Models via Sparse Queries for Unified 4D Scene Understanding and Planning
Chenxu Dang, Jie Wang, Guang Li, Zhiwen Hou, Zihan You, Hangjun Ye, Jie Ma, Long Chen, Yan Wang
ECCV Under Review 2026
This paper proposes SparseOccVLA, a unified VLA framework that bridges VLM reasoning and semantic occupancy via sparse occupancy queries, enabling efficient joint scene understanding, forecasting, and planning with state-of-the-art performance.
Hybrid guided variational autoencoder for visual place recognition
Wang Ni, You Zihan, Neftci Emre, Schoepe Thorben
2026
This paper proposes a compact event-based VPR framework using a spiking neural network–driven guided VAE, achieving robust, energy-efficient, and generalizable indoor localization under varying conditions.
Dual-Priv Pruning: Efficient Differential Private Fine-Tuning in Multimodal Large Language Models
Qianshan Wei, Jiaqi Li, Zihan You, Yi Zhan, Kecen Li, Jialin Wu, Xinfeng Li Hengjun Liu, Yi Yu, Bin Cao, Yiwen Xu, others
International Conference on Machine Learning Under Review 2025
Co-first Author; This paper proposes Dual-Priv Pruning, a differential privacy fine-tuning framework for MLLMs that combines visual token pruning and gradient-update pruning to reduce noise-induced degradation and computational overhead while preserving model utility.
CV
Education
SouthEast University, BSc in Intelligent Perception of Engineering, Third Grade
- Supervisor: Prof. Advisor Yan Wang from AIR Tsinghua University
Experience
Intern, China Automotive Innovation Co.(CAIC), Autonomous Driving Development Department, China 2024
- Led development of a BEV-based 3D object detection model (MambaBEV) and proposed the TemporalMamba module, achieving 51.7% NDS / 42.7% mAP on nuScenes while improving global context modeling and large-object detection.
- Applied deep learning, autonomous driving, and software engineering skills (PyTorch, Python) to reproduce and optimize end-to-end driving algorithms (e.g., UniAD, VAD), upgrade 3D lane detection (BEVLaneDet + BEVDet), and build data processing/visualization tools to improve dataset quality and system performance.
Research Intern , Tsinghua University, Institute for AI industry Research, China Jun. 2025 – present
- Focused on scene-adaptive Vision–Language–Action (VLA) modeling for autonomous driving, investigating how Large Language Models can be effectively integrated with world models and planning under dynamic traffic conditions.
- Proposed SAMoE-VLA, a soft-weighted, scene-adaptive Mixture-of-Experts VLA architecture, addressing the limitations of static token-level MoE routing in LLMs when applied to driving scenarios with highly varying scene complexity.
- Designed a Conditional Cross-Modal Causal Attention mechanism to unify world state modeling, language reasoning, and action generation within a temporally consistent causal framework.
- Introduced BEV-conditioned expert routing for planning, enabling adaptive expert weighting and merging based on traffic scene context and interaction complexity.
- Achieved state-of-the-art results on the nuScenes planning benchmark and LangAuto closed loop benchmark.
- Extended the core scene-adaptive VLA + MoE paradigm to sparse occupancy–driven representation learning and vehicle–infrastructure cooperative planning, demonstrating the generality and scalability of the proposed approach across multiple autonomous driving settings.
Skills
- Programming: Python, C++, C, MATLAB, LaTeX
- Framework and tools: mmdet, mmdet3d, deepspeed, torch, Docker, Git, OBS
- Languages: English (Native), Chinese (Native)
Awards & Honors
- National First Prize in The 27th China Robot and Artificial Intelligence Competition
- National Second Prize in RoboMaster National Super Competition
- Outstanding Paper Award at the 14th Southeast University Undergraduate Academic Conference (2025)
- Excellent creative works at the Society of Automotive Engineers Annual Conference (SAECCE2024)
- Provincial Second Prize in the Jiangsu Students Mechanical Innovation Design Competition