Yilong Zhu

Yilong Zhu

WBI Team Lead at RoboScience

I received my PhD from the Aerial Robotics Group at HKUST, advised by Prof. Shaojie Shen, with research in localization, mapping, cross-view perception, and learning-based navigation.

Currently, I am the WBI Team Lead at RoboScience, working under the guidance of Prof. Lin Shao. My work focuses on robot manipulation, whole-body control, and cross-embodiment learning.

Previously, I worked on VLA spatial generalization as a research intern at China Merchants Lion Rock AI Lab and led mapping and localization algorithms at Unity Drive Innovation.

Email GitHub Google Scholar

News

Paper "The Moving Eye: Enhancing VLA Spatial Generalization via Hybrid Dynamic Data Collection" (co-first author) accepted to IROS 2026. Project page

Successfully defended my PhD thesis: "Robot Navigation from Explicit Geometry to Implicit Models". Grateful to my committee: Prof. Kun Xu (Chair), Prof. Shaojie Shen (Supervisor), Prof. Wei Zhang, Prof. Zili Meng, Prof. Yang Gao, and external examiner Prof. Dimitrios Kanoulas (UCL, UK).

PhD thesis defense presentation PhD defense committee photo

Released my iOS app RTK Helper, a professional high-precision positioning tool on the App Store.

Earlier news

Attended the Meta Research Summit in Seattle and met Prof. Shuran Song, Prof. Danfei Xu, Prof. Xiaolong Wang, and other researchers in the field. Grateful to Jianhao Jiao and Yifu Tao for their support and for the opportunity to exchange ideas around Project Aria and embodied intelligence.

Meta Research Summit in Seattle

Invited to serve as Session Co-Chair for "Autonomous Vehicles 3" at IROS 2025. Looking forward to facilitating discussions in Hangzhou, China.

Paper accepted to IROS 2025 on Visual Localization using Novel Satellite Imagery (first author, Oral). See you in Hangzhou, China.

Paper accepted to IEEE Transactions on Instrumentation and Measurement on Global Optimal Solutions to Scaled Quadratic Pose Estimation Problems.

Paper accepted to IEEE Transactions on Instrumentation and Measurement on Globally Optimal Estimation of Accelerometer-Magnetometer Misalignment.

Paper accepted to IEEE Robotics and Automation Letters on Efficient Camera Exposure Control for Visual Odometry via Deep Reinforcement Learning.

Research Interests

My research connects robot learning, perception, and control to build generalizable embodied systems. At RoboScience, my current work spans three directions:

Egocentric Perception & Physical Intuition

Do embodied AI models possess physical intuition from egocentric experience? We build a probing benchmark for fine-grained hand-object interactions, exposing where current vision-language models succeed and fail at predicting forces, compliance, and tactile outcomes from first-person video. [Poster] [Dataset]

Spatial Generalization for Vision-Language-Action Models

Enhancing VLA spatial generalization via hybrid dynamic data collection. By mixing multi-fixed and moving camera viewpoints, we break shortcut learning (camera-base and object-position coupling) and achieve robust manipulation under unseen camera poses and object configurations. The strategy generalizes across VLA architectures including ACT, Diffusion Policy, Pi0, and Gr00t. [Project Page]

Simultaneous Localization and Mapping (SLAM)

Developing tightly-coupled optimization frameworks using LiDAR, IMU, and UWB to achieve drift-free, real-time state estimation.

Bird's Eye View (BEV) Localization

Exploring BEV-based geometric and semantic alignment between ego-view and satellite-view images to enable cross-view localization.

Robust LiDAR-Inertial Localization (RLIL)

A system integrating motion distortion correction, IMU bias estimation, and Kd-tree-accelerated scan matching. RLIL achieves centimeter-level accuracy in GNSS-denied environments.

RLIL Overview

Dynamic-Aware Localization

Constructing static TSDF maps with dynamic object removal and using scan-to-map deviations to filter dynamic points in real-time.

Dynamic Localization

Generative Models for Cross-view Understanding

Applying diffusion models to generate semantically aligned BEV representations from monocular images.

Generative Models

DiffLoc: Semantic-Guided BEV Generation

Integrates IPM, Navier-Stokes inpainting, and CLIP-based description to synthesize BEV representations for matching against satellite imagery.

DiffLoc

Vision-Language Navigation (VLN) & Long-Horizon Planning

Integrating fast reactive modules with slow semantic planners leveraging satellite imagery and VLMs to enable long-horizon, cross-modal Vision-Language Navigation and early-warning instructions.

Notes & Blog

Study notes on Vision-Language-Action (VLA) model architectures, hosted at vla.yilong-zhu.com.

VLA Model Architecture Notes (8 entries)
[VLA] DreamZero (WAM) — World Action Model for zero-shot task generalization. Combines video frame prediction with robot action sequence generation via a causal diffusion transformer (Causal WAN DiT), perception encoders, video VAE compression, and multi-robot support.
[VLA] Pi0 — Physical Intelligence's flagship VLA model. Uses PaliGemma (SigLIP + Gemma 2B) as the vision-language backbone with a flow-matching action expert for continuous control.
[VLA] Pi0.5 — Updated Pi0 variant with refined architecture for improved sample efficiency and generalization.
[VLA] Pi0-FAST — Discretizes continuous actions into tokens and uses PaliGemma for autoregressive next-token action prediction, trading expressiveness for inference speed.
[VLA] ACT — Transformer encoder-decoder policy with action chunking. A strong imitation learning baseline that predicts short horizons of actions per forward pass.
[VLA] SmolVLA — Lightweight VLA model built on SmolVLM2-500M-Video-Instruct, targeting deployable scale without sacrificing multi-modal grounding.
[VLA] WALL-OSS / WALL-X — Cross-embodied VLA using Qwen2.5-VL as the backbone with a mixture-of-experts action head for handling diverse robot morphologies.
[VLA] X-VLA — Diffusion-based VLA with Florence2 as the vision backbone. Generates action sequences through iterative denoising.