Yilong Zhu

Yilong Zhu

WBI Team Lead at RoboScience

I earned my PhD at HKUST's Aerial Robotics Group with Prof. Shaojie Shen, studying localization, mapping, cross-view perception, and robot navigation.

Currently at RoboScience, I work on robot manipulation, whole-body control, and cross-embodiment learning with guidance from Prof. Lin Shao.

Previously, I interned at China Merchants Lion Rock AI Lab on VLA generalization and led mapping and localization at Unity Drive Innovation.

News

Do Embodied AI Models Have Physical Intuition? An Egocentric Probing Benchmark

Accepted to RSS 2026 Workshop First author Poster

The Moving Eye: Enhancing VLA Spatial Generalization via Hybrid Dynamic Data Collection

Accepted to IROS 2026 Co-first author Project page

PhD defense: Robot Navigation from Explicit Geometry to Implicit Models

Grateful to my committee: Prof. Kun Xu (Chair), Prof. Shaojie Shen (Supervisor), Prof. Wei Zhang, Prof. Zili Meng, Prof. Yang Gao, and external examiner Prof. Dimitrios Kanoulas (UCL, UK).

Open full-size PhD thesis defense presentation Open full-size PhD defense committee photo

Released RTK Helper for iOS

High-precision positioning App Store

Meta Research Summit in Seattle

Attended the Meta Research Summit in Seattle and met Prof. Shuran Song, Prof. Danfei Xu, Prof. Xiaolong Wang, and other researchers in the field. Grateful to Jianhao Jiao and Yifu Tao for their support and for the opportunity to exchange ideas around Project Aria and embodied intelligence.

Open full-size Meta Research Summit group photo

Invited as Session Co-Chair for Autonomous Vehicles 3

IROS 2025 Hangzhou, China

From Satellite to Street: Semantic and Depth Information for Enhanced Geo-Localization

Accepted to IROS 2025 First author · Oral Publication

Earlier news (before June 2025)

sQPEP: Global Optimal Solutions to Scaled Quadratic Pose Estimation Problems

Accepted by IEEE Transactions on Instrumentation and Measurement (TIM)

AM-Align: Globally Optimal Estimation of Accelerometer-Magnetometer Misalignment

Accepted by IEEE Transactions on Instrumentation and Measurement (TIM)

Efficient Camera Exposure Control for Visual Odometry via Deep Reinforcement Learning

Accepted by IEEE Robotics and Automation Letters (RA-L)

Research Interests

More research

FEEL Experiment Clips

First-person hand-object interactions with synchronized fingertip force measurements.

Simultaneous Localization and Mapping (SLAM)

Developing tightly-coupled optimization frameworks using LiDAR, IMU, and UWB to achieve drift-free, real-time state estimation.

Bird's Eye View (BEV) Localization

Exploring BEV-based geometric and semantic alignment between ego-view and satellite-view images to enable cross-view localization.

Robust LiDAR-Inertial Localization (RLIL)

A system integrating motion distortion correction, IMU bias estimation, and Kd-tree-accelerated scan matching. RLIL achieves centimeter-level accuracy in GNSS-denied environments.

RLIL Overview

Dynamic-Aware Localization

Constructing static TSDF maps with dynamic object removal and using scan-to-map deviations to filter dynamic points in real-time.

Dynamic Localization

Generative Models for Cross-view Understanding

Applying diffusion models to generate semantically aligned BEV representations from monocular images.

Generative Models

DiffLoc: Semantic-Guided BEV Generation

Integrates IPM, Navier-Stokes inpainting, and CLIP-based description to synthesize BEV representations for matching against satellite imagery.

DiffLoc

Vision-Language Navigation (VLN) & Long-Horizon Planning

Integrating fast reactive modules with slow semantic planners leveraging satellite imagery and VLMs to enable long-horizon, cross-modal Vision-Language Navigation and early-warning instructions.

Selected Work

IROS 2026 · Co-first author

The Moving Eye

Mixing fixed and moving views raised task success from 43% to 83% in a moving-camera test, compared with fixed-view training.

RSS 2026 Workshop · First author

FEEL Benchmark

Tests whether embodied AI can infer forces and material behavior from first-person video.

Satellite training and street-to-satellite localization pipelines using semantic features and geometry

IROS 2025 · First author · Oral

From Satellite to Street

Uses semantic and depth cues to match street-level observations to satellite imagery.

Notes & Blog

Study notes on Vision-Language-Action (VLA) model architectures, hosted at vla.yilong-zhu.com.

VLA Model Architecture Notes (8 entries)
[VLA] DreamZero (WAM) — World Action Model for zero-shot task generalization. Combines video frame prediction with robot action sequence generation via a causal diffusion transformer (Causal WAN DiT), perception encoders, video VAE compression, and multi-robot support.
[VLA] Pi0 — Physical Intelligence's flagship VLA model. Uses PaliGemma (SigLIP + Gemma 2B) as the vision-language backbone with a flow-matching action expert for continuous control.
[VLA] Pi0.5 — Updated Pi0 variant with refined architecture for improved sample efficiency and generalization.
[VLA] Pi0-FAST — Discretizes continuous actions into tokens and uses PaliGemma for autoregressive next-token action prediction, trading expressiveness for inference speed.
[VLA] ACT — Transformer encoder-decoder policy with action chunking. A strong imitation learning baseline that predicts short horizons of actions per forward pass.
[VLA] SmolVLA — Lightweight VLA model built on SmolVLM2-500M-Video-Instruct, targeting deployable scale without sacrificing multi-modal grounding.
[VLA] WALL-OSS / WALL-X — Cross-embodied VLA using Qwen2.5-VL as the backbone with a mixture-of-experts action head for handling diverse robot morphologies.
[VLA] X-VLA — Diffusion-based VLA with Florence2 as the vision backbone. Generates action sequences through iterative denoising.