SpatialTrackerV2: 3D Point Tracking Made Easy

CAD&CG, Zhejiang University; University of Oxford; Ant Research; Pixelwise AI; NUS

Yuxi Xiao, Jianyuan Wang, Nan Xue, Nikita Karaev, Iurii Makarov, Bingyi Kang, Xin Zhu, Hujun Bao, Yujun Shen, Xiaowei Zhou

Project Page | BibTeX | Goolge Drive

🚀 Latest Updates & News

🎉 What's New & Exciting! 🎉

---

🔥 Stay tuned for the most exciting developments! We're constantly pushing the boundaries of 3D tracking technology! 🔥

📅 Recent Highlights

🏆 25 June 2025: 🥇 SpatialTrackerV2 has been ACCEPTED by ICCV 2025! 🥇
📄 See you in Hawaii!

🎯 23 June 2025: 🤖 Try our amazing Huggingface Space Demo: https://huggingface.co/spaces/Yuxihenry/SpatialTrackerV2
✨ Experience the magic of 3D point tracking in your browser!

---

💡 Coming Soon: More incredible features on the way! Keep watching this space! 👀

TODO List

1. Release Offline Version

   • SpaTrack2 + Moge ➔ supports unposed RGB as input.
   • SpaTrack2+ MegaSAM ➔ supports Posed RGBD as input.
   • SpaTrack2 + VGGT ➔ make VGGT works in Dynamic Scenes.

2. Release Online Version

   • Sliding windows version.

3. More Releases

   • Some Ceres Python Bindings designed for SpatialTracker and Dynamic Reconstruction.
   • More supports for other Depth Model, i.e., DepthAnything, StereoFoundation, UniDepth, Metric3D.

Set up the environment

To set up the environment for running the SpaTrack model, follow these steps:

1. Clone the Repository:

   git clone git@github.com:henry123-boy/SpaTrackerV2.git
   cd SpaTrackerV2
   

2. Create a Virtual Environment: It's recommended to use a virtual environment to manage dependencies.

   conda create -n SpaTrack2 python=3.11
   conda activate SpaTrack2
   

3. Install Dependencies:

   Install the torch dependencies pip (tested with torch2.4).

   python -m pip install torch==2.4.1 torchvision==0.19.1 torchaudio==2.4.1 --index-url https://download.pytorch.org/whl/cu124
   

   Install the required Python packages using pip.

   python -m pip install -r requirements.txt
   

4. Install SpaTrack2 Visualizer:

   cd viser
   python -m pip install -e .
   

By following these steps, you should have a working environment ready to run the SpaTrack model.

Download the Checkpoints

mkdir -p checkpoints

Step1: Download the checkpoint of Moge from here, and put the model.pt into ./checkpoints/

Step2: Download the checkpoint of SpaTrack2 from GoolgeDrive, and place it into ./checkpoints/

Quick Start

We gave two examples to illustrate the usage of SpaTrack2. Firstly, please download ckpts and examples via:

sh scripts/download.sh

Type1: Posed RGBD video (Example0)

We provide an example who has Posed RGBD input with MegaSAM.

python inference.py --data_type="RGBD" --data_dir="assets/example0" --video_name="snowboard" --fps=1

Type2: unposed RGB video (Example1)

python inference.py --data_type="RGB" --data_dir="assets/example1" --video_name="xhs" --fps=6

Visualize your results

We provide two types of visualization, i.e. viser and TAPIP3D. The guidance will be displayed in the terminal after running inference.py.

🌟 Recommended: Gradio Demo with SAM 🌟

Please follow the instructions in the app_3rd README to configure the dependencies. Then,

python -m pip install gradio==5.31.0 pako

Our gradio demo enable the user to track the points on the target object easily, just try:

python app.py

Here is an example with our GUI: