MBA-SLAM

Motion Blur Aware Dense Visual SLAM with Radiance Fields Representation


arXiv 2024


Peng Wang1,2* Lingzhe Zhao2* Yin Zhang1,2 Shiyu Zhao2 Peidong Liu2
1Zhejiang University     2Westlake University
* denotes equal contribution.
arXiv Code

Given a sequence of severe motion blurred images and depth, MBA-SLAM accurately estimates the local camera motion trajectory of each blurred image and recovers the high quality 3D scene. It surpasses prior vSLAMs on both sharp datasets and motion blurred datasets.

Abstract

We propose a dense visual SLAM pipeline (i.e. MBA-SLAM) to handle severe motion-blurred inputs. Our approach integrates an efficient motion blur-aware tracker with either neural radiance fields or Gaussian Splatting based mapper. By accurately modeling the physical image formation process of motion-blurred images, our method simultaneously learns 3D scene representation and estimates the cameras' local trajectory during exposure time, enabling proactive compensation for motion blur caused by camera movement.

In our experiments, we demonstrate that MBA-SLAM surpasses previous state-of-the-art methods in both camera localization and map reconstruction, showcasing superior performance across a range of datasets, including synthetic and real datasets featuring sharp images as well as those affected by motion blur, highlighting the versatility and robustness of our approach.

Method

Tracking: Given the current blurry frame, the mapper first renders a virtual sharp image of the lastest blurry keyframe from the 3D scene. Our motion blur-aware tracker directly estimates the camera motion trajectory during the exposure time. Mapping: Our mapper generates virtual sharp images along the camera trajectory, following the standard rendering procedures of Radiance Fields or Gaussian Splatting. The blurry image can then be synthesized by averaging these virtual images, adhering to the physical image formation model of motion-blurred images.

Results

ArchViz Dataset

Rendering

Input CoSLAM ESLAM Point-SLAM SplaTAM Ours-NeRF Ours-GS GT

Reconstruction

CoSLAM ESLAM SplaTAM Ours-NeRF Ours-GS Reference

BibTeX

@article{wang2024mbaslam,
      title     = {MBA-SLAM: Motion Blur Aware Dense Visual SLAM with Radiance Fields Representation},
      author    = {Wang, Peng and Zhao, Lingzhe and Zhang, Yin and Zhao, Shiyu and Liu, Peidong},
      journal   = {arXiv preprint arXiv:2411.08279},
      year      = {2024}
  }