Panoptic Mapping

This package contains panoptic_mapping, a general framework for semantic volumetric mapping. We provide, among other, a submap-based approach that leverages panoptic scene understanding towards adaptive spatio-temporally consistent volumetric mapping, as well as regular, monolithic semantic mapping.

Multi-resolution 3D Reconstruction, active and inactive panoptic submaps for temporal consistency, online change detection, and more.

Table of Contents

Credits

• Paper
• Video

Setup

• Installation
• Datasets

Examples

• Running the Panoptic Mapper
• Monolithic Semantic Mapping
• Running the RIO Dataset

Other

• Contributing

Paper

If you find this package useful for your research, please consider citing our paper:

• Lukas Schmid, Jeffrey Delmerico, Johannes Schönberger, Juan Nieto, Marc Pollefeys, Roland Siegwart, and Cesar Cadena. "Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency" arXiv preprint arXiv:2109.10165 (2021). [ArXiv]

  @ARTICLE{schmid2021panoptic,
    title={Panoptic Multi-TSDFs: a Flexible Representation for Online Multi-resolution Volumetric Mapping and Long-term Dynamic Scene Consistency},
    author={Schmid, Lukas and Delmerico, Jeffrey and Sch{\"o}nberger, Johannes and Nieto, Juan and Pollefeys, Marc and Siegwart, Roland and Cadena, Cesar},
    journal={arXiv preprint arXiv:2109.10165},
    year={2021}
  }

Video

A short video overview explaining the approach will be released upon publication.

Installation

Installation instructions for Linux. The repository was developed on Ubuntu 18.04 with ROS melodic and also tested on Ubuntu 20.04 with ROS noetic.

Prerequisites

1. If not already done so, install ROS (Desktop-Full is recommended).

2. If not already done so, create a catkin workspace with catkin tools:

   # Create a new workspace
   sudo apt-get install python-catkin-tools
   mkdir -p ~/catkin_ws/src
   cd ~/catkin_ws
   catkin init
   catkin config --extend /opt/ros/$ROS_DISTRO
   catkin config --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo
   catkin config --merge-devel

Installation

1. Install system dependencies:

   sudo apt-get install python-wstool python-catkin-tools

2. Move to your catkin workspace:

   cd ~/catkin_ws/src

3. Download repo using SSH:

   git clone [email protected]:ethz-asl/panoptic_mapping.git

4. Download and install package dependencies using ros install:

   • If you created a new workspace.

   wstool init . ./panoptic_mapping/panoptic_mapping.rosinstall
   wstool update

   • If you use an existing workspace. Notice that some dependencies require specific branches that will be checked out.

   wstool merge -t . ./panoptic_mapping/panoptic_mapping.rosinstall
   wstool update

5. Compile and source:

   catkin build panoptic_mapping_utils
   source ../devel/setup.bash

Datasets

The datasets described in the paper and used for the demo can be downloaded from the ASL Datasets.

To a utility script is provided to directly download the data:

roscd panoptic_mapping_utils
export FLAT_DATA_DIR="/home/$USER/Documents"  # Or whichever path you prefer.
chmod +x panoptic_mapping_utils/scripts/download_flat_dataset.sh
./panoptic_mapping_utils/scripts/download_flat_dataset.sh

Additional data to run the mapper on the 3RScan dataset will follow.

Examples

Running the Panoptic Mapper

This example explains how to run the Panoptic Multi-TSDF mapper on the flat dataset.

1. First, download the flat dataset:

   export FLAT_DATA_DIR="/home/$USER/Documents"  # Or whichever path you prefer.
   chmod +x panoptic_mapping_utils/scripts/download_flat_dataset.sh
   ./panoptic_mapping_utils/scripts/download_flat_dataset.sh
   

2. Replace the data base_path in launch/run.launch (L10) and file_name in config/mapper/flat_groundtruth.yaml (L15) to the downloaded path.

3. Run the mapper:

   roslaunch panoptic_mapping_ros run.launch
   

4. You should now see the map being incrementally built:

   

5. After the map finished building, you can save the map:

   rosservice call /panoptic_mapper/save_map "file_path: '/path/to/run1.panmap'" 
   

6. Terminate the mapper pressing Ctrl+C. You can continue the experiment on run2 of the flat dataset by changing the base_path-ending in launch/run.launch (L10) to run2, and load_map and load_path in launch/run.launch (L26-27) to true and /path/to/run1.panmap, respectively. Optionally, you can also change the color_mode in config/mapper/flat_groundtruth.yaml (L118) to change to better highlight the change detection at work.

   roslaunch panoptic_mapping_ros run.launch
   

7. You should now see the map being updated based on the first run:

   

Monolithic Semantic Mapping

This example will follow shortly.

Running the RIO Dataset

This example will follow shortly.

Contributing

panoptic_mapping is an open-source project, any contributions are welcome!

For issues, bugs, or suggestions, please open a GitHub Issue.

To add to this repository:

• Please employ the feature-branch workflow.
• Setup our auto-formatter for coherent style (we follow the google style guide):

  # Download the linter
  cd <linter_dest>
  git clone [email protected]:ethz-asl/linter.git
  cd linter
  echo ". $(realpath setup_linter.sh)" >> ~/.bashrc
  bash
  roscd panoptic_mapping/..
  init_linter_git_hooks
  # You're all set to go!
  

• Please open a Pull Request for your changes.
• Thank you for contributing!