Basalt for Monado

This is a fork of Basalt with some modifications so that it can be used from Monado for SLAM tracking. Many thanks to the Basalt authors.

Follow this file for instructions on how to get Basalt up and running with Monado. This README tries to be as concise as possible, but there are many details that need to be addressed on it, so please do not skip any section, otherwise it is likely that it won't work. Having said that, this guide has been tested in limited setups, so please report any changes you had to make in order to get it working in different ones.

Index

• Basalt for Monado
  • Index
  • Installation
    • Build and Install Directories
    • Dependencies
    • Build Basalt
    • Running Basalt
    • Monado Specifics
  • Notes on Basalt Usage
  • Using Real Hardware

Installation

This was tested on both Ubuntu 20.04 and 18.04, be sure to open an issue if the steps don't work for you. The main branch of this fork is xrtslam.

Build and Install Directories

To not clutter your system directories, let's set two environment variables, $bsltdeps and $bsltinstall that point to existing empty build and install directories respectively. These directories will contain everything produced in this guide besides installed apt dependencies.

# Change the paths accordingly
export bsltinstall=/home/mateo/Documents/apps/bsltinstall
export bsltdeps=/home/mateo/Documents/apps/bsltdeps

Let's extend our system paths with those.

export PATH=$bsltinstall/bin:$PATH
export PKG_CONFIG_PATH=$bsltinstall/lib/pkgconfig:$PKG_CONFIG_PATH # for compile time pkg-config
export LD_LIBRARY_PATH=$bsltinstall/lib/:$LD_LIBRARY_PATH # for runtime ld
export LIBRARY_PATH=$bsltinstall/lib/:$LIBRARY_PATH # for compile time gcc

Dependencies

Most dependencies will be automatically built by basalt, however there are some known issues you might need to deal with (click to open the ones that might affect you).

Issues with GCC 11

If you are using GCC 11 you might also get some issues with pangolin as there is now a name clash with Pagolin _serialize() name, it should be fixed in newer versions of GCC-11. For fixing it yourself, you can cherry-pick these commits, or use a different GCC version. (see this discord thread in the Monado server for more info).

Build Basalt

cd $bsltdeps
git clone --recursive git@gitlab.freedesktop.org:mateosss/basalt.git
./basalt/scripts/install_deps.sh
sed -i "s#/home/mateo/Documents/apps/bsltdeps/#$bsltdeps/#" basalt/data/monado/*.toml
cd basalt && mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=$bsltinstall -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_TESTS=off -DBASALT_INSTANTIATIONS_DOUBLE=off
make install -j12

Running Basalt

This step is optional but you can try Basalt without Monado with one of the following methods:

• Through an EuRoC dataset (be sure to download one first): basalt_vio --dataset-path /path/to/euroc/V1_01_easy --cam-calib $bsltdeps/basalt/data/euroc_ds_calib.json --dataset-type euroc --config-path $bsltdeps/basalt/data/euroc_config.json --marg-data ~/Desktop/euroc_marg_data --show-gui 1
• With a RealSense T265 (you'll need to get a t265_calib.json yourself as detailed below but meanwhile you can try with this file instead): basalt_rs_t265_vio --cam-calib $bsltdeps/basalt/data/t265_calib.json --config-path $bsltdeps/basalt/data/euroc_config.json
• With a RealSense D455 (and maybe this also works for a D435): basalt_rs_t265_vio --is-d455 --cam-calib $bsltdeps/basalt/data/d455_calib.json --config-path $bsltdeps/basalt/data/euroc_config.json

Monado Specifics

You'll need to compile Monado with the same Eigen used in Basalt, and with the same flags. For that, set these with CMake: -DEIGEN3_INCLUDE_DIR=$bsltdeps/basalt/thirdparty/basalt-headers/thirdparty/eigen -DCMAKE_C_FLAGS="-march=native" -DCMAKE_CXX_FLAGS="-march=native" otherwise Monado will automatically use your system's Eigen, and having mismatched Eigen version/flags can cause a lot of headaches.

Run an OpenXR app like hello_xr with the following environment variables set

export EUROC_PATH=/path/to/euroc/V1_01_easy/ # Set euroc dataset path. You can get a dataset from http://robotics.ethz.ch/~asl-datasets/ijrr_euroc_mav_dataset/vicon_room1/V1_01_easy/V1_01_easy.zip
export EUROC_LOG=debug
export EUROC_HMD=false # if false, a fake controller will be tracked, else a fake HMD
export EUROC_PLAY_FROM_START=true
export SLAM_LOG=debug
export SLAM_SUBMIT_FROM_START=true
export SLAM_CONFIG=$bsltdeps/basalt/data/monado/euroc.toml # Point to Basalt config file for Euroc
export OXR_DEBUG_GUI=1 # We will need the debug ui to start streaming the dataset

Finally, run the XR app and press start in the euroc player debug ui and you should see a controller being tracked with Basalt from the euroc dataset.

Notes on Basalt Usage

• Tracking is not perfect, this and this show how it looks, as well as the problems that it has (difficulties with rotation-only movements, wiggliness on fast movements, etc)
• This fork only works with Stereo-IMU setups, but adapting Basalt to work with other configurations should feasible (see granite).
• Basalt is fast. While the standard sampling rate is stereo 640x480 at 30fps I've been able to make it work at 848x480 at 60fps without problems on a laptop.
• Some things that might cause crashes:
  • Using images with bad exposure and gain values, or being in a dark room.
  • Shaking causes drift that can diverge if maintained for long periods of time.
  • Continuously making sudden 90 degree rotations in which the new scene does not share features with the previous scene.
  • Moving too fast and/or making rotation only movements over extended periods of time.

Using Real Hardware

Monado has a couple of drivers supporting SLAM tracking (and thus Basalt). Here is how to set them up:

• RealSense Driver
• WMR Driver