Go to file
Mateo de Mayo dd90a07092 Add usage documentation
Rewrites README.md and adds documentation and tools for both RealSense and WMR devices.
2022-05-06 16:16:01 -03:00
cmake_modules Build file improvements: pkgconfig, BUILD_TESTS flag, and other fixes 2022-05-06 15:52:27 -03:00
data Add usage documentation 2022-05-06 16:16:01 -03:00
doc Add usage documentation 2022-05-06 16:16:01 -03:00
docker Update Docker images 2021-10-16 15:37:24 +02:00
include/basalt Register internal stages timestamps 2022-05-06 15:38:53 -03:00
python/basalt ICCV'21 square root marginalization paper code release 2021-10-15 15:09:15 +02:00
scripts Add view_offset setting for cameras with low overlap 2022-05-06 15:21:29 -03:00
src Add slam_tracker adapter for Monado 2022-05-06 16:11:01 -03:00
test fix unsed-variable warning 2022-03-16 19:14:38 +01:00
thirdparty Add slam_tracker adapter for Monado 2022-05-06 16:11:01 -03:00
.clang-format add ./scripts/clang-format-all.sh and fix formatting 2019-04-24 13:16:06 +02:00
.clang-tidy Clang Tidy 2021-05-08 16:03:15 +00:00
.gitignore Build file improvements: pkgconfig, BUILD_TESTS flag, and other fixes 2022-05-06 15:52:27 -03:00
.gitlab-ci.yml ci: add monterey arm build 2021-12-14 04:15:49 +01:00
.gitmodules Update basalt-headers to support camera model rt8 2022-05-06 15:11:12 -03:00
.style.yapf ICCV'21 square root marginalization paper code release 2021-10-15 15:09:15 +02:00
CMakeLists.txt Add slam_tracker adapter for Monado 2022-05-06 16:11:01 -03:00
LICENSE Initial commit 2019-04-14 21:08:32 +02:00
README.md Add usage documentation 2022-05-06 16:16:01 -03:00

README.md

Basalt for Monado

This is a fork of Basalt with some modifications so that it can be used from Monado for SLAM tracking. Many thanks to the Basalt authors.

Follow this file for instructions on how to get Basalt up and running with Monado. This README tries to be as concise as possible, but there are many details that need to be addressed on it, so please do not skip any section, otherwise it is likely that it won't work. Having said that, this guide has been tested in limited setups, so please report any changes you had to make in order to get it working in different ones.

Index

Installation

This was tested on both Ubuntu 20.04 and 18.04, be sure to open an issue if the steps don't work for you.

Build and Install Directories

To not clutter your system directories, let's set two environment variables, $bsltdeps and $bsltinstall that point to existing empty build and install directories respectively. These directories will contain everything produced in this guide besides installed apt dependencies.

# Change the paths accordingly
export bsltinstall=/home/mateo/Documents/apps/bsltinstall
export bsltdeps=/home/mateo/Documents/apps/bsltdeps

Let's extend our system paths with those.

export PATH=$bsltinstall/bin:$PATH
export PKG_CONFIG_PATH=$bsltinstall/lib/pkgconfig:$PKG_CONFIG_PATH # for compile time pkg-config
export LD_LIBRARY_PATH=$bsltinstall/lib/:$LD_LIBRARY_PATH # for runtime ld
export LIBRARY_PATH=$bsltinstall/lib/:$LIBRARY_PATH # for compile time gcc

Dependencies

Most dependencies will be automatically built by basalt, however there are some known issues you might need to deal with (click to open the ones that might affect you).

Issues with GCC 11

If you are using GCC 11 you might also get some issues with pangolin as there is now a name clash with Pagolin _serialize() name, it should be fixed in newer versions of GCC-11. For fixing it yourself, you can cherry-pick these commits, or use a different GCC version. (see this discord thread in the Monado server for more info).

Build Basalt

cd $bsltdeps
git clone --recursive git@gitlab.freedesktop.org:mateosss/basalt.git
./basalt/scripts/install_deps.sh
sed -i "s#/home/mateo/Documents/apps/bsltdeps/#$bsltdeps/#" basalt/data/monado/*.toml
cd basalt && mkdir build && cd build
cmake .. -DCMAKE_INSTALL_PREFIX=$bsltinstall -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_TESTS=off -DBASALT_INSTANTIATIONS_DOUBLE=off
make install -j12

Running Basalt

This step is optional but you can try Basalt without Monado with one of the following methods:

  • Through an EuRoC dataset (be sure to download one first): basalt_vio --dataset-path /path/to/euroc/V1_01_easy --cam-calib $bsltdeps/basalt/data/euroc_ds_calib.json --dataset-type euroc --config-path $bsltdeps/basalt/data/euroc_config.json --marg-data ~/Desktop/euroc_marg_data --show-gui 1
  • With a RealSense T265 (you'll need to get a t265_calib.json yourself as detailed below but meanwhile you can try with this file instead): basalt_rs_t265_vio --cam-calib $bsltdeps/basalt/data/t265_calib.json --config-path $bsltdeps/basalt/data/euroc_config.json
  • With a RealSense D455 (and maybe this also works for a D435): basalt_rs_t265_vio --is-d455 --cam-calib $bsltdeps/basalt/data/d455_calib.json --config-path $bsltdeps/basalt/data/euroc_config.json

Monado Specifics

You'll need to compile Monado with the same Eigen used in Basalt, and with the same flags. For that, set these with CMake (or equivalent flags for meson): -DEIGEN3_INCLUDE_DIR=$bsltdeps/basalt/thirdparty/basalt-headers/thirdparty/eigen -DCMAKE_C_FLAGS="-march=native" -DCMAKE_CXX_FLAGS="-march=native" otherwise Monado will automatically use your system's Eigen, and having mismatched Eigen version/flags can cause a lot of headaches.

Run an OpenXR app like hello_xr with the following environment variables set

export EUROC_PATH=/path/to/euroc/V1_01_easy/ # Set euroc dataset path. You can get a dataset from http://robotics.ethz.ch/~asl-datasets/ijrr_euroc_mav_dataset/vicon_room1/V1_01_easy/V1_01_easy.zip
export EUROC_LOG=debug
export EUROC_HMD=false # if false, a fake controller will be tracked, else a fake HMD
export SLAM_LOG=debug
export SLAM_CONFIG=$bsltdeps/basalt/data/monado/euroc.toml # Point to Basalt config file for Euroc
export OXR_DEBUG_GUI=1 # We will need the debug ui to start streaming the dataset

Finally, run the XR app and press start in the euroc player debug ui and you should see a controller being tracked with Basalt from the euroc dataset.

Notes on Basalt Usage

  • Tracking is not perfect, this and this show how it looks, as well as the problems that it has (difficulties with rotation-only movements, wiggliness on fast movements, etc)
  • This fork only works with Stereo-IMU setups, but adapting Basalt to work with other configurations should feasible (see granite).
  • Basalt is fast. While the standard sampling rate is stereo 640x480 at 30fps I've been able to make it work at 848x480 at 60fps without problems on a laptop.
  • Some things that might cause crashes:
    • Using images with bad exposure and gain values, or being in a dark room.
    • Shaking causes drift that can diverge if maintained for long periods of time.
    • Continuously making sudden 90 degree rotations in which the new scene does not share features with the previous scene.
    • Moving too fast and/or making rotation only movements over extended periods of time.

Using Real Hardware

Monado has a couple of drivers supporting SLAM tracking (and thus Basalt). Here is how to set them up: