159 lines
6.8 KiB
Markdown
159 lines
6.8 KiB
Markdown
# Basalt for Monado
|
|
|
|
This is a fork of [Basalt](https://gitlab.com/VladyslavUsenko/basalt) with some
|
|
modifications so that it can be used from Monado for SLAM tracking. Many thanks
|
|
to the Basalt authors.
|
|
|
|
Follow this file for instructions on how to get Basalt up and running with
|
|
Monado. This README tries to be as concise as possible, but there are many
|
|
details that need to be addressed on it, so please do not skip any section,
|
|
otherwise it is likely that it won't work. Having said that, this guide has
|
|
been tested in limited setups, so please report any changes you had to make
|
|
in order to get it working in different ones.
|
|
|
|
## Index
|
|
|
|
- [Basalt for Monado](#basalt-for-monado)
|
|
- [Index](#index)
|
|
- [Installation](#installation)
|
|
- [Build and Install Directories](#build-and-install-directories)
|
|
- [Dependencies](#dependencies)
|
|
- [Build Basalt](#build-basalt)
|
|
- [Running Basalt](#running-basalt)
|
|
- [Monado Specifics](#monado-specifics)
|
|
- [Notes on Basalt Usage](#notes-on-basalt-usage)
|
|
- [Using Real Hardware](#using-real-hardware)
|
|
|
|
## Installation
|
|
|
|
This was tested on both Ubuntu 20.04 and 18.04, be sure to open an issue if the
|
|
steps don't work for you. The main branch of this fork is
|
|
[`xrtslam`](https://gitlab.freedesktop.org/mateosss/basalt/-/tree/xrtslam).
|
|
|
|
### Build and Install Directories
|
|
|
|
To not clutter your system directories, let's set two environment variables,
|
|
`$bsltdeps` and `$bsltinstall` that point to existing empty build and install
|
|
directories respectively. These directories will contain everything produced in
|
|
this guide besides installed apt dependencies.
|
|
|
|
```bash
|
|
# Change the paths accordingly
|
|
export bsltinstall=/home/mateo/Documents/apps/bsltinstall
|
|
export bsltdeps=/home/mateo/Documents/apps/bsltdeps
|
|
```
|
|
|
|
Let's extend our system paths with those.
|
|
|
|
```bash
|
|
export PATH=$bsltinstall/bin:$PATH
|
|
export PKG_CONFIG_PATH=$bsltinstall/lib/pkgconfig:$PKG_CONFIG_PATH # for compile time pkg-config
|
|
export LD_LIBRARY_PATH=$bsltinstall/lib/:$LD_LIBRARY_PATH # for runtime ld
|
|
export LIBRARY_PATH=$bsltinstall/lib/:$LIBRARY_PATH # for compile time gcc
|
|
```
|
|
|
|
### Dependencies
|
|
|
|
Most dependencies will be automatically built by basalt, however there are some
|
|
known issues you might need to deal with (click to open the ones that might
|
|
affect you).
|
|
|
|
<details>
|
|
<summary>Issues with GCC 11</summary>
|
|
|
|
If you are using GCC 11 you might also get some issues with pangolin as there is now a
|
|
[name clash with Pagolin `_serialize()` name](https://github.com/stevenlovegrove/Pangolin/issues/657),
|
|
it [should be fixed](https://gcc.gnu.org/bugzilla/show_bug.cgi?id=100438#c12)
|
|
in newer versions of GCC-11. For fixing it yourself, you can cherry-pick
|
|
[these commits](https://github.com/stevenlovegrove/Pangolin/pull/658/commits),
|
|
or use a different GCC version.
|
|
(see
|
|
[this discord thread](https://discord.com/channels/556527313823596604/556527314670714901/904339906288050196)
|
|
in the Monado server for more info).
|
|
</details>
|
|
|
|
### Build Basalt
|
|
|
|
```bash
|
|
cd $bsltdeps
|
|
git clone --recursive git@gitlab.freedesktop.org:mateosss/basalt.git
|
|
./basalt/scripts/install_deps.sh
|
|
sed -i "s#/home/mateo/Documents/apps/bsltdeps/#$bsltdeps/#" basalt/data/monado/*.toml
|
|
cd basalt && mkdir build && cd build
|
|
cmake .. -DCMAKE_INSTALL_PREFIX=$bsltinstall -DCMAKE_BUILD_TYPE=RelWithDebInfo -DBUILD_TESTS=off -DBASALT_INSTANTIATIONS_DOUBLE=off
|
|
make install -j12
|
|
```
|
|
|
|
### Running Basalt
|
|
|
|
This step is optional but you can try Basalt without Monado with one of the following methods:
|
|
|
|
- Through an EuRoC dataset (be sure to [download
|
|
one](http://robotics.ethz.ch/~asl-datasets/ijrr_euroc_mav_dataset/vicon_room1/)
|
|
first): `basalt_vio --dataset-path /path/to/euroc/V1_01_easy --cam-calib
|
|
$bsltdeps/basalt/data/euroc_ds_calib.json --dataset-type euroc --config-path
|
|
$bsltdeps/basalt/data/euroc_config.json --marg-data ~/Desktop/euroc_marg_data
|
|
--show-gui 1`
|
|
- With a RealSense T265 (you'll need to get a `t265_calib.json` yourself as
|
|
detailed [below](#configuring-basalt) but meanwhile you can try with [this
|
|
file](https://gitlab.com/VladyslavUsenko/basalt/-/issues/52) instead):
|
|
`basalt_rs_t265_vio --cam-calib $bsltdeps/basalt/data/t265_calib.json
|
|
--config-path $bsltdeps/basalt/data/euroc_config.json`
|
|
- With a RealSense D455 (and maybe this also works for a D435):
|
|
`basalt_rs_t265_vio --is-d455 --cam-calib
|
|
$bsltdeps/basalt/data/d455_calib.json --config-path
|
|
$bsltdeps/basalt/data/euroc_config.json`
|
|
|
|
### Monado Specifics
|
|
|
|
You'll need to compile Monado with the same Eigen used in Basalt, and with the
|
|
same flags. For that, set these with CMake:
|
|
`-DEIGEN3_INCLUDE_DIR=$bsltdeps/basalt/thirdparty/basalt-headers/thirdparty/eigen
|
|
-DCMAKE_C_FLAGS="-march=native" -DCMAKE_CXX_FLAGS="-march=native"` otherwise
|
|
Monado will automatically use your system's Eigen, and having mismatched Eigen
|
|
version/flags can cause a lot of headaches.
|
|
|
|
Run an OpenXR app like `hello_xr` with the following environment variables set
|
|
|
|
```bash
|
|
export EUROC_PATH=/path/to/euroc/V1_01_easy/ # Set euroc dataset path. You can get a dataset from http://robotics.ethz.ch/~asl-datasets/ijrr_euroc_mav_dataset/vicon_room1/V1_01_easy/V1_01_easy.zip
|
|
export EUROC_LOG=debug
|
|
export EUROC_HMD=false # if false, a fake controller will be tracked, else a fake HMD
|
|
export EUROC_PLAY_FROM_START=true
|
|
export SLAM_LOG=debug
|
|
export SLAM_SUBMIT_FROM_START=true
|
|
export SLAM_CONFIG=$bsltdeps/basalt/data/monado/euroc.toml # Point to Basalt config file for Euroc
|
|
export OXR_DEBUG_GUI=1 # We will need the debug ui to start streaming the dataset
|
|
```
|
|
|
|
Finally, run the XR app and press start in the euroc player debug ui and you
|
|
should see a controller being tracked with Basalt from the euroc dataset.
|
|
|
|
## Notes on Basalt Usage
|
|
|
|
- Tracking is not perfect, [this](https://youtu.be/mIgRHmxbaC8) and
|
|
[this](https://youtu.be/gxu3Ve8VCnI) show how it looks, as well as the
|
|
problems that it has (difficulties with rotation-only movements, wiggliness on
|
|
fast movements, etc)
|
|
- This fork only works with Stereo-IMU setups, but adapting Basalt to work with
|
|
other configurations should feasible (see
|
|
[granite](https://github.com/DLR-RM/granite)).
|
|
- Basalt is _fast_. While the standard sampling rate is stereo 640x480 at 30fps
|
|
I've been able to make it work at 848x480 at 60fps without problems on a
|
|
laptop.
|
|
- Some things that might cause crashes:
|
|
- Using images with bad exposure and gain values, or being in a dark room.
|
|
- Shaking causes drift that can diverge if maintained for long periods of
|
|
time.
|
|
- Continuously making sudden 90 degree rotations in which the new scene does not share
|
|
features with the previous scene.
|
|
- Moving too fast and/or making rotation only movements over extended periods
|
|
of time.
|
|
|
|
## Using Real Hardware
|
|
|
|
Monado has a couple of drivers supporting SLAM tracking (and thus Basalt). Here is how to set them up:
|
|
|
|
- [RealSense Driver](doc/monado/Realsense.md)
|
|
- [WMR Driver](doc/monado/WMR.md)
|