How to run VINS-Fusion-gpu on Jetson Xavier / Orin NX

* Version

 

Jetson : JetPACK 4.5.1

Ubuntu : 18.04

 

 

1. Install Jetpack 4.5.1 to the Jetson NX.

 

(SD card version, KOREAN)

https://yongee.tistory.com/45

Jetson Xavier NX에 sd카드 활용하여 JetPack 설치

 

 

verson check

sudo apt-cache show nvidia-jetpack

 

2. Install ROS melodic

 

http://wiki.ros.org/melodic/Installation/Ubuntu

melodic/Installation/Ubuntu - ROS Wiki

 

 

3. Install Eigen

 

(reference)

https://github.com/zinuok/VINS-Fusion

GitHub - zinuok/VINS-Fusion: VINS-Fusion setup for different nvidia boards: Jetson TX2, Xavier, NX

wget -O eigen-3.3.9.zip https://gitlab.com/libeigen/eigen/-/archive/3.3.9/eigen-3.3.9.zip 
unzip eigen-3.3.9.zip
cd ~/eigen-3.3.9
mkdir build && cd build
cmake ../ 
make -j4
sudo make install

Total number of cores in Xavier is 6, but sometimes it stucks when you use whole core to build

So I recommend to use only 4 of 6 cores.

 

 

 

4. Install Ceres

 

reference

https://github.com/zinuok/VINS-Fusion

GitHub - zinuok/VINS-Fusion: VINS-Fusion setup for different nvidia boards: Jetson TX2, Xavier, NX

sudo apt-get install -y cmake libgoogle-glog-dev libatlas-base-dev libsuitesparse-dev
wget http://ceres-solver.org/ceres-solver-1.14.0.tar.gz
tar zxf ceres-solver-1.14.0.tar.gz
cd ceres-solver-1.14.0
mkdir build && cd build
cmake -DEXPORT_BUILD_DIR=ON \
        -DCMAKE_INSTALL_PREFIX=/usr/local \
        ../
make -j 4 # number of cores
make test -j 4
sudo make install

Like 3, I recommend to use only 4 cores.

 

Sometimes, 30th test, evaluation_callback_test can be failed

But you can just ignore it.

 

5. Install OpenCV

Version : 3.4.1

reference

https://m.blog.naver.com/tinz6461/222135158313

[Ubuntu] OpenCV 3.4 설치

 

(before doing, please install all apt-get things in the blog above)

mkdir opencv_install
cd opencv_install
wget -O opencv.zip https://github.com/opencv/opencv/archive/3.4.1.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/3.4.1.zip
unzip opencv.zip
unzip opencv_contrib.zip

 

 

before building you have to modify a file related to cuda

sudo vim /usr/local/cuda/include/cuda_gl_interop.h

and go to line 61, and modify like below

//#if defined(__arm__) || defined(__aarch64__)
//#ifndef GL_VERSION
//#error Please include the appropriate gl headers before including cuda_gl_interop.h
//#endif
//#else
#include <GL/gl.h>
//#endif

comment all lines except #include <GL/gl.h>

 

and build opencv

cd opencv-3.4.1
mkdir build && cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE \
-D CMAKE_INSTALL_PREFIX=/usr/local \
-D WITH_TBB=OFF \
-D WITH_IPP=OFF \
-D WITH_1394=OFF \
-D BUILD_WITH_DEBUG_INFO=OFF \
-D BUILD_DOCS=OFF \
-D INSTALL_C_EXAMPLES=ON \
-D INSTALL_PYTHON_EXAMPLES=ON \
-D BUILD_EXAMPLES=OFF \
-D BUILD_TESTS=OFF \
-D BUILD_PERF_TESTS=OFF \
-D WITH_QT=ON \
-D WITH_CUDA=ON \
-D WITH_OPENGL=ON \
-D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib-3.4.1/modules \
-D WITH_V4L=ON \
-D WITH_FFMPEG=ON \
-D WITH_XINE=ON \
-D WITH_NVCUVID=OFF \
-D BUILD_NEW_PYTHON_SUPPORT=ON \../

make -j4
sudo make install
sudo sh -c 'echo '/usr/local/lib' > /etc/ld.so.conf.d/opencv.conf'
sudo ldconfig

 

 

6. install cv_bridge

In case you install opencv by source, you have to install cv_bridge also

 

* refernce

https://github.com/ros-perception/vision_opencv/tree/melodic

GitHub - ros-perception/vision_opencv

cd [your workspace]/src
git clone -b melodic https://github.com/ros-perception/vision_opencv.git

cd [your workspace]
catkin_make

before doing catkin_make, you have to specify CV's version to cv_bridge

# cv_bridge, CMakeLists.txt file

find_package(OpenCV 3 REQUIRED
  COMPONENTS
    opencv_core
    opencv_imgproc
    opencv_imgcodecs
  CONFIG
)
include(/usr/local/share/OpenCV/OpenCVConfig.cmake)

add include(/usr/local/share/OpenCV/OpenCVConfig.cmake) line

 

 

7. Install librealsense

version : 2.48.0

Download .zip file

https://github.com/IntelRealSense/librealsense/releases

Releases · IntelRealSense/librealsense

cd [librealsense_dir]
mkdir build && cd build

cmake .. \
-DFORCE_RSUSB_BACKEND=true \
-DBUILD_WITH_CUDA=true \
-DCMAKE_BUILD_TYPE=release


make
sudo make install

 

 

 

 

 

8. Install Realsense SDK

https://yongee.tistory.com/50

[Jetson] Jetson Xavier NX에 Realsense SDK(realsense-viewer) 설치

 

 

 

 

 

 

 

9.

git clone below package and build

 

https://github.com/yongeePark/VINS-Fusion-gpu-XavierNX-D435i

GitHub - yongeePark/VINS-Fusion-gpu-XavierNX-D435i: This repository is a version of VINS-Fusion with gpu acceleration. It can ru

 

catkin_make

 

 

* etc

(Currently it is changed to opencv4 with orin, ubuntu 20.04 version)

When you use orin, you have to change the opencv version

or modify the code inside the vins by following below links.

 

https://github.com/HKUST-Aerial-Robotics/VINS-Fusion/issues/167

VINS-fusion on Ubuntu 20.04 · Issue #167 · HKUST-Aerial-Robotics/VINS-Fusion

https://github.com/Dovyski/cvui/issues/52

Compile error in the examples with OpenCV 4.0.0-alpha · Issue #52 · Dovyski/cvui

https://github.com/HKUST-Aerial-Robotics/VINS-Mono/issues/275

catkin_make · Issue #275 · HKUST-Aerial-Robotics/VINS-Mono

https://github.com/HKUST-Aerial-Robotics/VINS-Mono/pull/385/files?w=1 

fix: fix build for ros noetic by huahang · Pull Request #385 · HKUST-Aerial-Robotics/VINS-Mono