TzuTaLin's blog

6/17/2016

OpenCL on Android

Introduction and setup
In android, Android/Google doesn't support OpenCL officially. Fortunately, many chip vendors like Qualquam90/ Nvidia/ Intel / MTK provide their libraries to support OpenCL on Android.

To use OpenCL on Android. If you want to use OpenCL, you need to check if there is OpenCL library on the device. You can download apps from Google Play to query OpenCL information like the version of OpenCL, device type, and benchmark tests. You can also go through List of Android Device with OpenCL support to see the benchmarks.

The OpenCL libraries for the major chip vendors can be found in the devices. The followings are the location of the OpenCL library:

Qualquam(QUALCOMM Adreno)/Intel(HD Graphics)/Nvidia

/system/vendor/lib/libOpenCL.so
or /system/lib/libOpenCL.so (older devices)

ARM Mali:

/system/vendor/lib/egl/libGLES_mali.so
or /system/lib/egl/libGLES_mali.so

PowerVR:

/system/vendor/lib/libPVROCL.so

Create NDK project to include OpenCL headers, compile your C/C++ code and link to CL shared library, and test them on the device as executable. Moreover, you can write JNI glue code and Java code to run on Java layer.

Quick setup and use Boost.Compute which is a GPU/parallel-computing library for C++ based on OpenCL.

You can refer to Boost.Compute-Android. If you like it, please give a star to this git repository.

Image annotation tool / Create bounding box for training images

5/15/2016

Ubuntu 安裝新細明體

下載字型

$ wget http://mingliu.myweb.hinet.net/MingLiu/MingLiU.zip ; uznip MingLiU.zip ;

$ sudo mkdir -p /usr/share/fonts/myfonts; sudo cp MingLiU.ttc /usr/share/fonts/myfont/新細明體.ttc

重新載入字型

$ fc-cache -v /usr/share/fonts/myfonts

查看所有已安裝的所有字型

$ fc-list

4/24/2016

Facial Landmarks implemented by DLib on Android

In the past, Google had provided Vision API to detect facial landmark. However, if you would like to use that, it requires Android Play Services SDK level 26 or greater. Besides, the number of landmarks might not be enough for you. Luckily, after the release of dlib, v18.10, it includes a number of new minor features. The main addition in that release is an implementation of an excellent paper from this year's Computer Vision and Pattern Recognition Conference:

One Millisecond Face Alignment with an Ensemble of Regression Trees by Vahid Kazemi and Josephine Sullivan

The implementation of facial landmark extraction is fast enough, and it has 68 landmarks. Therefore, I spent a while to implement this feature to Android platform.

Hope it can be helpful to Android developers. If it is helpful to you, please give me stars on my Github repositories, dlib-android and dlib-android-app.

Setup and install Web server on ubuntu

In the article, I will show simple commands to install apache web server for local development and run a web page on Ubuntu.

Install Apache

$ sudo apt-get install apache2

Open your browser, and go to http://localhost/

Now, you can see the HTML file in /var/www/html/. Then, you can replace html file with yours.

For example, I clone my own web page from my github, and display it on browser.

$ git clone https://github.com/tzutalin/tzutalin.github.io.git

$ cd tzutalin.github.io; sudo cp -rf . /var/www/html/

Finally, open your browser and type "localhost", then you can see the web page

Note:

If you would like to register your own domain name, you can find DNS providers like GoDaddy, Namecheap, 1&1 Internet, Dotster.com, etc.

4/20/2016

Add Singleton Pattern to AndroidStudio Live Templates

AndroidStudio provides Live Templates to increase productivity for Android developers.

You can complete and insert code snippets into your code by typing their abbreviation and pressing tab. You can see and change those settings in the Editor section of the Preferences.

You can add your own templates. E.g. I add Singleton pattern, thus I can implement and add Singleton Pattern in a class quickly by type my defined abbreviation('singleton').

abbreviation : singleton
description : Generate Singleton template for the current class
template text:

private $class$() {
    
}

private static class $class$LazyHolder {
   private static final $class$ INSTANCE = new $class$();
}

public static $class$ getInstance() {
   return $class$LazyHolder.INSTANCE;
}

Remember to let it be applicable in Java

Finally, in your java editor

3/19/2016

The list of vision-based SLAM / Visual Odometry open source and papers

RGB(Monocular)

LSD-SLAM
Source : https://github.com/tum-vision/lsd_slam
Publications :
LSD-SLAM: Large-Scale Direct Monocular SLAM, J. Engel, T. Schöps, D. Cremers, ECCV '14
Semi-Dense Visual Odometry for a Monocular Camera, J. Engel, J. Sturm, D. Cremers, ICCV '13
Available on ROS : Yes

ORB-SLAM
Source : https://github.com/raulmur/ORB_SLAM
Website: http://webdiis.unizar.es/~raulmur/orbslam/
Publications : ORB-SLAM: A Versatile and Accurate Monocular SLAM System
Available on ROS : Yes

Nister's Five Point Algorithm for Essential Matrix estimation, and FAST features, with a KLT tracker
Source : https://github.com/avisingh599/mono-vo
Website: http://avisingh599.github.io/vision/monocular-vo/
Publications: http://avisingh599.github.io/assets/ugp2-report.pdf
Available on ROS : No

RGB-D

OpenCV RGBD-Odometry (Visual Odometry based RGB-D images)

Source : https://github.com/tzutalin/OpenCV-RgbdOdometry

Website: https://github.com/tzutalin/OpenCV-RgbdOdometry

Publications: Real-Time Visual Odometry from Dense RGB-D Images, F. Steinbucker, J. Strum, D. Cremers, ICCV, 2011
Available on ROS : No

Dense Visual SLAM for RGB-D Cameras

Source : https://github.com/tum-vision/dvo_slam

Website: http://vision.in.tum.de/data/software/dvo

Publications: Dense Visual SLAM for RGB-D Cameras

Available on ROS : Yes

RTAB MAP - Real-Time Appearance-Based Mapping

Source : https://github.com/introlab/rtabmap

Website: http://wiki.ros.org/rtabmap

Publications:

Online Global Loop Closure Detection for Large-Scale Multi-Session Graph-Based SLAM, 2014

Appearance-Based Loop Closure Detection for Online Large-Scale and Long-Term Operation, 2013

Available on ROS : Yes

ORB2-SLAM
Source : https://github.com/raulmur/ORB_SLAM2
Website: http://webdiis.unizar.es/~raulmur/orbslam/
Publications : ORB-SLAM: A Versatile and Accurate Monocular SLAM System

Available on ROS : Yes

Useful third parties

Basic library
OpenCV
Eigen
Sophus
ROS
PointCloud

Loop detection
dorian3d

Graph Optimization
ceres-solver
g2o
gtasm
Vertigo

Map library
Grip Map
OmniMapper
OctoMap

Other useful references

Openslam : Introduce some other SLAM systems and open sources
Kintinuous : Real-time large scale dense visual SLAM system
InfiniTAM : A Framework for the Volumetric Integration of Depth Images
ElasticFusion : Real-time dense visual SLAM system

RGB-D dataset download

TUM Universtiy

KTTI Vision benchmark

RGB-D tools

https://vision.in.tum.de/data/datasets/rgbd-dataset/tools

2/25/2016

Ubuntu 解決標楷體問題

安裝ubuntu14.04時, 如果選擇英語語系, 會幫你安裝中文套件, 預設字體是標楷體

解決方法;
$ sudo vim /etc/fonts/conf.d/69-language-selector-zh-tw.conf

然後分別加入

<string>WenQuanYi Zen Hei</string>
<string>WenQuanYi Micro Hei</string>

<string>WenQuanYi Micro Hei Mono</string>

2/23/2016

Dense Visual SLAM for RGB-D Cameras ( dvo_slam setups )

What is DVO SLAM

http://vision.in.tum.de/data/software/dvo

Requierements

Software: Ubuntu 12.04 using ROS Fuerte or Ubuntu 14.04 using ROS Indigo

Hardware: RGB-D cameras like RealSense r200, Asus Xtion, Kinect, etc.

(Optional)

If you use ubuntu 12.04 with ROS Fuerte and camera is Kinect or Xtion, you have to setup your camera first.

$ lsusb–v
$ sudo apt-get install ros-fuerte-openni-kinect

if it is 0601 not 0600(old verison)

$ sudo apt-get install --reinstall libopenni-sensor-primesense0
$ sudo gedit /etc/openni/GlobalDefaults.ini

Set `UsbInterface=2`

$ sudo reboot

After reboot, launch openni script, it will open your camera

$ roscore

$ roslaunch openni_launch openni.launch

Use image_view tool to see your RGB and depth images

$ rosrun image_view image_view image:=/camera/rgb/image_color

$ rosrun image_view image_view image:=/camera/depth/image

Clone the source

$ git clone https://github.com/jefftee/dvo_slam.git

$ rosmake dvo_core dvo_ros dvo_slam dvo_benchmark

Ubuntu 12.04 using ROS Fuerte:

$ git clone -b fuerte https://github.com/tum-vision/dvo_slam.git

or clone it from my repository (I add some launch files)

$ git clone -b fuerte git@bitbucket.org:TzuTaLin/dvo_slam.git

Build it


$ export ROS_PACKAGE_PATH=$ROS_PACKAGE_PATH:~/dvo_slam

$ sudo apt-get install ros-fuerte-libg2o liblapack-dev libblas-dev freeglut3-dev libqglviewer-qt4-dev libsuitesparse-dev libx11-dev

$ rosmake dvo_core dvo_ros dvo_slam dvo_benchmark

Clean

$ roscd your_package_name && make clean

Run keyframe tracker
$ rosrun dvo_slam camera_keyframe_tracker

Run tracker without keyframe
$ rosrun dvo_ros camera_tracker

Run and show configure, and Visualize

$ roslaunch dvo_slam qucikstart.lauch

Run benchmark
Download the dataset:
http://vision.in.tum.de/rgbd/dataset/freiburg1/rgbd_dataset_freiburg1_xyz.tgz

Extract it and go the folder

$ associate.py rgb.txt depth.txt > assoc.txt

You can download the tool from https://vision.in.tum.de/data/datasets/rgbd-dataset/tools

Than in the dataset folder, run:

$ roslaunch dvo_benchmark benchmark.launch keep_alive 
or
$ roslaunch dvo_benchmark benchmark.launch

Debug (Optional)

$ rosrun rqt rq

Change the name for registerd topics (Optional)

https://github.com/tum-vision/dvo_slam/blob/d25b65bff58860bb5a3a39a742459db3ba98eaa6/dvo_ros/src/camera_base.cpp

1/22/2016

Build and install OpenCV and contrib lib on Ubuntu / Android

I am going to show you how to clone the opencv source from Git repository. After cloing the source, I will build and install it.

Build for ubuntu

Download the source of opencv and opencv_contrib from github and checkout to the tag, 3.1.0:

cd ~/<my_working _directory>
git clone https://github.com/Itseez/opencv.git
git clone https://github.com/Itseez/opencv_contrib.git
cd ~/<my_working _directory>/opencv
git checkout -b 3.1.0 3.1.0
cd ~/<my_working _directory>/opencv_contrib
git checkout -b 3.1.0 3.1.0

Use cmake:

cd opencv
mkdir release
cd release
cmake -DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules/ CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..

If your environment contains CUDA, by default, it will build it with CUDA. If you don't want to build it with CUDA, add -DWITH_CUDA=OFF to your cmake command line.

Start to build and install to /usr/local/include and /usr/local/lib. You can change the installed dir by change CMAKE_INSTALL_PREFIX

make -j8 # -j8 runs 8 jobs in parallel.
         # Change 8 to number of hardware threads available.
sudo make install

You can check your current opencv version:

$ pkg-config --modversion opencv

It will be 3.1.0 !

You can also look at the headers or libs installed. Using pkg-config can tell you where they are:

$ pkg-config --cflags opencv
$ pkg-config --libs opencv

Actually, you can use the latest stable OpenCV version available in sourceforge without building the source.

How to speed up building process?
You can use Ninja to speed up instead of using Makefile

cd opencv
mkdir release
cd release
cmake -G Ninja -DOPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules/ CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
ninja -j8
ninja install

Build for Android version:

export ANDROID_NDK=[YourNDKPath]
cd ~/<my_working _directory>git clone https://gist.github.com/tzutalin/f07f5633fd599001101006561689062b
sh build-android-opencv.sh

You can change your target API in build_android_opencv.sh. For example, you can build the target for arm by changing the variable, ANDROID_ABI, in build_android_opencv.sh which you clone from the above commands.

The final output will be installed and located in ~/<my_working _directory>/android_opencv/

Alternatively, you can refer to https://github.com/tzutalin/build-opencv-for-android

References:
http://code.opencv.org/projects/opencv/wiki/Building_OpenCV4Android_from_trunk
http://docs.opencv.org/3.1-beta/doc/tutorials/introduction/linux_install/linux_install.html
https://github.com/Itseez/opencv/wiki/Building_OpenCV4Android_from_trunk
https://github.com/alexkarargyris/Caffe_OpenCV_Android_App

NDK Download

https://developer.android.com/ndk/downloads