Running Qt GUI apps with Docker

  In the past three weeks, I was playing around Docker and CI/CD pipeline, and I was getting familiar with Docker and its toolbox. In the meanwhile, I tried to dockerize most of my own applications, tools, and algorithms which can run, build, or test inside Docker container.
  Here is one of my projects on Github, it's called labelImg which was written in Python + PyQt. You can follow the below videos or the snippet of the commands to run Python + PyQt GUI application inside the Docker container.

Step1: Clone the source code from Github

$ git clone https://github.com/tzutalin/labelImg.git

Step2: Pull the Docker image which is based on Python + PyQt. You can refer to its Dockerfile.

$ docker pull tzutalin/py2qt4

Step3: Get started with the application and container

docker run -it \
--user $(id -u) \
-e DISPLAY=unix$DISPLAY \
--workdir=$(pwd) \
--volume="/home/$USER:/home/$USER" \
--volume="/etc/group:/etc/group:ro" \
--volume="/etc/passwd:/etc/passwd:ro" \
--volume="/etc/shadow:/etc/shadow:ro" \
--volume="/etc/sudoers.d:/etc/sudoers.d:ro" \
-v /tmp/.X11-unix:/tmp/.X11-unix \
tzutalin/py2qt4

Then, you will see that we can run GUI inside the container.

If you want to get more detailed information, you can check out the following video.

Create insecure and private docker resistry

Pre-requirements on the host and clients:

Installed Docker and Docker-compose

Host

Create a folder for Docker registry

$ mkdir docker-registry;cd $_;mkdir data

Create a  docker-compose.yml for docker-compose.

registry:

  restart: always

  image: registry:2

  ports:

    - 5000:5000

  environment:

    REGISTRY_STORAGE_FILESYSTEM_ROOTDIRECTORY: /data

  volumes:

    - ./data:/data

Get any image from the hub and tag it to point to your registry:

$ docker pull ubuntu && docker tag ubuntu localhost:5000/ubuntu

Then, push it to your registry:

$ docker push localhost:5000/ubuntu

Clients

Because the above setting is an insecure and private registry. You need to add --insecure-registry to /etc/default/docker as below.

DOCKER_OPTS="$DOCKER_OPTS --insecure-registry=HOST:5000"

Afterward, you can pull the image from host server by running the below command

$ docker pull HOST:5000/ubuntu

Setup Jenkins on Ubuntu

Installation

wget -q -O - https://pkg.jenkins.io/debian/jenkins-ci.org.key | sudo apt-key add -
sudo sh -c 'echo deb http://pkg.jenkins.io/debian-stable binary/ > /etc/apt/sources.list.d/jenkins.list'
sudo apt-get update
sudo apt-get install jenkins

Upgrade

Once installed like this, you can update to the later version of Jenkins (when it comes out) by running the following commands:

sudo apt-get update
sudo apt-get install jenkins

Setting

Add user jenkins to your username group to allow it to read and write the your folder

sudo usermod -a -G YOUR_USERNAME jenkins

If you would like to change the port, you can edit the /etc/default/jenkins to replace the line

HTTP_PORT=8080

by

HTTP_PORT=8880

sudo service jenkins restart 

Then, try to open 0.0.0.0:8880, you will see authentication page. Now, you need to enter the password, you should get /var/libs/jenkins/secrets/ to see the password.


After that, you can go to a configuration page to set up your env variable.
http://localhost:8880/configure

For example, if you are using Android and gradle to build, you should setup ANDROID_HOME, ANDROID_NDK_HOME,  and JAVA_HOME as env variable.
  ANDROID_HOME : /home/darrenl/tools/android-sdk-linux

ANDROID_NDK_HOME

JAVA_HOME = /usr/lib/jvm/java-1.8.0-openjdk-amd64

Extra nice plugins

* Locale Plugin
* Email Extension Plugin
* Role Strategy Plugin

Run Jenkins with Do

Please refer to
https://github.com/tzutalin/docker/tree/master/Jenkins
https://github.com/jenkinsci/docker/blob/master/Dockerfile
https://github.com/maxfields2000/dockerjenkins_tutorial/tree/master/tutorial_07


Other references
Jenkins: Change Workspaces and Build Directory Locations
How to change build periodic
Jenkis pipeline with docker
Use email-plugin
Use email-plugin2
Create a test smtp server 
Simple project which use travis

flock c/c++ Sample Code

flock() is to apply or remove an advisory lock on an open file. It is available on most of OS, and I have tried it and it can work on Linux and Android OS. It can resolve the problem which muti-processes access the same file.

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/file.h>
#include <iostream>

void lockUnLock() {
  int fd, i;
  char path[] = "test.txt";
  fd = open(path, O_WRONLY | O_CREAT);
  if (fd != -1) {
    std::cout << "open file " << path << std::endl;
    std::cout << "Please input a number to lock the file" << path << std::endl;
    scanf("%d", &i);
    if (flock(fd, 2) == 0) {
      std::cout << "The file was locked " << std::endl;
    } else {
      std::cout << "The file was not locked " << std::endl;
    }
    std::cout << "please input a number to unlock the file " << std::endl;
    scanf("%d", &i);
    if (flock(fd, 8) == 0) {
      std::cout << "The file was unlocked. " << std::endl;
    } else {
      std::cout << "The file was no unlocked. " << std::endl;
    }
    close(fd);
  } else {
    std::cout << "Cannot open file " << path << std::endl;
  }
}

int main() {
  lockUnLock();
  return 0;
}

Src code: : https://gist.github.com/tzutalin/ad4277e8ee24392b9ea23765452d6528
Note: You can also use semget/semop to do that, but Android Bionic does not support that..

JNI Should I call DeleteLocalRef

In JNI, FindClass method returns a local reference.
Sample code:

 jclass arrayListClass = env->FindClass("java/util/ArrayList");
 ....
 env->DeleteLocalRef(arrayListClass);

Is it necessary to call DeleteLocalRef ? In fact, it is unnecessary to call DeleteLocalRef in most of the cases. However, JNI has a limited (but configurable) number of local references available. So, the best practice is to delete if the reference is created in a loop

C++ Version header template

Make a note for the usage of my version.hpp.
I can not only get the string of version from LIB_VERSION but also get major/minor/patch versions separately.

#pragma once
//  A *string*, LIB_VERSION, in the form "x.y.[z]"
//  where x is the major version number,
//  y is the minor version number,
//  and z is the patch level

#define LIB_VERSION_MAJOR    0
#define LIB_VERSION_MINOR    4
#define LIB_VERSION_PATCH    12

#define AUX_STR_EXP(__A)     #__A
#define AUX_STR(__A)         AUX_STR_EXP(__A)

#define LIB_VERSION          AUX_STR(LIB_VERSION_MAJOR) "."
        \ AUX_STR(LIB_VERSION_MINOR) "." 
        \ AUX_STR(LIB_VERSION_PATCH)
  

Besides, I wrote a python script to update the patch.

REG_VERSION_LINE = r"ASUS_VISION_LIB_VERSION_PATCH.  +([0-9:]+)"

def changeSoVersion(versionFile):
    # Read in the file
    filedata = None
    with open(versionFile, 'r') as file:
        filedata = file.read()
        matches = re.finditer(REG_VERSION_LINE, filedata)
        for matchNum, match in enumerate(matches):
            oldVersion = match.group(1)
            newVersion = str(int(match.group(1)) + 1)
            print oldVersion
            print newVersion
            if len(oldVersion) > 0 and len(newVersion) > 0:
                filedata = filedata.replace(oldVersion, newVersion)

    # Write the file out again
    with open(versionFile, 'w') as file:
        file.write(filedata)
  

什麼是 closed-form solution and numerical solution

最近正在學解決multivariate analysis和optimalization問題, 遇到一個英文名詞close-form solution. close-form solution中文文獻稱閉合解. 白來來說就是給予一些觀察到的資料, 然後問題可以用函數和數學運算來表示, 這樣的方程式就是close-form solution or numerical solution, 差別在於一個是exact, 一個是approximate. 舉例來說, 在Linear regression中的 Least square equation就是close-form solution, Non-linear regression是numerical solution.

[Interview type questions] Task sequence arragnement

Given a task sequence and the cool down time(k), rearrange the task sequence such that the execution time is minimal.

Solution:

 public static void main(String[] args) {
  char[] task = findBestTaskArrangement(String.valueOf("AAABBB").toCharArray(), 2);
  System.out.println(task);
  System.out.println("Total time: " + computeTatalTaskTime(task, 2));
  // ABABAB
  // Total time:8
 }
 
 public static char[] findBestTaskArrangement(char[] tasks, int k) {
  int n = tasks.length;
  Map<Character, Integer> map = new HashMap<>();
  for (char task : tasks) {
   map.put(task, map.getOrDefault(task, 0) + 1);
  }

  PriorityQueue<Task> queue = new PriorityQueue<>(new Comparator<Task>() {

   @Override
   public int compare(Task a, Task b) {
    return b.frequency - a.frequency;
   }
  });

  for (Map.Entry<Character, Integer> entry : map.entrySet()) {
   queue.offer(new Task(entry.getKey(), entry.getValue()));
  }
  
  tasks = new char[n];
  int i = 0;
  while (!queue.isEmpty()) {
   int c = 0;
   List<Task> nextRoundTask = new ArrayList<>();
   while(c++ < k && !queue.isEmpty()) {
    Task task = queue.poll();
    task.frequency--;
    // Locate the next empty slot
    tasks[i++] = task.id;
    if (task.frequency > 0)
     nextRoundTask.add(task);
   }
   for (Task task : nextRoundTask) {
    queue.offer(task);
   }
  }

  return tasks;
 }

 // helper class
 public static class Task {
  char id;
  int frequency;

  Task(char i, int f) {
   id = i;
   frequency = f;
  }
 }

 public static int computeTatalTaskTime(char[] tasks, int k) {
  HashMap<Character, Integer> taskLastTimeMap = new HashMap<>();
  int currTime = 0;
  for (char task : tasks) {
   if (taskLastTimeMap.containsKey(task)) {
    int exceptedTime = taskLastTimeMap.get(task) + k + 1;
    if (exceptedTime > currTime) {
     currTime = exceptedTime;
    } else
     currTime++;
   } else {
    currTime++;
   }
   taskLastTimeMap.put(task, currTime);
  }
  return currTime;
 }

[Interview type questions] MinQueue

Leetcode had a question about min stack, but there is no question about min queue. In this article, I wrote a simple one for min queue.

Solution:

 public static void main(String[] args) {
  MinQueue queue = new MinQueue();
  // Input 3 -> 1 -> 4 -> 2
  queue.offer(3).offer(1).offer(4).offer(2);
  //  0 0 0 1 1 1
  while(queue.isEmpty() == false) {
   System.out.println(queue.getMin());
   queue.poll();
  }
  // Output should be 1 -> 1 -> 2 -> 2
 }
 
 static class MinQueue {
  private Queue<Integer> mQueue = new LinkedList<>();
  private Deque<Integer> mMinDeque = new ArrayDeque<>();
  
  MinQueue offer(int val) {
   int min = val;
   while(!mMinDeque.isEmpty() && mMinDeque.getLast() > val) {
    mMinDeque.pollLast();
    min = val;
   }
   int size = mQueue.size() - mMinDeque.size();
   for (int i = 0; i <= size; i++) {
    mMinDeque.addLast(min);
   }
   mQueue.offer(val);
   return this;
  }
  
  int poll() {
   mMinDeque.poll();
   return mQueue.poll();
  }
  
  boolean isEmpty() {
   return mQueue.isEmpty();
  }
  
  int getMin() {
   return mMinDeque.peek();
  }
 }       

Given a million points (x, y), give an O(n) solution to find the k's points closest to (0, 0).

Use quick select/partial sorting to resolve.
Solution:

 static class Point {
  int x;
  int y;

  public Point(int x, int y) {
   this.x = x;
   this.y = y;
  }

  public double getDistFromCenter() {
   return Math.sqrt(x * x + y * y);
  }
 }

 public static void main(String[] args) {
  Point[] points = new Point[7];
  points[0] = new Point(0, 0);
  points[1] = new Point(1, 7);
  points[2] = new Point(2, 2);
  points[3] = new Point(2, 2);
  points[4] = new Point(3, 2);
  points[5] = new Point(1, 4);
  points[6] = new Point(1, 1);
  int k = 3;
  qSelect(points, k - 1);
  for (int i = 0; i < k; i++) {
   System.out.println("" + points[i].x + "," + points[i].y);
  }
  // Output will be
  //        0,0
  //        1,1
  //        2,2
 }

 // in-place qselect and zero-based
 static void qSelect(Point[] points, int k) {
  int l = 0;
  int h = points.length - 1;
  while (l <= h) {
   int partionInd = partition(l, h, points);
   if (partionInd == k) {
    return;
   } else if (partionInd < k) {
    l = partionInd + 1;
   } else {
    h = partionInd - 1;
   }
  }
 }

 static int partition(int l, int h, Point[] points) {
  // Random can be better
  // int p = l + new Random.nextInt(h - l + 1);
  int p = l + (h - l) / 2;
  int ind = l;
  swap(p, h, points);
  Point comparePoint = points[h];
  for (int i = l; i < h; i++) {
   if (points[i].getDistFromCenter() < comparePoint.getDistFromCenter()) {
    swap(i, ind, points);
    ind++;
   }
  }
  swap(ind, h, points);
  return ind;
 }

 static void swap(int i, int j, Point[] points) {
  Point temp = points[i];
  points[i] = points[j];
  points[j] = temp;
 }