README.md

Self-adaptive-Feature-tracking-with-PCA-Lucas-Kanade

The inability to change size has always been a drawback of sliding window tracking. If the previous frame of the current frame is used as the reference frame, the error rate is often superimposed. Therefore, this is still not the best solution. If only using traditional feature tracking methods such as SIFT, SURF or Lucas-Kanade, it's impossible to track a specific object and there is no defined object frame to define the overall features of the object to be tracked. Using Deep Learning (DL) for object tracking such as Siamese Tracker requires training of the object to be tracked, and the size of the tracking bounding box cannot be defined arbitrarily while tracking. We propose to use Principal Component Analysis (PCA) as the feature extraction mechanism and Lucas-Kanade (LK) tracking optical flow as the object size prediction:

1. No time-consuming DL training is required for the objects, every object is trackable.
2. The object frame size can be defined arbitrarily.
3. Automatically detects and adjusts object size even if it changes.

This repostory is based on my previous repostory Feature-tracking-with-PCA. If you are insterented how to track based on PCA, please check the detail from there.

Figure 1: Visualization tracking result. Only tracked with PCA, i.e. Feature-tracking-with-PCA,(left). This method (middle). Optical flow that generated by Lucas-Kanade algorithm used to assist tracking in this repostory (right).

*Figure 2: * Pipeline of our feature tracking algorithm.

Figure 3: Parameters compariosn before and after self-adaptation. Minimum SSR (top). Width and Height of tracked window (bottom).