This assignment is designed to excercise your understanding of David Bolme's work on Correlation Filters. The goal is less about a general implementation than it is about a working demonstration. So, for example, in this assignment you will be implementing ASEF rather than MOSSE. Therefore, while you may want to experiment with tracking (see below), the primary reason for doing this assignment is to build code that will construct and display the correlation filters that develop while tracking a known object given ground-truth knowledge of the object position.
Given any arbitrary video there is a startup issue and also a ground-truth issue that must be addressed when implementing a correlation tracker. Namely, where is the object of interest in frame 1 and then in subsequent frames. In the YouTube videos of David Bolme's tracker startup was based upon a rectangle drawn by a person and then updates in the first few frames were based upon a best match using a still reltaively untrained correlation filter. That means there were details involved that we've not fully discussed to get the tracker off to a solid start.
So, this assignment is not primarily about building at tracker. Instead, you are to use a video sequence where tracking is easily accomplished using a simple algorithm. Namely, the frigate bird video from assignment 1. What you are to do in this assignment is implement the ASEF algorithm and show vidually the developing correlation filter as the frigate bird video progresses.
When your program is run four disctinct windows will be displayed and these will update for each successive frame.
The hardest part of this assignment will be implementing the quotient operation in the frequency domain. When in doubt seek help from the instructor and others in the class to make sure that your code is a faithful rendition of the algebra presented in lecture.
Along the way, you will probably want to display and inspect the FFT of the current frame, the target correlation surface, and the result of dividing the one by the other.
Here is one incredibly minor by helpful detail, the
moveWindow
command in openCV is useful for laying out your four windows in such
a way that each is readily seen as the video plays.
Find a single additional video where you can also demonstrate the development of ASEF filters. By default, find some other object easily tracked using a simple feature so that you can generate the goal image for each frame. That is sufficient for this assignment. However, if you choose, you may want to try a video where after some number of frames (16, 32, etc.) your system starts using the correlation filter from frame k to first identifiy the most likely position of the object in fram k+1 and build the next goal image accordinly. In other words, you may wish to build a tracker.
You are expected to work in pairs - teams - for this project. Unless worked out between yourselves and approved by me in advance, I strongly suggest you keep your partner from Assignment 2. The grading will be done in the same fashion as for Assignment 2, in face-to-face interviews with the instructor.