Video Analysis: Heat maps and Motion Detection

This experiment was all about video analysis. The idea and goal behind the experiment was to get more experience with image and video.

During the experiment I build myself a video crawler, which crawls over the video and does some kind of analysis on the way. The analysis part of the crawler has a flexible API to enable for example artificial intelligence interfaces.

The software tracks motion in the video and draws cumulative and running heat maps of the movement:

  • Running: previous 40 frames
  • Cumulative: cumulative up to that point.

Different kind of movement becomes visible with different set of parameters. Parameters have to be different for example if one wants to detect fast or slow motion. You might also want to change a noise parameter, which sets a threshold for significant motion. Insignificant motion is just ignored.

Let’s look at an example analysis of a video clip. The clip was shot in the finnish Bommari and it shows me walking past a climbing wall. Exciting indeed 😀

Figure 1 is the background for the video. Please, notice the smiley face on the left hand side.


Figure 1. Video background.

May the walk begin. In figure 2 the video has run for 300 frames and I have entered the figure. Most of the heat is located at the lower part of the heat map. This makes sense because there are two (relatively small) feet moving over the background. Figures 3 and 4 are showing the running heat maps at frames 330 and 395.


Figure 2. Running heat map at frame 300.


Figure 3. Running heat map at frame 330.


Figure 4. Running heat map at frame 395.

Figure 5 shows the cumulative heat map at frame 485. Running heat maps were mapping a narrow time window, cumulative heat maps on the other hand give a better overview of the motion.

Figure 5. Cumulative heat map at frame 485.

When you look at figure 5 you can see that there are areas that did not detect any motion. For example the area under the smiley face (left) is completely blue, the same with the spots at (600, 500), (300, 900) and (450, 1100). Why is that? Take a look at the video background (figure 1). These areas in the background seem to be dark. I was wearing dark clothes so the motion was not detected in these areas.