Synergizing appearance and motion with low rank representation for vehicle counting and traffic flow analysis

Appearance and motion, which are complementary, account for a dominant proportion of visual information. We propose to synergize them using a low-rank representation framework for the estimation and analysis of traffic flow. Taking advantage of the downward-looking camera configuration, we do the processing only on the measure line, called virtual gantry, instead of dealing with the whole frame, resulting in much improved efficiency. Enforcing the low-rank constraint on the spatiotemporal image which is generated via stacking pixels on virtual gantry over time, we introduce the block-sparse robust principal component analysis algorithm, in which the motion cue is leveraged to highlight the foreground and realize vehicle detection with high accuracy. The motion flow is further exploited for size normalization to classify vehicles into lite, small, medium, and large categories. Benefiting from the low-rank representation, our method is parameter insensitive, robust to illumination changes, and requires no training. We perform extensive experiments on the 24/7 videos collected over the highways in China and Singapore, obtaining nearly 100% accuracy. Meanwhile, insightful observations on the obtained traffic information are given, which could be very valuable to the users, especially to the traffic management sectors.