CNNs in Video analysis - An overview, biased to fast methods 

Speaker: Michael Gygli, Head of AI @ gifs.com, / gyglimichael
Slides: goo.gl/FFKdqN
Presentation abstract:
In this presentation I will give an overview of automatic video analysis using Convolutional Neural Networks (CNN) and present recent advances. Thereby the focus will lie on fast algorithms that can be used in production systems. The talk consists of three parts.
First, I will discuss C3D, a spatio-temporal neural network that is widely used for video analysis tasks such as action recognition. In comparison to competing approaches, C3D directly operates on raw pixel inputs, allowing it to run at close to 400 FPS on a modern GPU. Then, I will present my recent method for shot boundary detection with fully convolutional CNNs. It’s model architecture is similar to C3D, but more compact and fully convolutional in time. Thanks to these changes, the shot detection runs at more than 120x-real-time speed, thus it can analyze full-length movies in less than a minute.
Finally, the presentation finishes with our approach to automatically find highlights in videos. Our system first detects shots, which are then scored by a combination of C3D, audio features and a feed-forward neural network (FNN).
References:
Learning Spatiotemporal Features with 3D Convolutional Networks (C3D)(arxiv.org/abs/...)
Ridiculously Fast Shot Boundary Detection with Fully Convolutional Neural Networks (arxiv.org/abs/...)
Video2GIF: Automatic Generation of Animated GIFs from Video (arxiv.org/abs/...)