MIT develops AI technology for drones to maintain framing while filming movie sequences

Movies and TV Series have seen a surge of aerial tracking shots using drones. When such sequences are filmed, there are usually two people operating the drone at the same time. The drone and the camera each require a separate operator, and the shots have to be carefully choreographed to avoid collisions. A team of researchers from MIT and ETH Zurich have developed a new technology that makes filming from drones safer, simpler and more reliable.

Daniela Rus, a professor at MIT and lead author of the new paper says, "There are other efforts to do autonomous filming with one drone. They can follow someone, but if you turn 180 degrees, then you will show your back to the drone. This is a much coarser approach than what we are able to do. With our solution, if you turn 180 degrees, our drones are able to circle around and get back to your face. What we are able to do is richer and offers more ways to describe how you would like the scene."

The director specifies how much of the screen a face should occupy, whether it should be a profile, a straight on, over the shoulder, or a three quarter view from either side. All of these parameters can be set for any number of objects to be tracked. The director can specify how each of these parameters should be weighted against any other, for example, maintaining the right distance might be more important than the location of the object on screen. A weight can also be given to minimise occlusion, so that one actor is not blocked from view by another. The drone does the rest of the work on its own.

The system can also guarantee that the drone will not collide into any moving or stationary objects in the environment, as long as it has complete information about the surroundings. The drones were tested with subjects trying to actively collide with the drone by walking briskly towards it, but the drones avoided the collisions and still tried to maintain the framing. The technical wizardry here is that the drone calculates the position of all moving objects in the scene two seconds into the future, and adapts its flight path accordingly.

Published Date: May 18, 2017 06:49 pm | Updated Date: May 18, 2017 06:49 pm