Google can recognize elements and actions in a video with artificial intelligence

Essentially the Cloud Video Intelligence API allows users to search through videos in the same way as they search through text.


Google has announced the Cloud Video Intelligence API, which is currently in private beta that allows users to search through videos based on their content. Users can specify objects, such as "elephant" or "palm trees" as well as actions such as "smiling" or "running". The Api can even find specific shots within a video where the searched for content appears. Third party developers can integrate the capabilities into the next generation of applications.

Google's artificial intelligence breaks down the video into shots, and provides an analysis of the content of each shot along with a probability score. The probability score is a percentage that represents how confident the machine intelligence is in recognising the elements in a shot. The recognition fails at recognising objects at times, elephants can be recognised as cattle, tigers as cats, and animated characters can be interpreted as stuffed toys. However, the machine intelligence seems to be able to use its models to recognise aspects of scenes that humans may not be able to guess, such as if a shot of an animal is taken in the wild or in a zoo.

tiger-zoo-google

Mikael Wahlberg, VP Product Development at Cantemo, one of the early clients of Google to use the Cloud Video Intelligence API says "Thanks to the Google Cloud Video Intelligence API, we have been able to very quickly process and understand the content of video down to the individual frame, with an impressively rich taxonomy. This will revolutionize media management and take it to the next generation."

Essentially the Cloud Video Intelligence API allows users to search through videos in the same way as they search through text. The API is in the private beta stage, which means Google will be taking feedback from the early users to improve the product. The API uses deep learning models based on the TensorFlow and similar frameworks, and applied on large scale video platforms such as YouTube.


Find latest and upcoming tech gadgets online on Tech2 Gadgets. Get technology news, gadgets reviews & ratings. Popular gadgets including laptop, tablet and mobile specifications, features, prices, comparison.