Facebook demonstrates Prisma like photo processing for live video

Facebook demonstrated Prisma like photo filters for live video in a Live panel discussion hosted by the Wall Street Journal. Prisma is a photo enhancement application that turns every day images into artworks in the styles of famous painters using artificial intelligence and neural networks. Prisma was based on already existing open source technology, the brilliance was in making it as fast and seamless as slapping on an Instagram filter to a photo. [caption id=“attachment_343491” align=“aligncenter” width=“640”]  Image: Facebook /WSJ[/caption] The Prisma CEO Alexey Moiseenkov has tried out a Facebook live with co-operation with Facebook, that added prismatic effects in real time . He did not comment on any plans by Facebook to acquire Prisma. In the Wall Street Journal panel discussion, Facebook demonstrated a prototype application to show the direction they want to go head in while implementing augmented reality tools to enhance video streams. [caption id=“attachment_343492” align=“aligncenter” width=“640”]  Image: Facebook /WSJ[/caption] The art filters are similar to Prisma in the sense that instead of slapping on filters to the images, the image is redrawn using artificial intelligence. The idea here is that machines are made to learn the styles of famous painters, and then the machines redraws images in the same style as the artists. Facebook has not developed its own technology for this. The image processing is based on a paper called style transfer, which was in turn based on a paper called A Neural Algorithm of Artistic Style. Convolutional neural nets and computer vision together take an image and turn it into something that looks like The Starry Night. [caption id=“attachment_343499” align=“aligncenter” width=“640”]  Van Ghogh’s The Starry Night[/caption] The filters are meant to be a continuation of what Instagram did. Instagram filters made everyday photos look better, and as an increasing number of people across Facebook platforms are using videos, real time enhancements of videos is what Facebook is developing technologies for. This will not only help publishers and broadcasters, but also regular users, and citizen journalists. Filters in the style of Georgia O’Keeffe, Vincent van Gogh and the rotoscoped movie Waking Life. [caption id=“attachment_343494” align=“aligncenter” width=“640”]  Image: Facebook /WSJ[/caption] Like Prisma, Facebook also takes existing technology and makes it faster. However, this is much faster than what Prisma could achieve as the processing is done at twenty four frames per second, in real time. Facebook engineers have made the prototype smooth, as in there are no dropped frames, stuttering or blurring during motion. This is a big achievement in terms of pushing the limits of realtime image processing. [caption id=“attachment_324634” align=“aligncenter” width=“640”]  Various Prisma filters[/caption] The application showcased was just a technical demonstration to show the capabilities of Facebook in terms of implementing artificial intelligence and realtime video processing. There is no app or product that is planned at this stage. The filters or similar technologies may make it to Instagram, Facebook or Messenger in the future. The idea here is to convert the cameras in phones into powerful, creative tools for the users. You can watch the entire demonstration here. Skip to 13:00 for the real time Prisma style filters bit.

However, a real test of a real time artificial intelligence based image processing capability will be seen when users in the wild can go live with the filters. Prisma takes a variable amount of time to process the images and the service shuts off under heavy load. Android users can now apply the image effects on their phones itself, a feature introduced when Prisma added filters specifically oriented for India, including  Thota Vaikuntam and Horn Ok Please .