Adobe recently worked together with the Beckman Institute for Advanced Science and Technology to develop a new background extraction system that would dismiss the old green screens of Hollywood filmmaking. Driven by Artificial Intelligence research, the new system works more accurately and intelligently, without any need for a green screen.
Background extraction techniques
Adobe’s newly developed technology for background extraction is the result of a machine learning algorithm that, according to the researchers’ papers, demonstrated “the superiority” of their algorithm over previous methods.
Their technique simply works by doing what it aims to do: extracting silhouettes from images or videos. No matter the background, the deep-learning-based algorithm was able to perform well where previous algorithms have flunked. For example, on images with similar foreground and background colors or those with complicated textures.
With older techniques, using a bright green or blue screen was inevitable. The current film editing software requires the actors to stand in front of a monotone backdrop to be able to identify areas to fill. Everything within the frame will be replaced with a different image, except the subject. A lot of films are made with actors doing their actions in front of a blank monotone screens that are later replaced with props from an ancient war, like in Peter Jackson’s epic “The Lord of the Rings Trilogy.” A hefty part of the trilogy’s $297 million budgeting could go to Green Screen Technology, or also known as chroma key.
‘Deep image matting.'
Adobe has been a leading provider of visual effects for both the film industry and photography. The team developed Photoshop, a photo editing software, and After Effects for video effects and animation. This new "deep image matting," however, has the potential to be used not only in the film industry but also the far-reaching ends of Virtual Reality/Augmented Reality.
An Adobe Research paper, titled “Deep Image Matting,” revealed that the picture matting algorithm was capable of performing the seemingly impossible task. Using a data set of 49,300 training images and 1,000 testing images, the deep image matting experiments outperformed its predecessors. The method even went as far as excelling in real outdoor backgrounds with all sorts of lighting situations.
