Share

How Artificial Intelligence is Changing Video Editing

Artificial Intelligence (AI) has made some huge strides in machine learning and neural networks over the last few years, and it is starting to feature more prominently in the form of chat bots, virtual assistants, and various other applications. However aside from the obvious direct use of AI, it also has a role to play in other areas – and video editing in particular is one application where AI could have a very major part indeed.

The Watson Story

Possibly the most compelling story of the usage of AI in video editing is that IBM’s Watson supercomputer created a trailer that was used to promote a new horror film called Morgan. At first glance the implications of an AI-created trailer may seem huge, but it is important to note that this story was largely a marketing stunt.

The real story behind the ‘AI trailer’ for Morgan is much more grounded in reality: The team at IBM essentially showed Watson a large collection of horror trailers that it analyzed and studied, then they showed it the Morgan movie and let it identify scenes that should be used in the trailer.

Ultimately it was a real person who edited the final scenes and created the trailer, but using input from Watson’s AI.

As much as that might seem to detract from the story – it is still an achievement and an illustration of how AI is already being used in video editing. In this case Watson was responsible for combing through footage and selecting scenes of interest, thus cutting down the workload of the editor who ultimately pieced those scenes together into a trailer.

AI-Assisted Video Editing is Already Here

Aside from the famous Watson story, there are a lot of other ways in which AI-assisted video editing has been incorporated into video editors and other apps. One of the most recognizable (albeit basic) is Snapchat, which uses AI to carry out its facial recognition that is a key part of their filters.

Similarly Apple’s latest video app entitled Clips has a decent amount of AI baked in. Not only does it utilize facial recognition, but its real time filters also use them heavily as does its ‘Live Titles’ feature that uses speech recognition to insert captions into videos.

While these may seem like relatively basic uses of AI to capture video and edit it, it is still early days yet. Recently Adobe unveiled a sneak peek of its own foray into AI where it utilized machine learning to improve selfies. Although it dealt with images and not videos, the ability of Adobe’s AI to study images and mimic their qualities is something that could eventually be adapted to video as well.

Fully AI-Edited Videos On the Horizon

Aside from simply assisting real people edit videos, AI could conceivably perform the entire job – from start to finish. Some platforms such as Shred and Magisto already do so – though with varying degrees of success.

Admittedly little is known about Shred’s use of AI, but Magisto is a bit more forthcoming. It analyzes the video and audio and attempts to piece together a story by recognizing faces, speech, movement, and other factors. Once it has an outline, it then selects a music track and adjusts the scenes to match it.

In the future it is likely that more video editors may include this type of functionality, to allow the quick creation of videos that are entirely AI-edited. Of course it may not be able to compete creatively with the work of a professional video editor – but it will make video creation more accessible to those without that skillset.

Written by Louis Pasture

Comment this news or article