Algorithm watching Wolf of Wallstreet

wolf

Tolles Computervision-Experiment von Støj: An algorithm watching a movie trailer: „A program removing everything but the objects it recognises when watching the trailer for The Wolf of Wall Street.“ Vor drei Jahren gab es schonmal ein ähnliches Experiment von Benjamin Grosser, der damals allerdings „nur“ den visuellen Eyetracking-Fixpunkt der Software nachgezeichnet hatte. (via Prosthetic Knowledge)

Hier jedenfalls Wolf of Wallstreet durch die Augen eines Object Detection Algorithms:

An algorithm watching a movie trailer is a project by Lasse Korsgaard and Andreas Refsgaard from the creative coding studio Støj, that tries to explore how an object detection algorithm experiences a movie trailer.

Object detection is the process of identifying specific objects such as persons, cars and chairs in digital images or video. For most humans this task requires little effort regardless of how the objects may vary in different sizes, scales and rotations or whether they are partially obstructed from view. For long these tasks have been difficult for computers to solve, but recent developments have shown impressive improvements in accuracy and speed, even while detecting multiple objects in the same image.

We wondered what a fast paced movie trailer would look like seen through the lens of an object detection algorithm. Therefore we sent all individual frames from the movie trailer for The Wolf of Wall Street are through Yolo-2 with a threshold of 0.15, meaning that it only reacts to objects detected with a confidence of 15% or higher.