A.I.s erzählen Geschichten aus Fotoserien, generieren Bilder aus Text und erzeugen eine Stimme

Gepostet vor 9 Monaten, 21 Tagen in #Misc #Tech #AI #AlgoCulture

Share: Twitter Facebook Mail

Drei hübsche neue AI-Bits, alle via CreativeAI:

1. ein Neural Network, das Stories aus Bilderserien generiert (Paper: Storytelling of Photo Stream with Bidirectional Multi-thread Recurrent Neural Network). Zur Erinnerung: Vor zwei Monaten waren wir noch bei (ziemlich guten) Beschreibungen von Einzelbildern.

ai1 ai2

Visual storytelling aims to generate human-level narrative language (i.e., a natural paragraph with multiple sentences) from a photo streams. A typical photo story consists of a global timeline with multi-thread local storylines, where each storyline occurs in one different scene. Such complex structure leads to large content gaps at scene transitions between consecutive photos. Most existing image/video captioning methods can only achieve limited performance, because the units in traditional recurrent neural networks (RNN) tend to "forget" the previous state when the visual sequence is inconsistent. In this paper, we propose a novel visual storytelling approach with Bidirectional Multi-thread Recurrent Neural Network (BMRNN). First, based on the mined local storylines, a skip gated recurrent unit (sGRU) with delay control is proposed to maintain longer range visual information. Second, by using sGRU as basic units, the BMRNN is trained to align the local storylines into the global sequential timeline. Third, a new training scheme with a storyline-constrained objective function is proposed by jointly considering both global and local matches. Experiments on three standard storytelling datasets show that the BMRNN model outperforms the state-of-the-art methods.

2. So ungefähr das Gegenteil: Ein Neural Network generiert Bilder aus Textbeschreibungen (Paper: Generative Adversarial Text to Image Synthesis). AI-generierte „Fotos“ sind nach wie vor ziemlich LoRes, dürfte noch ein bisschen dauern, bis da wirklich praxistaugliche Ergebnisse bei rauskommen. Es braucht allerdings nicht wenig Vorstellungskraft, um sich vorzustellen, wozu solche Algorithmen in ein paar Jahren in der Lage sind, zum Beispiel „diktierte“ Comics im Stil von Jack Kirby oder „Ey Siri, generier mir mal ’ne Steppenwolf-Verfilmung mit Wolverine in der Hauptrolle.“

ai3 ai4 ai5

Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal. However, in recent years generic and powerful recurrent neural network architectures have been developed to learn discriminative text feature representations. Meanwhile, deep convolutional generative adversarial networks (GANs) have begun to generate highly compelling images of specific categories such as faces, album covers, room interiors etc. In this work, we develop a novel deep architecture and GAN formulation to effectively bridge these advances in text and image modeling, translating visual concepts from characters to pixels. We demonstrate the capability of our model to generate plausible images of birds and flowers from detailed text descriptions.

3. Ein Neural Network generiert eine Stimme. Hierzu einfach ’nen Kurzfilm von Chris Cunningham vorstellen, am besten Rubber Johnny:

This is a recursive neural network (LSTM type) with 680 neurons and 3 layers trying to find patterns in audio and reproduce them as well as it can. It's not a particularly big network considering the complexity and size of the data, mostly due to computing constraints, which makes me even more impressed with what it managed to do.

The audio that the network was learning from is voice actress Kanematsu Yuka voicing Hinata from Pure Pure. I used 11025 Hz, 8-bit audio because sound files get big quickly, at least compared to text files - 10 minutes already runs to 6.29MB, while that much plain text would take weeks or months for a human to read.

I was using the program "torch-rnn" (https://github.com/jcjohnson/torch-rnn/), which is actually designed to learn from and generate plain text. I wrote a program that converts any data into UTF-8 text and vice-versa, and to my excitement, torch-rnn happily processed that text as if there was nothing unusual.

Next Level, photorealistic Style-Transfer

In ihrem neuen Paper stellen Fujun Luan, Sylvain Paris, Eli Shechtman und Kavita Bala eine neue Style-Transfer-Methode vor: Figure 1:…

Blacked Out Censorship-Poetry Generator

Schöne Spielerei von Max Kreminski, ein JS-Droplet, das Websites in Blacked Out Poetry verwandelt, basierend auf Liza Dalys █ Blackout…

This Bot kills Fascists

So-So-Working-Object-Detection-Algorithm + Woodie Guthrie = Fascists.exe | „This bot kills fascists“.

„Alexa? Are you connected to the CIA?“

„I always try to tell the truth.“ This reminds me of those Guilty-Dog-Videos:

Palm Generator

Es sollte mehr entspannte Algorithmen mit Urlaubs-Attitüde geben wie den hier: „The Palm Generator is a Three.js module to create…

AI Brainscans

Graphcore aus Bristol visualisieren künstliche Intelligenzen und Neural Networks: Inside an AI 'brain' - What does machine learning look like?…

A Banana Keytar and more from Stupid Hackathon: Inverted Eyetracker-Pong, Robot Porn Addict or the Shitty Sharpie Tattoo Gun)

Ein weiterer Fav vom Stupid Hackathon NYC 2017, die Banana-Keytar von Amanda Lange. Auch geil: der Twitter-Bot Robot Porn Addiction,…

Fotorealistische Pics aus der Gameboy-Camera

Roland Meertens generiert fotorealistische Farbbilder aus den Pics der Gameboy-Camera: Creating photorealistic images with neural networks and a Gameboy Camera.…

Pix2Pix: Neural Network Katzen-Compositing als Browsertoy

Schönes Spielzeug von Christopher Hesse, der ein Neural Network auf Bild-Paaren trainiert hat und mit dem man nun Katzen, Schuhe…

Automatic Handgun Detection via Machine Learning

The latest Step into an OCP-approved Ed-209-compatible Future: Automatic Handgun Detection Alarm in Videos Using Deep Learning (PDF). Usage Guide:…

ALF-Trump and other algorithmic Abominations

Großartiger neuer Twitter-Feed von Chris Rodley: Algorithmic Horror – Concept art for horror movies generated by an algorithm mit so…