Text 2 Bird

b00a

Neural Networks generieren Vögel aus Textbeschreibungen in 256x256 Pixeln (das nennen AI-Forscher „High Res“, haha). Die Methode sah qualitätsmäßig vor ein paar Monaten noch weitaus schlimmer aus und beschränkte sich oft auf 64x64 Pixel. Wir sind zwar noch ein paar Schritte entfernt von „Siri, generate a Böhmermann in blau and Trump-Hair and make him sing 'Nazi Punks Fuck Off'“, aber das zeichnet sich definitiv bereits am futuristischen Horizont ab. Wenn dann die Fake-Videos mit jedem und allem Youtube flooden, können wir ja nochmal über „Fake-News“ reden. We ain't seen nothing yet.

Paper: StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks (via Procedural Generation)

b1dAbstract:
Synthesizing photo-realistic images from text descriptions is a challenging problem in computer vision and has many practical applications. Samples generated by existing text-to-image approaches can roughly reflect the meaning of the given descriptions, but they fail to contain necessary details and vivid object parts. In this paper, we propose stacked Generative Adversarial Networks (StackGAN) to generate photo-realistic images conditioned on text descriptions.

The Stage-I GAN sketches the primitive shape and basic colors of the object based on the given text description, yielding Stage-I low resolution images. The Stage-II GAN takes Stage-I results and text descriptions as inputs, and generates high resolution images with photorealistic details. The Stage-II GAN is able to rectify defects and add compelling details with the refinement process.

Samples generated by StackGAN are more plausible than those generated by existing approaches. Importantly, our StackGAN for the first time generates realistic 256 × 256 images conditioned on only text descriptions, while state-of-the-art methods can generate at most 128 × 128 images. To demonstrate the effectiveness of the proposed StackGAN, extensive experiments are conducted on
CUB and Oxford-102 datasets, which contain enough object appearance variations and are widely-used for text-toimage generation analysis.

b0

b1

b1a

b1b

b1c

b2

b3