Abstract Art with ML

Published July 1, 2018

Did you know that randomly initialised neural networks actually produce pretty cool pictures?

Well, I didn’t, until I recently discovered pattern producing networks. This post will be all about them, and will contribute a pattern generator running in your browser, so you can experience these networks for yourself. But first, let me explain.

Table of Contents

Compositional Pattern Producing Networks

A compositional pattern producing network, or CPPN in short, is a network, in this case we will mainly focus on neural nets, that given some parameters produces a visual pattern. The main idea here is to slide the network over all the x‑y-coordinates in an image and produce a 3‑channel (rgb) color output for each pixel in the output image. We can look at this network ff in the following way:

[rgb]=f(x,  y)\begin{bmatrix}r\\g\\b \end{bmatrix} = f\left(x, \; y \right)

This network, since it’s continuous and differentiable, outputs locally correlated values, meaning, sampling the network on two different very close points will lead to a very similiar output value. This basically results in the property that the image we can generate from it, we could call smooth.

Another cool property this has, is that it has ​“infinite” resolution, because you can just scale the coordinates the network receives as inputs.

Example of an image generated using a CPPN
Figure 1: One example of a compositional pattern producing network, using a simple 3‑layer network with tanh as the activation function.

Parameters

Now we could simply run the network as it is. And this in fact works. But we take this a little step further by adding certain other inputs to the network with the aim of having the network generate more complex images.

For example we can add the radius rr and an adjustable parameter α\alpha. With these modifications our network ff can be described like the following.

[rgb]=f(x,  y,  r,  α)\begin{bmatrix}r\\g\\b \end{bmatrix} = f\left(x, \; y, \; r, \; \alpha \right)

with r=x2+y2r = \sqrt{x^2 + y^2}. The radius not only provides a nice non-linearity but it also enables the network to correlate the output color to the distance to the origin, because points on the same circumference receive the same values for rr.

While the radius rr changes with xx and yy, the α\alpha parameter is static over the course of the image. In essence, you can think of this parameter as the z‑parameter. When sampling from the 3‑dimensional cube with parameters [x  y  z]T\left[ x \; y \; z \right]^T, we look at the slice at position z=αz = \alpha.

You can get very creative with these parameters and we’ll explore more exotic configurations later on.

A image rendered using a 9-layer DenseNet

Figure 2: What about a 9‑layer DenseNet? well see for yourself.

Initialisation

The output of a neural network is defined a) by its inputs which we talked about in the last section and b) by its weights. The weights therefore play a crucial role in how the network behaves and thus what the output image will look like.

In the example images throughout this post, I mainly sampled the weights θRm\boldsymbol\theta \in \mathbb{R}^m from a Gaussian distribution N\mathcal{N} with a mean of zero and with a variance dependent on the number of input neurons and a parameter β\beta which I could adjust to my taste.

θiN(0,  βNin)\theta_i \sim \mathcal{N}\left(0, \;\frac{\beta}{N_{\mathrm{in}}}\right)
Black and white art

Figure 3: We can also ask the network to just spit a single value out, and interpret that as a black and white image.

You Generate Your Own Art

Now to the fun part. Here is a progressive image generator based on the principle of CPPNs. You can adjust the Z‑value and β\beta, choose if you want a black and white image and explore the seed space. Note, that the random number generator used for the Gaussian distribution will always produce the same values for the same seed, which is cool, because you can share the seed and retrieve the same result.

Figure 4: Interactive art generator.

I compiled a list of example seeds that I found quite compelling while I played with the tool. Try these seeds for the start and then explore the space yourself! :)

And these are just some examples. Try changing the seed, the time or the variance yourself and maybe, just maybe, you come across some masterpiece! 😛

Figure 5: Example seeds. Click on an image to process it in the generator.

Exploring Exotic Architectures

In this section, I want to show you some results I’ve been getting with some more exotic architectures.

The swirl 1
The swirl 2

Figure 6: Parameterising images on β=cos(10r)\beta = \mathrm{cos}\left(10r\right)

As you can see, these images behave very differently. A single parameter more makes a big difference in the activations of the networks.

Reactor 1
Reactor 2

Figure 7: Symmetry using f(x2,y2,α)f\left(x^2, y^2, \alpha\right)

Wrapping up

Now that we have gone through several architectures, explored multiple configurations and looked at a bunch of images, it’s time to wrap up. But I want to wrap up by pointing to some possible improvements.

  • Train these networks using backpropagation. That’s the obvious one. What’s not obvious, is on what to train it.
  • As a follow up to training: one way to supercharge this method of creating art would be to incorporate human feedback e.g. using adversarial networks. For instance, humans could be shown two images which they have to choose the one they prefer from. Then, an adversarial network could learn the probability that a given image is chosen by a human, and use the gradient of this adversarial network for backpropagation on the generator network.
  • Also, there are a lot of things I didn’t try. Therefore, one could explore even more of the architecture and parameter space. Changing bias initialisation, kernel initialisation, using different network topologies, or even trying different color spaces.

In terms of use case: these images make great color and gradient inspiration! Also, I just recently replaced my Spotify playlist covers with these. They make pretty great album artworks (don’t they?).

Anyway, that’s it for now. I hope you have enjoyed this blog post. If you did, I’d appreciate if you consider subscribing to my blog via RSS or JSON-Feed. I’ll try to publish blog posts regulary here. See you then!


I pub­lished the code used to gen­er­ate the images in this post as well as the source code for Figure 4 on GitHub. As men­tioned, the Python code is based on 1 and runs using Ten­sor­flow. You can find the repos­i­to­ry here.