Jan Huenermann

Abstract Art with ML

Having some fun creating art with neural networks.

Did you know that randomly initialised neural networks actually produce pretty cool pictures?

Well, I didn't, until I recently discovered pattern producing networks. This post will be all about them, and will contribute a pattern generator running in your browser, so you can experience these networks for yourself. But first, let me explain.

Compositional Pattern Producing Networks

A compositional pattern producing network, or CPPN in short, is a network, in this case we will focus mainly on neural nets, that given some parameters produces a visual pattern. The main idea here is to slide the network over all the x-y-coordinates in an image and produce a 3-channel (rgb) color output for each pixel in the output image. We can look at this network ff in the following way: (rgb)=f(xy)\begin{pmatrix}r\\g\\b\end{pmatrix} = f\begin{pmatrix}x\\y\end{pmatrix}

This network, since it's continuous and differentiable, outputs locally correlated values, meaning, sampling the network on two different very close points will lead to a very similiar output value. This basically results in the property that the image we can generate from it, we could call smooth.

Another cool property this has, is that it has "infinite" resolution, because you can just scale the coordinates the network receives as inputs.

Parameters

Now we could simply run the network as it is. And this in fact works. But we take this a little step further by adding certain other inputs to the network with the aim of having the network generate more complex images.

For example we can add the radius rr and an adjustable parameter α\alpha. With these modification our network ff looks like this (rgb)=f(xyrα)\begin{pmatrix}r\\g\\b\end{pmatrix} = f\begin{pmatrix}x\\y\\r\\\alpha\end{pmatrix} with r=x2+y2r = \sqrt{x^2 + y^2}. The radius not only provides a nice non-linearity but it also enables the network to correlate the output color to the distance to the origin, because points on the same circumference receive the same values for rr.

While the radius rr changes with xx and yy, the α\alpha parameter is static over the course of the image. In essence, you can think of this parameter as the z-parameter. When sampling from the 3-dimensional (x y z) cube we look at the slice at position z=αz = \alpha.

You can get very creative with these parameters and we'll explore more exotic configurations later on.

Initialisation

The output of a neural network is defined a) by its inputs which we talked about in the last section and b) by its weights. The weights therefore play a crucial role in how the network behaves and thus what the output image will look like.

In the example images throughout this post, I mainly sampled the weights WW from a Gaussian distribution (N\mathcal{N}), with a mean of zero and with a variance depended on the number of input neurons and a parameter β\beta which I could adjust to my taste. W(Nin,β)=N(μ=0,σ=β1Nin)W(N_{in}, \beta) = \mathcal{N}(\mu = 0, \sigma = \beta \frac{1}{N_{in}})

Now to the fun part. Here is a progressive image generator based on the principle of CPPNs. You can adjust the Z-value, the variance (which equals β\beta in the above description), choose if you want a black and white image (B/W) and explore the seed space. Note, that the random number generator used for the Gaussian distribution will always produce the same values for the same seed, which is cool, because you can share the seed and retrieve the same result.

I compiled a list of example seeds that I found quite compelling while I played with the tool. Try these seeds for the start and then explore the space yourself! :)

"Complexity" 6130076054
"Streams" 2321123
"Lines" 9183923745
"Alice's purple rabbit" 6828398570
"The Storm" 3742851
"Prism" 3742851
"Gradient" 3742849
"Excess" 2321175
"Stained" 6828398584
Figure 2: example seeds. Click on an image to process it in the generator.

And these are just some examples. Try changing the seed, the time or the variance yourself and maybe, just maybe, you come across some masterpiece! 😛

Exploring other architectures

In this section, I want to show you some results I've been getting with some more exotic architectures.

As you can see, these images behave surprisingly very differently. A single parameter more makes a huge difference in the activations of the networks.

Wrapping up

Now that we have gone through several architectures, explored multiple configurations and looked at a bunch of images, it's time to wrap up. But I want to wrap up by pointing to some possible improvements.

In terms of use case: these images make great color and gradient inspiration! Also, I just recently replaced my Spotify playlist covers with these. They make pretty great album artworks (don't they?). (Flume, hit me up ;-) )

Anyway, that's it for now. I hope you have enjoyed my first blog post. If you did, I'd appreciate if you consider subscribing to my blog (via RSS or JSON-Feed). I'll try to publish blog posts regulary here. See you then!

Acknowledgements

I want to thank David Ha (@hardmaru) for his work on his blog series on CPPNs[1] and his open source publications which were a great resource to me during development of this blog post. Also, Dennis Kerzig's implementation of CPPNs[2] served me as a great base for generating these pictures in Python.

Code

I published the code used to generate the images in this post as well as the source code for figure 1 [^] on GitHub. As mentioned, the Python code is based on [2] and runs using Tensorflow. You can find the repository here.

References

  1. Generating Abstract Patterns with TensorFlow [link]
    David Ha
  2. CPPN in Keras 2 [link]
    Dennis Kerzig