, , , , ,

Mathemat-ART-ical Fract-ART Augumented with Deep Learning

I am fascinated by the combinations of mathematics and art for several reasons; one reason is that it is so difficult to show the beauty of mathematics to non-experts. As a mathematician I am often frustrated that my work cannot be understood by many people that are important to me. Using mathematics to create beautiful visual graphics is an opportunity to show this beauty. It can also be used in educational purposes, of course, or to raise inspiration and motivation. Artificial intelligence is another branch of science which is not very well accessible to the wider audience and work needs to be done to popularise the concepts involved in it. Especially today when the field of machine intelligence is transforming as we breathe.

The above pictures may act as a proof of concept that math and AI can be used to create great art today. I do not mean that any computer has had any role in the creative part of the work, don’t worry about that! But computer was used as a very powerful tool for me to create these images. They were created in two steps. The first step was to create the fractal, or the attractor, using an Iterated Function System. I did it using Javascript generating the image on an HTML5 canvas element. This I will explain in a separate blog post in the future, stay tuned. That gives me a picture like this:

Then I go to Github and download a pre-trained open source deep neural network which is programmed in Python using TensorFlow by someone with the nickname lengstrom. This network does what is called “Fast Style Transfer” and is based on the amazing research done in Stanford. The original idea is described here. I will explain this in a forthcoming blog-post in more detail as well. In a nutshell, you train a network to recognise hundreds of image categories (cats, bats, seahorses… you know)  and then you identify neurons that are sensitive to the style of the image (texture, colours etc), label these neurons with ‘S’ and other neurons which are responsible for features such as lines and circles, call these ‘F’. Now we feed a painting to this network:

Udnie (1913) by Francis Picabia

Look at the activation of the S-neurons and denote their current activation pattern by S*. Then we feed my fractal from above into the same network and look at the activation of the F-neurons and denote their current activation by F*. Then we try to find an image (from all possible images, yes) which, when fed into the network, produces the S* activation in S-neurons and F*-activation in F-neurons, thus it must be a hybrid of those two. And not any hybrid, but such that the style comes from the painting and the features (lines etc) come from the fractal and we get this:

which is my favourite of all the above. This was, by the way, an overly simplistic explanation with many details omitted, don’t take it too seriously ;). If you are interested in this kind of stuff, stay tuned: I am about to publish an interview with a true artist (and not an amateur like me) Tuomas Tuomiranta, who is originally a mathematician and uses serious mathematics (from differential equations to conformal maps) to create art. His motivation is not to show so much the beauty of math, but to use it for artistic purposes, but I see that he still achieves both. Also, I am aware that I promised blogposts on at least 3 different topics above….

The easiest way to stay up to date is to subscribe to my newsletter.


0 replies

Leave a Reply

Want to join the discussion?
Feel free to contribute!

Leave a Reply

Your email address will not be published. Required fields are marked *