Making memories with an algorithm: Andreas Rau on Echo, No Resonance

Andreas Rau on his latest series in collaboration with Here & Now.

The magic of the moment

Enabled by technology, we record our lives as never before. Every remotely meaningful moment is turned into a photo or video, and shared on social media. In this abundance of recorded information, moments that don’t receive this kind of permanence gain a special beauty. Moments where the Here and Now is important, and nothing else. For in essence, the beauty of the unrecorded lies in its ability to preserve the sacredness of the moment, the pure power that lies within the ephemeral.

As a jazz musician, I know a thing or two about the magic of the moment and the unique energy of creation that happens in the Here and Now. While you certainly can rehearse, there’s no way to plan for the freedom and creative force that can emerge in these moments.

This same concept is at the very core of generative art: The system may be well-defined, yet you can’t plan for what will emerge from it in the end. It’s this uncertainty that I find fascinating: The infinite space of possible outputs, the energy of the moment and the underlying risk of failure.

Therefore, I was immediately intrigued when Here & Now approached me about their new concept of live cinema — a movie that would be performed, shot, and displayed in real time, with no way of going back, and with no recording available after its creation. A blend of theatre and cinema of sorts, with a generative series using the film as its input as the only record of the film.

Knowing Here & Now from their fabulous curation and experiences on Tezos, it was easy to say yes to this collaboration.

Making memories with an algorithm

Using an algorithm to create the only permanent memory of an ephemeral experience sounds like trying to invent a memory making machine — a rather impossible undertaking. And even though computers these days have plenty of memory, and can record down to pixel perfection, the most important quality of our own memories would be lacking from such recordings: meaning.

Our memories aren’t pixel perfect. On the contrary, they are fragmented and distorted versions of reality, tightly connected to our emotions in that very moment, and highly influenced by our biases. And this is what makes them meaningful, after all. We remember moments for how we felt, not for what we saw. Without memories connected to it, any recording — even in the highest fidelity — remains a stale version of the actual moment.

But in the process of making memories, we are constantly applying our emotional filters. As human beings, we have a tendency to remember extraordinary events and forget the ordinary moments that make up the majority of our existence. Often, we fail to see the magic within these seemingly ordinary moments while living through them. Through the eyes of an algorithm, however, all moments are the same — regardless of their immediate emotional significance to us humans.

Unlike us, an algorithm can choose moments at random, and, through this selection, attach meaning to them. And this is exactly what happens in Echo, No Resonance: The algorithm will select random moments from the live film, and turn them into artworks, thus putting ordinary and extraordinary moments side by side in the collection of artworks to be minted.

In this process of creation, the algorithm acts as a metaphor for how our memories are shaped. It transforms the pixels of the film, the raw data that make up a particular moment, into fragmented and distorted versions of this reality. The outcome, similar to our memories, is not an exact representation of what was visible, but rather an alternate view into the same world.

The process the algorithm goes through to create the final artwork is, in its essence, rather simple:

[1] Downscale and reduce the colors of the original image
[2] Train a Markov chain (a simple probabilistic model) on the sequence of visible pixels
[3] Produce images from the model, and assemble slices from these images into the final artwork

In a play with different levels of abstraction, this process is executed to different degrees, yielding familiar yet distorted versions of the image we saw. Sometimes, there will be visible remnants from the original visible in the output. At other times, everything is abstracted into simple shapes, leaving plenty of room for interpretation and, your own memories.

Should you decide to purchase a mint pass for Echo, No Resonance, you will not only get access to streaming the film in the very moment of its creation — a world premiere — but you will also receive an algorithmically created artwork, derived from one particular moment in the film. You will be able to watch the movie together with plenty of other film enthusiasts around the world, and the algorithm will, metaphorically speaking, watch together with you. While you may remember certain events from the film, the algorithm will select random moments and elevate the magic that lies within them.

In this way, I hope that you might start to think differently about the significance, the meaning, of these moments. For there might be magic and beauty hidden in moments where you least expect it. There might be some such moments in the film, but there certainly are plenty of them in your life. You just need to keep your eyes, and your mind, wide open.

Echo, No Resonance is a joint creation, a harmonious dance between generative artist Andreas Rau and the artistic collective HERE & NOW, presented on Verse by NGUYEN WAHED. Each artwork by Andreas Rau serves as a unique memory of a live cinematic performance; an ephemeral film event that can only be witnessed once in real-time in the moment of its creation. The movie will be shot, performed, and edited live on the 8th of December around the streets of Melbourne and streamed in real-time for collectors on verse.works and in a cinema in Melbourne, Australia.

This once-only live cinematic experience will be fed into Andreas Rau’s algorithm and a series of 400 artworks will be selected by the artist.

Explore the algorithm below: