Illusion

While stumbling through reddit, I came across a generative art piece by Abe Pazos. I felt very inspired by the way he used gradients to emulate self-shadowing of the shapes making up the picture and wanted to build something on the basis of this piece.

Different levels of tessellation.

I ended up creating an procedurally created and animated landscape using the image.

Breakdown

Basic setup

To create the landscape I tesselated a simple plain to have enough vertecies available for the displacement later on. I wanted the scene to have a retro-wavy style, with synthy music and 3D wireframe models. Luckily Unreal Engine 4 comes with wireframe shading capabilties right out of the box.
Based on the image by Abe Pazos, we get two masks. The Glow mask indicateds where we want the terrain to be animated (unfortunate naming xD). The Terrain Mask indicates where we want the terrain to be extruded but not animated. We actually use these two mask to indicate where the terrain should glow as well. Additionally we sample noise from a noise texture. Different noise texture will yield different effects. You could also use procedural noise.

Material settings. Texture setup.
Material settings. Texture setup.

Material

The material focuses on three values, calculating the base color for the wireframe edges, figuring out which part of the mesh should glow and where the mesh should be deformed. Below you'll find the shader formulated in Unreal's node based shader graph.

Material graph / shader.
Material graph / shader.

First we move a noise texture via uv panning. Then I mask the noise values with the red and the blue channels [1] of the glow map.
These mapped noise values give us the overall shape of the animated part of the terrain. To add height variance in the animation we sample the game time and use it generate a sine wave. Because we are only interested in values between 0 and 1 we mirror the negative part of the sine wave along the x axis. The tricky part now is to extrude the terrain in relation to the planes surface. This is archived by scaling the product of the sine wave and masked noise values for red and blue channel with the normal vector of the vertices of our surface in world space. For the static terrain we basically do the same, only difference being that the terrain mask is sampled instead of the animated noise. Those three values will then be added together and used as information on how much the mesh should be displaced at any give uv coordinate. To allow for maximum effect, we use a tessellation factor of 16. In the video below you see the impact different levels of tessellation have on the terrain.

Different levels of tesselation.

To emphazise the retro-wavy style a bit more, we simply work with the lighting to change the base colors appearance in the scene. This then later gets combined with the emissive / glowing values and results in the final color. For an extra bit of detail, we position the light so that the animated parts of terrain self-shadow other parts of the terrain. In my opinion, this gives the effect much more impact.

Glowing parts (blue is not glowing). Base color. Everything combined + lighting.
Glowing parts (blue is not glowing). Base color. Everything combined + lighting.

After setting up the scene and material, we now just have to tweak the values a bit to fit the aesthetic and music [2]. Now when the sphere is moving through the game world, you'll actually be able to figure out the direction from which the audio is originating. This works both in the VR and non-VR version - if you wear headphones.

Actually values for the material.
Actually values for the material.

'Interactive' part

As you can see in the preview of the material instance editor, the material looks rather interesting when mapped on a sphere. We simply place a sphere into the world, assigning it the same material as the plane and attach an audio source to it [3]. Now when moving the sphere through the scene, we will actually be able to figure out from where the sound is originating. This works with both the VR and non-VR verions - if you wear headphones.

Head over to ich.io to download and try the experience for yourself.

I would love to hear your thoughts! If you want to get in touch, consider following me on twitter @blurryroots or sent me an e-mail.

You might also be interested in these posts:

Also have a look at this project, using Unreal Engine to create a music video.

Footnotes

1 Texture packing

In this setup I am using 4 textures to archive this effect. Because the glow mask is only using the red and blue channel, we could pack the terrain mask into the green channel and the noise map into the alpha channel - effectively reducing the texture count to two.

2 Beat detection

I did actually not use any form of beat detection (just the moving noise and the sine wave) to animate the terrain. I was curious how much I could get away with and still convey the feeling of synchronicity (and maybe a teensy bit of lazyness xD).

3 Binaural audio

This project uses binaural audio after the cutscene part. It basically approximates how human hearing works to play back audio placed within the 3D world.