Adobe MAX 2010 Opener

This year I had the honor to be part of the opener show for Adobe MAX 2010. I really appreciate that the people at Adobe working on the keynote took the risk to try this out.

Mike Chambers asked me already in April this year if I would like to do it. He had the idea of Erik Natzke generating art, me doing live coding and a DJ playing music. Eric Clark has been a fantastic supporter of Audiotool so I thought it would be really nice to have him do it. Luckily he agreed.

We had several meetings and finalized the idea of the performance. We really wanted to connect the work from everyone of us and create one piece. Otherwise it would have been only three persons doing their stuff. But eventually we got it to a point were I was visualizing the music Eric Clark was playing using pictures that Erik Natzke has been drawing on stage.

I as a developer always want to know how things work. So I also want to explain the technological aspect and challenge of such a performance.

Eric D. Clark was using Audiotool on a touch screen and we really wanted to show how Audiotool can shine there. The idea was of course to support gestures one might intuitively expect. This was done in Germany already. We are using a custom built event handling system so I had to add gesture support there. We do currently support only zooming the desktop since the schedule was so tight. In fact the week before going to MAX we also launched a collaboration of Audiotool and Coca-Cola. Everything was a little bit crazy.

However not as crazy as it went when we came to LA. When doing livecoding I usually prepare my final result already so I know what I want to create. You know where the little problems are and you can make the most out of the time you have. But because of the launch I did not have that much time to prepare. So I wrote one demo for me already in Germany. It was also my first time using the new 3D capabilities I was allowed to use which will be available as a public beta in 2011. Building software with a beta version of something is always fun because it can break for unknown reasons etc.

I also wrote a server that streams arbitrary recording devices into Flash at a very low latency. This was very important of course since I needed to access what Eric Clark was outputting. I did a version that does FFT and normal amplitudes but went only with the amplitudes since an FFT terrain looks not as interesting. It can. But then you have to do more analysis.
On the flight to LA I build another program that would help me save some time during the performance. It is basically automatically compiling and launching a Flash Player. A lot of people asked me about this actually. But you can built it for yourself very easy or I can even release the source code. It is really not much to do. I also contributed this code to Flexmojos as the continuous compilation goal. In fact this was my first idea. But since I did not have internet access most of the time and I was developing this on my tiny netbook I thought I want to have something as slik as possible.
Then there was another server involved I quickly wrote for Erik Natzkes machine. That one was also nothing special. It simply watches a folder and serves always the latest file. That way Erik only had to save an image to a certain folder with AIR and I would get it.
Last but not least we also used FlashMate. My old companion when it comes to multi monitor development. FlashMate hooks into the window procedure of the Flash Player and simply repositions the window always where you left it.

Because of the tools I built I never had to worry about starting the compiler, moving the Flash Player window around etc.

The program I wrote was actually quite simple. It was a subdivided plane in 3D space and I was drawing the audio feed I got from Eric Clark onto a BitmapData. That BitmapData was used as a heightmap for the vertices of the plane. In fact each pixel was assigned to exactly one vertex. That way I could get a really nice and smooth terrain. Thanks to Molehill this was rendering even at 1920×1080 without any hickups.
The heightmap was also used as a lightmap to fake some kind of ambient occlusion. I know that this was rather cheap. But keep in mind that we also had a limited amount of time. Even worse: We had some cue points in the performance. Like when Serge Jespers came on stage and was giving me the audio cable. At that point everyone was expecting me to be ready to visualize the amplitudes. Otherwise the whole thing would not have made any sense.

However we also wanted to show even more at the end because it would have been boring to see only the terrain flight the whole time. That is the reason why I flipped the terrain and started to rotate it. We agreed to this on Saturday, two days before the keynote. But luckily this is a rather cheap copy and paste operation. You take the code you already have and rotate the terrain by 180° around the z-axis. The rest of the animation was really only rotating everything around the y- and z-axis.
During the keynote we were also on IFB and at one point we were told that we have to stretch for five more minutes. That is the reason why the terrain was bouncing, freezing and whatever at the end.

I really enjoyed this whole experience. I love to have a look behind the scenes and it was an absolut pleasure to work together with everyone at the keynote. Friendly people all over the place. And a lot of hackers too.
However it was stressful. It is really not easy to type and concentrate in front of such a big audience. And it is not like a Flash event where you know most people anyways. All of a sudden there is the international press and all the executives sitting front row.

But I really liked the whole thing. Especially the last five minutes freestyle coding. I also want to thank Erik and Eric. The artwork matched the music very well. And I love the idea of us going on stage, each with a blank canvas on its own, creating something in a limited amount of time and having it connect as one piece.

This year I had the honor to be part of the opener show for Adobe MAX 2010. I really appreciate that the people at Adobe working on the keynote took the risk to try this out. Mike Chambers asked me already in April this year if I would like to do it. He had the idea of Erik Natzke generating art, me doing live coding and a DJ playing music. Eric Clark has been a fantastic supporter of Audiotool so I thought it would be really nice to have him do it. Luckily he agreed. We had several meetings and finalized the idea of the performance. We really wanted to connect the work from everyone of us and create one piece. Otherwise it would have been only three persons doing their stuff. But eventually we got it to a point were I was visualizing the music Eric Clark was playing using pictures that Erik Natzke has been drawing on stage. I as a developer always want to know how things work. So I also want to explain the technological aspect and challenge of such a performance. Eric D. Clark was using Audiotool on a touch screen and we really wanted to show how Audiotool can shine there. The idea was of course to support gestures one might intuitively expect. This was done in Germany already. We are using a custom built event handling system so I had to add gesture support there. We do currently support only zooming the desktop since the schedule was so tight. In fact the week before going to MAX we also launched a collaboration of Audiotool and Coca-Cola. Everything was a little bit crazy. However not as crazy as it went when we came to LA. When doing livecoding I usually prepare my final result already so I know what I want to create. You know where the little problems are and you can make the most out of the time you have. But because of the launch I did not have that much time to prepare. So I wrote one demo for me already in Germany. It was also my first time using the new 3D capabilities I was allowed to use which will be available as a public beta in 2011. Building software with a beta version of something is always fun because it can break for unknown reasons etc. I also wrote a server that streams arbitrary recording devices into Flash at a very low latency. This was very important of course since I needed to access what Eric Clark was outputting. I did a version that does FFT and normal amplitudes but went only with the amplitudes since an FFT terrain looks not as interesting. It can. But then you have to do more analysis. On the flight to LA I build another program that would help me save some time during the performance. It is basically automatically compiling and launching a Flash Player. A lot of people asked me about this actually. But you can built it for yourself very easy or I can even release the source code. It is really not much to do. I also contributed this code to Flexmojos as the continuous compilation goal. In fact this was my first idea. But since I did not have internet access most of the time and I was developing this on my tiny netbook I thought I want to have something as slik as possible. Then there was another server involved I quickly wrote for Erik Natzkes machine. That one was also nothing special. It simply watches a folder and serves always the latest file. That way Erik only had to save an image to a certain folder with AIR and I would get it. Last but not least we also used FlashMate. My old companion when it comes to multi monitor development. FlashMate hooks into the window procedure of the Flash Player and simply repositions the window always where you left it. Because of the tools I built I never had to worry about starting the compiler, moving the Flash Player window around etc. The program I wrote was actually quite simple. It was a subdivided plane in 3D space and I was drawing the audio feed I got from Eric Clark onto a BitmapData. That BitmapData was used as a heightmap for the vertices of the plane. In fact each pixel was assigned to exactly one vertex. That way I could get a really nice and smooth terrain. Thanks to Molehill this was rendering even at 1920x1080 without any hickups. The heightmap was also used as a lightmap to fake some kind of ambient occlusion. I know that this was rather cheap. But keep in mind that we also had a limited amount of time. Even worse: We had some cue points in the performance. Like when Serge Jespers came on stage and was giving me the audio cable. At that point everyone was expecting me to be ready to visualize the amplitudes. Otherwise the whole thing would not have made any sense. However we also wanted to show even more at the end because it would have been boring to see only the terrain flight the whole time. That is the reason why I flipped the terrain and started to rotate it. We agreed to this on Saturday, two days before the keynote. But luckily this is a rather cheap copy and paste operation. You take the code you already have and rotate the terrain by 180° around the z-axis. The rest of the animation was really only rotating everything around the y- and z-axis. During the keynote we were also on IFB and at one point we were told that we have to stretch for five more minutes. That is the reason why the terrain was bouncing, freezing and whatever at the end. I really enjoyed this whole experience. I love to have a look behind the scenes and it was an absolut pleasure to work together with everyone at the keynote. Friendly people all over the place. And a lot of hackers too. However it was stressful. It is really not easy to type and concentrate in front of such a big audience. And it is not like a Flash event where you know most people anyways. All of a sudden there is the international press and all the executives sitting front row. But I really liked the whole thing. Especially the last five minutes freestyle coding. I also want to thank Erik and Eric. The artwork matched the music very well. And I love the idea of us going on stage, each with a blank canvas on its own, creating something in a limited amount of time and having it connect as one piece.