But then again, the playback was done using JavaScript only and calculated in realtime.

Audiotool is a complex piece of software so you might ask how one could torture themselves by implementing it in JavaScript? We didn’t. Instead we started building our own vision of a cross-platform application framework a couple of months ago.

Introducing project Hiddenwood.

Hiddenwood is a collection of libraries and tools specifically designed to support different devices and platforms. The core libraries are the “driver layer” and always platform-specific with a platform-independent interface.
On top of that we provide a basic layer of libraries like our UI system, animation framework or managed collections which guarantee 0% garbage collection activity and have been battle-tested in Audiotool.

The framework is all about speed and consistency. The rendering pipeline is optimized for OpenGL and although we offer something similar to Flash’s display list a lot of features are not available because they would compromise the speed.

Speaking about speed: we are always interested in staying as native as possible on our target platform. So for the browser we emit JavaScript, for Android you will get the full DalvikVM performance and for the desktop you will get JVM performance. This approach has also another very important aspect. If you want to go platform-specific for certain features you can do that.
For instance if we want to render Audiotool songs on the server using a fork-join pool for our audio calculation this is possible and might not make sense on an Android device.

You write Java code and the supported platforms are native desktop applications, Android (minimum requirements are Gingerbread and OpenGL ES 2.0) and modern browsers. Now for browsers we even go one step further and support multiple options. That means if WebGL is not available we simply fallback to a normal canvas based render-engine. The same applies to some of the Android drivers.

iOS is of course important as well and we are actively researching the best option that will give us the most flexibility and performance.

We are currently working on two real applications built with Hiddenwood. So far it is a real pleasure to enjoy quick build times and simply test what you want on the desktop with great debugging capabilities. When you are ready you can try the same app on Android or in the browser — which might take a little bit longer to compile.

Because we see Hiddenwood as an application framework there are a lot of goodies built-in like a sprite-sheet based class generator. Think Image mixerBackground = Textures.mixer.background(); where mixer was the folder name and background the name of the file.

We believe that as a developer you really do not care about what kind of technology you are using and just want a great result. We also think that you should be able to reuse platform-independent code across multiple projects. However we do not want to take power away from the developer because if you know what you are doing: go for it.

Of course we are not the only ones with this idea. Nicolas Cannasse saw the signs years ago and invented haXe which gives you a comparable experience and Google released playN a couple of weeks ago which takes a similar approach (and requires another 25 committers :P).

But when we started Hiddenwood we wanted the Java tooling experience and playN was not public at that time. We also think that a game engine is not what you want to use for all kinds of applications. So we like to be able to give people the freedom to build their own game engine on top of Hiddenwood — and calculate physics in a different thread peut-être.
Speaking about threading: the only possible solution that works across all platforms is a shared-nothing architecture which we put in place. However if you write platform specific code you can use of course everything the platform offers and a lot of the Hiddenwood core libraries like the network- or cache-layer make use of multiple threads.

In the end what makes Hiddenwood special in my opinion is that we do not believe in write once run anywhere because that just does not make sense. The essence and philosophy behind Hiddenwood is to write platform-agnostic code using kickass-libraries and being able to reuse that. Audiotool on a tablet would look completely different from Audiotool running in a browser. And Audiotool on iOS would probably be also a little bit different from Audiotool on an Android device because there are simply different paradigms you should respect.

I hope that we can share more information with you soon. With the news of mobile Flash Player being deprecated and the ongoing demand for cross-platform development we have exciting times ahead of us. I am also super excited about the (beautiful <3) applications which we are going to release in the not so distant future.

This year I had the honor to be part of the opener show for Adobe MAX 2010. I really appreciate that the people at Adobe working on the keynote took the risk to try this out.

Mike Chambers asked me already in April this year if I would like to do it. He had the idea of Erik Natzke generating art, me doing live coding and a DJ playing music. Eric Clark has been a fantastic supporter of Audiotool so I thought it would be really nice to have him do it. Luckily he agreed.

We had several meetings and finalized the idea of the performance. We really wanted to connect the work from everyone of us and create one piece. Otherwise it would have been only three persons doing their stuff. But eventually we got it to a point were I was visualizing the music Eric Clark was playing using pictures that Erik Natzke has been drawing on stage.

I as a developer always want to know how things work. So I also want to explain the technological aspect and challenge of such a performance.

Eric D. Clark was using Audiotool on a touch screen and we really wanted to show how Audiotool can shine there. The idea was of course to support gestures one might intuitively expect. This was done in Germany already. We are using a custom built event handling system so I had to add gesture support there. We do currently support only zooming the desktop since the schedule was so tight. In fact the week before going to MAX we also launched a collaboration of Audiotool and Coca-Cola. Everything was a little bit crazy.

However not as crazy as it went when we came to LA. When doing livecoding I usually prepare my final result already so I know what I want to create. You know where the little problems are and you can make the most out of the time you have. But because of the launch I did not have that much time to prepare. So I wrote one demo for me already in Germany. It was also my first time using the new 3D capabilities I was allowed to use which will be available as a public beta in 2011. Building software with a beta version of something is always fun because it can break for unknown reasons etc.

I also wrote a server that streams arbitrary recording devices into Flash at a very low latency. This was very important of course since I needed to access what Eric Clark was outputting. I did a version that does FFT and normal amplitudes but went only with the amplitudes since an FFT terrain looks not as interesting. It can. But then you have to do more analysis.
On the flight to LA I build another program that would help me save some time during the performance. It is basically automatically compiling and launching a Flash Player. A lot of people asked me about this actually. But you can built it for yourself very easy or I can even release the source code. It is really not much to do. I also contributed this code to Flexmojos as the continuous compilation goal. In fact this was my first idea. But since I did not have internet access most of the time and I was developing this on my tiny netbook I thought I want to have something as slik as possible.
Then there was another server involved I quickly wrote for Erik Natzkes machine. That one was also nothing special. It simply watches a folder and serves always the latest file. That way Erik only had to save an image to a certain folder with AIR and I would get it.
Last but not least we also used FlashMate. My old companion when it comes to multi monitor development. FlashMate hooks into the window procedure of the Flash Player and simply repositions the window always where you left it.

Because of the tools I built I never had to worry about starting the compiler, moving the Flash Player window around etc.

The program I wrote was actually quite simple. It was a subdivided plane in 3D space and I was drawing the audio feed I got from Eric Clark onto a BitmapData. That BitmapData was used as a heightmap for the vertices of the plane. In fact each pixel was assigned to exactly one vertex. That way I could get a really nice and smooth terrain. Thanks to Molehill this was rendering even at 1920×1080 without any hickups.
The heightmap was also used as a lightmap to fake some kind of ambient occlusion. I know that this was rather cheap. But keep in mind that we also had a limited amount of time. Even worse: We had some cue points in the performance. Like when Serge Jespers came on stage and was giving me the audio cable. At that point everyone was expecting me to be ready to visualize the amplitudes. Otherwise the whole thing would not have made any sense.

However we also wanted to show even more at the end because it would have been boring to see only the terrain flight the whole time. That is the reason why I flipped the terrain and started to rotate it. We agreed to this on Saturday, two days before the keynote. But luckily this is a rather cheap copy and paste operation. You take the code you already have and rotate the terrain by 180° around the z-axis. The rest of the animation was really only rotating everything around the y- and z-axis.
During the keynote we were also on IFB and at one point we were told that we have to stretch for five more minutes. That is the reason why the terrain was bouncing, freezing and whatever at the end.

I really enjoyed this whole experience. I love to have a look behind the scenes and it was an absolut pleasure to work together with everyone at the keynote. Friendly people all over the place. And a lot of hackers too.
However it was stressful. It is really not easy to type and concentrate in front of such a big audience. And it is not like a Flash event where you know most people anyways. All of a sudden there is the international press and all the executives sitting front row.

But I really liked the whole thing. Especially the last five minutes freestyle coding. I also want to thank Erik and Eric. The artwork matched the music very well. And I love the idea of us going on stage, each with a blank canvas on its own, creating something in a limited amount of time and having it connect as one piece.

A big thank you goes out to the organizers and the Flash community. It really shows that the opensource work I do is appreciated and not meaningless.

Part of the prize I already received has been donated to Eugene Zatepyakin. I think he is doing great work on ASSURF and was in desperate need of an FDT license ;)

Thank you. I hope I will be able to surprise you with some new stuff later this year.

We’ve been there before, and intermediate layers between the platform and the developer ultimately produces sub-standard apps and hinders the progress of the platform.

That is so right. The first thing that comes to my mind is Java on the Mac: sub-standard, castrated and broken. The second argument is even better. Intermediate layers can hinder the progress of a platform. So true. Apple, the intermediate layer between you and the iPhone, stops you from improving the platform. You have built something cool — like Unity3D — and can empower creative people with better tools? Sorry, you are out.

The only way to improve the progress of a platform is by opening it up. Yes, I think we have been there before.

Sounds strange doesn’t it? Thank god this was just a fictional interview and will never become reality.

The recording of the jam session is also available. I just want to note that I used the processing library inside Eclipse for the speedcoding. This time I also practiced at home unlike for other jam sessions. The visualizer was an homage to a table designed by Daft Punk for habitat.

Before FOTB I was not sure if I can present Apparat at all. For me as an autodidact it is not always easy to solve certain issues. Mario Klingemann talked about Magic Moments in Brighton. I guess I had one after a long time of thinking and failing. I got really into the flow and everything became so clear. Stay tuned for more detailed posts on the technique behind the different compilers.

However this was a really special conference for me. The standing ovations made me completely happy and I was absolutely overwhelmed by the response. After all those sleepless nights it was a huge gift of appreciation for my work. I will definitely never forget that moment and all I can say is thank you. If you spend so much time working on an opensource project this is really the best that can happen. “Leaving the Sandbox” came from a long background. The end of the session was a really emotional for me. Thank you John for inviting me.

This is not why I am angry at all. Actually the last straw was this post by Ted Patrick. So we are the community and we can change the Flash Player. Of course our Adobe, MAKE SOME NOISE! campaign was a success but I am really frustrated. It is definitly not easy to contribute to the Flex SDK or to the Flash Player directly.

Have a look at this report. I took the time and fixed the [Embed] tag of the compiler to support a quality attribute and a special way of embedding PNG files which significantly reduces the filesize of your SWF. After taking the time and generating the patch nothing happened. No response and not even the ability to submit a patch.
I have even more examples. I do not know many Flash developers that ever compiled Tamarin but if you take the time to modify it you can do cool stuff. Again take a look at this report. I provided the complete source code for my request and spent a day or so on getting Tamarin to compile and run with a playerglobal.abc that has a modified Math class bound to new native methods. Since a year has passed without any kind of response I can mark that time as wasted.

You may have noticed I am not so motivated to fix AS3V at the moment. The reason is quite simple.

Again, the existing Node classes are subject to change in the very near future and I can almost guarantee that any work based on the current intermediate representation will be incompatible with ongoing work.

Knowing that AS3V will break soon is not a really good motivation to continue working on it. There is no roadmap for the Flex SDK available. So basically linking against it or modifying it is some kind of timebomb you can not control. I still do not know when I will have to rewrite AS3V. Even more examples would include the story of AS3Doc which was a documentation tool I just finished a day before Adobe released ASDoc.

I hope you can understand why I am a little bit frustrated when reading something like how open Adobe is and how close they work with the community. At least to me it seems like the future of the Flash Platform is defined in some conference room in some Adobe office. But it seems like I am not alone with this opinion if you take a look at the comments of Ted’s post. How many people have complained about the lack of generics, threading and poor runtime performance?

There is always a way to make things better. I want to split them in three parts.

Contributing

Contributing to the SDK and to the Flash Player is a great option and I am thankful for having it. But this includes also contributing to the Flash Platform with tools.

• Extend the opensource Wiki for up-to-date specifications and allow people to contribute to them.
• Create an open ActionScript documentation.
• Do not develop something closed source to open it six months later like the TextLayoutFramework.
• Create a progress for code reviews and allow people to contribute without having to go through JIRA hell.
• Make a roadmap available so developers know what happens inside Adobe.

Language

To be honest no rational human being could say that ActionScript 3 (aka Flash) is superior to C# (aka Silverlight). Although love is not always rational of course. But in order to stay competitive here is my masterplan.

• Add support for generics.
• True method overloading.
• Threading support has to be implemented including synchronization features.
• Rethink the const keyword which is currently completly unusable.
• Implement standard OOP features like private constructors, abstract methods and classes.
• Create a collection framework making use of genercics.
• Create a robust concurrency framework.

Performance

Google’s V8 is getting faster and faster. Once JavaScript performance is better than ActionScript 3 it becomes ridiculous.

• Improve the VM performance especially for object allocation and method calls.
• A tracing JIT would be great, hotspot would be fantastic.
• Improve the performance of the compiler and its produced output.

I hope that noone misunderstands me. I still love what I am doing and the reason why I continously release projects like AS3V, PBDT, TDSI, etc. free of charge is simply because I want to improve our platform and workflow. I just hope someone at Adobe will read this post and understand how frustrating it is to develop such programs and to fix some stuff when not sitting right next to the product manager. Hopefully something changes. A man can still dream…

I have never been to the States before so I am really looking forward to it. At my session “AudioTool’s Private Parts” which had been completely renewed for the FFK09 in Germany I will talk of course about the technologies that our tool uses and show some examples using Hobnox Open Source. I will also show a live preview of AS3V running in Eclipse and how it integrates into the daily workflow.

Therefore I could not receive any e-mails going to that domain and a lot of links in the blog were broken. Everything should be fixed now and all the old links should work again. I am also able to answer my mails now. Unfortunately everything I received during the last weeks is completly lost.

Sorry for any inconvenience caused.

I was nearly a whole month offline and recharged my batteries for a challenging new year while chilling out at the beach and enjoying the sun. To make it short: I am back and I have lots of ideas in my mind.

I will also put some more sets and other tracks there if I find the time. The site will update randomly.

• Tones

The category “Special” has been introduced to cover high quality and highly innovative projects which do not fit into any other category like “Information”, “Education” or “Entertainment”. Regarding to the press statement Hobnox has been picked not only because of its high quality content and design but also because of the Noxtools — and this makes me especially happy.

After the FITC we went from Amsterdam to Berlin and celebrated the launch of the Hobnox platform and the winners of the Evolution contest.

If you were able to register at www.hobnox.com you are now finally able to try out our latest baby. I am looking forward to get some feedback.

Since more than two years I think I was looking at Das Keyboard and never had the cojones to buy it. Since five days now I am the proud owner of this masterpiece and I am more than happy with it. If you ever hesitated to buy it — do it. I never experienced a typing comfort like this. The mechanical key switches will charm your fingers and the sound of it is completly addictive too :)

Probably you noticed that I was working with Hydra a lot in the past. Everyone who worked with BitmapData will love Hydra and I have the honor to introduce you into the world of shaders. And even better: It is not only about Flash since Hydra filters work with AfterEffects too.