A part of the Flash Player is the ActionScript Virtual Machine (AVM) which is responsible for executing the code that is bundeled in your SWF file. This piece of software called Tamarin is open source and released under the MPL/GPL/LGPL Triple License.

Being able to look at the heart of the Flash Player does reveal a lot to the interesting reader. However such a virtual machine is a complex piece of software and you can be lost without any guidance and will maybe not even be able to compile it unless you are familiar with such systems. Jean-Philippe Auclair’s blog contains a lot of interesting Tamarin and Flash Player releated articles if you want to know more about the VM. Of course you can also read about MMgc — the garbage collection — directly at the Mozilla developer network.

This article is titled “Opening The Blackbox”. Although Tamarin might be some kind of blackbox to most of us the source code is available. We can dig into it and understand how it works. The Flash Player however is not completely open source and we do not know about its implementation details. Unless Adobe is opening up the source code we will not know which exact algorithm is used for BitmapData.noise for instance.

Fills an image with pixels representing random noise.

This quote taken from the documentation does not tell you much about how to calculate this noise. In this situation I would create some algorithm that fills a BitmapData also with random noise. But until we know which algorithm is used to calculate that noise it will never be the exact same noise as the one of the Flash Player. For now we will have to live with it. There is no (legal) way to figure out the algorithm.
Of course there are also other classes like ByteArray which have methods like readByte that do not need any extra explanation. But ByteArray also has some implementation specific details. How does it grow when you exceed its length for instance? Fortunately ByteArray is something you can find in the Tamarin source code. JITB uses a different algorithm however which you can find here.

But what else do we got? There are some funny things you will notice. Usually Sprite(x) is an explicit cast. This is true for any statement like this. However Number(x) is a conversion with specific details how a value is converted. This is also true for some other expressions like Boolean().

The playerglobal.swc contains all classes the Flash Player knows about. It should and it does with some minor differences. Of course it could be a bug on my end but some interfaces are missing. And they are suspicious since they cause/have special behaviour. Have a look at the AbcFrontend code. The Synthetic object contains part of the AST that is synthesized since those interfaces are missing in playerglobal.swc.

A lot of the Flash Player implementation details have made it into the DisplayObject and its subclasses. You cannot extend from DisplayObject and you cannot implement IBitmapDataDrawable. Even more interesting is the amount of classes that you cannot really use. flash.geom.Transform is such a class. How is concatenatedMatrix working if you would create your own instance of the Transform object?

Of course somewhere in the Flash Player will be some code that creates a Transform object. This will hold a reference to the original DisplayObject and concat the matrices up to the stage if you call concatenatedMatrix. The JITB implementation adds some special methods to the Transform class in such cases. Those methods are always prefixed with JITB$ if they are public. Another interesting fact is that you can modify the appearance of a DisplayObject by setting the matrix of the Transform object. This also has to change the scale, x, y and rotation values of the DisplayObject. There are a dozen of souch cases. Graphics would be an obvious candidate to talk about. I would advise you to take a look at Claus’ blog for everything you never wanted to know about shapes.

Also something you probably never cared about is the way how URLLoader works. I do not care about the specific implementation details either. However one thing is important to keep in mind. Loaders have to work asynchronous since they do not block. This means your Flash content runs while waiting for an event like Event.COMPLETE. This is where it gets interesting since threads enter the game. This would be a simple view: ActionScript runs in one thread, the loader in a second thread. First of all we have to keep in mind that the ActionScript code runs in a loop which is basically only triggered by events like Event.ENTER_FRAME or user input. Only the code in your DocumentClass is the one that is initiated without an event. So how does this look like in pseudo-code?

while(NoErrorOccurred && Running) {
  Dispatch Event.ENTER_FRAME to each DisplayObject
  Render DisplayList
  Dispatch Event.EXIT_FRAME to each DisplayObject
  Sleep(1000 / FrameRate - Elapsed)
}

Okay but how does an Event.COMPLETE make it into this loop when it comes from a different thread in which the loader runs? If the loader would simply dispatch this event when it completes the code would not run in the main thread. This means you could end up with ActionScript code running in parallel which would cause chaos and destruction. So what do you do? The current approach in JITB is to gather all events and dispatch them at the beginning of a frame. So basically the code looks now like this:

while(NoErrorOccurred && Running) {
  Dispatch All Gathered Events
  Dispatch Event.ENTER_FRAME to each DisplayObject
  Render DisplayList
  Dispatch Event.EXIT_FRAME to each DisplayObject
  Sleep(1000 / FrameRate - Elapsed)
}

The implementation specific details can be found in EventSystem.java class. There are also some utility methods like futureDispatch to make development easier. Dynamic audio requires to dispatch an event that is filled with content. The thread in which the audio is passed to the sound card has to wait until SampleDataEvent is filled with new content. This also reveals a problem. The audio calculation has to wait for a complete ActionScript frame until new material is available. In fact such synchronization points are problematic for a task that requires very low latency. You can hear this when your ActionScript code takes very long to process the new audio and you get a stutter-noise or audible gap. Unfortunately there is nothing you can do about this problem at the moment besides keeping the cost of your ActionScript code as low as possible.

Of course everything is only my interpretation. The Flash Player might not suffer from the problems I describe here since someone might have come up with a better implementation. In fact I would doubt that the Flash Player works for dynamic audio the same way I do it currently.

The last blog post of this series will be more about how you can implement parts of the Flash Player API in Java so that JITB understands them and vice versa. Of course we can create our own playerglobal.swc as well and add new native classes like GLSLShader which I demoed at FOTB.

ActionScript 3.0 is a dynamically typed language with function closures. Furthermore it has native XML support, scope-changes, implicit getters and setters etc. The main issue however is to convert the set of bytecodes ahead of time to statically typed Java code. JITB does not know anything about what is going on since it compiles all the code ahead of time.

JITB and TAAS

JITB makes use of TAAS. A three address code intermediate language. TAAS has its own set of rules and typesystem that I defined for myself. Method overloading is supported for instance. And there is no concept of array access. Code like x = a[0] is converted to x = a.getIndex(0). In TAAS terms the name of a method is also not worth a penny since it is always using a reference based lookup.

What I love about TAAS is that it has a really small instruction set. Array access is just a method call like any other method call. No extra instructions added. This makes optimizations really slick and easy to develop. Since the code is statically typed you also always know which method is being called and you could construct a call graph and do whole program optimizations which is a huge bonus. Escape analysis is something that is now finally possible for me.

JITB uses a frontend that parses ActionScript bytecode and creates an AST which is then transformed and passed to a backend that creates Java bytecode. The class and package structure is stored in a tree form. Method bodies are always stored in a graph representation. The interesting problem is to convert ActionScript into TAAS since it is no direct mapping and some statements are not supported in the same way you get them form the ActionScript bytecode. Then the next interesting problem is to convert this code into Java bytecode which is also not a direct mapping.

An example for such a problem are implicit getters and setters. TAAS does not have a concept of getters and setters but understands methods. So whenever the ActionScript bytecode has a GetProperty instruction I try to figure out if this is a field or not. If it is a field I generate a TLoad instruction, if it is an implicit getter it will be converted to a TCall instruction. However if it cannot be determined since the class might be dynamic a TCall instruction will be generated which uses the special TGetProperty method and this will be resolved at runtime.
When converting to Java a TLoad can be easily translated to a GETFIELD operation. When it is a TCall I first lookup whether or not it is a special native function. If it is a special function like TGetProperty I insert Java code like object.JITB$getProperty(name). If it is a normal function I simply insert an invoke operation.

Closures for Java

Java 1.6 does not support closures. This means Java does not understand the concept of a reference to a function. Code like addEventListener(Event.ENTER_FRAME, onEnterFrame) is not possible with Java since onEnterFrame is a reference to the onEnterFrame function which is defined somewhere else in your code.

I took the idea from Scala and how they support closures in the JVM. Basically when code like addEventListener(Event.ENTER_FRAME, onEnterFrame) occurrs it will be translated to the TAAS instruction TCall(TReg(0), TaasMethod(addEventListener, ...), List(TString("enterFrame"), TClosure(TaasMethod(onEnterFrame, ...))), None). This basically means that the method addEventListener is called on the object in register zero with two arguments of which one a closure and it has no result which is indicated by None.

This instruction is expanded to the following Java code. Please remember that it is always Java bytecode. I think it is just easier to understand if I write the corresponding Java code here.

this.addEventListener("enterFrame", new Function1() {
  override Object call(Event arg) { callVoid(arg); return null; }
  override void callVoid(Event arg) { onEnterFrame(arg); }
});

Function1 represents a function of arity one. The AVM can call any method as either void or with an expected result. If the method has a result and you call it via CallPropVoid the resulting object is ignored. If however you have a method that does not have a result and you call it via CallProperty the null object will be pushed on the stack.

The Java code in EventDispatcher which would call closures basically now only looks like this:

void dispatchEvent(Event event) {
  for(Function1 listener : listenersFor(event.type))
    listener.call(event)
}

Dynamic code in Java

Since I want to target Java 1.6 I do not make use of the Java 1.7 opcodes like invokedynamic. Because TAAS is already statically typed we have only in very rare cases the need to make a dynamic lookup. This is done by some helper classes or methods. Every ActionScript based class extends from jitb.lang.Object. The jitb.lang package represents the ActionScript’s toplevel. The Object class has certain helper methods like JITB$getProperty and JITB$setProperty. Those methods throw an error by default if an object is not dynamic but they can implement the behaviour to store values in a HashMap for dynamic classes.
However currently dynamic method lookup is not supported. Code like a['method'+Math.random()]() fails since it is impossible to know upfront which method is called. But this can be supported as well. It is just very cumbersome and annoying.

Emulating the AVM in the JVM

What kind of error do you expect for this kind of code: var x: String = null; trace(x.toString());? This creates a TypeError in ActionScript but a NullPointerException in Java. Same for things like var x: A = new A(); var y: B = B(x); which would result in a Java ClassCastException if A is not assignable to B.
JITB offers an option which is by default turned on the get the exact same exceptions like the AVM would throw them. This means that a Java cast is converted to a call like AVM.cast(x, B.class) with the option turned on or the simple Java CHECKCAST bytecode when the option is turned off. Basically expecting something to be null in a try-catch block is very silly in my honest opinion. So to get rid of this overhead you can disable this extra step of correctness.

Another interesting issue is the fact thatthe AVM can throw any object. Java however expects that a throwable object extends from the Throwable class. Of course the basic jitb.lang.Object class could simply extend Throwable. However I do not want that overhead and at any time. Also I do not want to break how classes extend each other. If you do something like throw "string" it is converted to throw new Throw("string"). Throw does extend Throwable and whenever it is catched the value is simply extracted.

To box or not to box?

ActionScript allows you do use int as an object with a lot of special behaviours. Same happens for values like Boolean or Number. var x: int; trace(x); outputs 0 but not null. But it is also possible to call 0.toString() which indicates that the integer value 0 is treated like an object that has methods like toString().
In Java however you have a real int primitive and the Integer object. 0.toString() does not compile but you would rather do Integer.toString(0).

Of course Java is much faster when using primitive data types when appropriate. JITB does this since it handles primitive types like real primtives for Java. This means a method like function add(x: int, y: int): int { return x + y} is converted to int add(int x, int y) { return x + y }. However when you have a method that expects an Object type a primitive will be boxed into a compatible type.

Last but not least what happens for code like 0.toString() or other methods? A possibility would be to emit code like new jitb.lang.Number(0).toString() but that does not make sense in most of the cases. Instead those calls are treated specially with a call like jitb.lang.Number.JITB$toString(0) and we get rid of an extra allocation.

Conclusion

I hope you understand a little bit better now how JITB works internally and what the main gotchas are in such an implementaiton. The journey is not over since I have to add support for anonymous methods and although I talked about things like throw I still have to add support for try-catch. Also a lot of other basic features are still missing but I doubt that they would cause any problems.

The other side of the story is the Flash Player API inside of JITB which is maybe worth a blog post like this as well since it reveals a lot of other funny and interesting gotchas.

Step #1: Parsing PixelBender data

PixelBender compiles down to PBJ which is a two-address based bytecode format. Basically it contains instructions like this one which I have taken from Nicolas’ blog:

  sampleNearest f2, text0[f0.rg]
  sampleNearest f3, text1[f0.rg]
  mov f4, f2
  mul f4, f3
  mov f1, f4

Those PBJ files are parsed by Apparat into high level structures. This happens basically the same way I do it for ActionScript bytecode. Thanks to the work of Nicolas Cannasse this was a walk in the park.

Step #2: Optimizing PixelBender bytecode

The PixelBender compiler does not produce optimal code as noted by Nicolas. You can see it in the code I just posted. Or you can simply compile and empty shader and take a look at the disassembled output. Apparat performs only two very simple optimizations for PixelBender which are written especially for PBJ. Thos include copy propagation and dead code elimination. The compiler does create code which is not bad thanks to LLVM but there are unnecessary copies going on which are removed by Apparat.

Step #3: Loop detection

PixelBender still has no loops since its initial introduction in October 1st in 2007. Tomorrow it is October 1st in 2010. Go figure. So since there are no loops the only way people write loops in PixelBender is basically by copy and paste or using preprocessor directives. With JITB I want to send real shaders to the graphics card and I do want to make use of loops. So I wrote a PBJ bytecode based copy and paste detector which is based on ideas of the PMD copy and paste detector. When a segment is detected in a sequence I convert it to a loop. That way I can reduce the amount of data I have to send to the graphics card. This is very important since in the case of the demos I showed at FOTB they contained around 50.000 instructions. And my cheap graphics card was complaining that there are too many instrucitons. With the loop conversion this number drastically decreased.

Step #4: Converting PixelBender bytecode to GLSL

It is very easy to convert PixelBender to GLSL. The two address code format can be translated with some minor tweaks to GLSL with a very simple mapping. However this is not really optimal since you have only a two address code. When exporting to GLSL I perform one additional step which expands the two address based code to a three address based code. That way I do not emit a lot of unnecessary copies again.

That is basically the recipie to convert PixelBender PBJ files to GLSL shaders that perform well on current graphics card hardware. There are additional steps involved to get OpenGL shaders to work like PixelBender in Flash but they are only minor tweaks. I have no comparable metrics right now. However I will create a JITB applet version so you can have a look for yourself soon.

This time recorded on Ubuntu. Did I mention 64bit already?
Do not miss my session at FOTB for a live demo.

I want to put a lot of things regarding JITB in perspective here as a followup to the various comments that I received via mail, reddit, etc.

Flash Replacement

JITB is not a Flash replacement. Imagine a F1 race car and a family van. The family van comes with air conditioning, a radio and a lot of comfort. For the F1 car you will probably have to wear a helmet and need an engineering team to get it running.
The important factor for me is that JITB will run very fast under certain conditions. A race car is also not a jeep. JITB will not support all features of the Flash Player. If you search for a Flash replacement you might want to take a look at lightspark.

Performance

Java is slow and the world is flat. That being said I will not talk about Java performance. JITB compiles to native Java code with a certain overhead. An ActionScript 3 closure is converted into an anonymous class for instance.
JITB converts ActionScript 3 bytecode to Java bytecode at runtime and performs various optimizations. This way we can leverage the speed of the JVM for ActionScript and get great results.
That being said there is also a problem of course. Startup time is not very good. JITB and its parent project Apparat are implemented in the Scala programming language and written for multi core architectures. The startup costs are high and JITB is only useful for long living applications. However compiling ActionScript to Java can be done AOT. This means you need to deal only with the normal JVM startup time. The Flash Player API is implemented in pure Java.

The compiler used for JITB is doing relatively naive optimizations at the moment. Namely constant folding, copy propagation, dead code elimination and strength reduction. An older proof of concept version I implemented a year ago featured also loop invariant code motion, inline expansion and tail-recursive optimizations. The new compiler framework is very powerful and I will add even more algorithms and a graph coloring register allocator for example. So there is some more NP-complete fun ahead making startup time even worse.

Purpose

We need a Flash Player that performs very fast for special purposes. We could use JITB at Audiotool to render mixdowns of tracks or other companies could use it to run their ActionScript code on the server side. You can use JITB to create an offline application that runs on a client machine. You can hack JITB to do what you want. Someone could potentially write a web framework for ActionScript and you could execute that code on Google App Engine if compiled AOT. And it is definitly possible to create Android applications using ActionScript. JITB’s terrain is everything but the browser in my opinion. However I can imagine that someone would be interested in creating a plugin version. With some clever simple caching algorithms it could make sense to get around startup costs but I am not interested in doing this.

Legal

There are no legal obligations to face since both the AVM2 and SWF specifications are published by Adobe. It is also not illegal to compile code for the JVM.

ActionScript Subset

By “a subset of the language” I really mean the language and not the API. JITB supports currently only statically typed ActionScript code. However the internal language model can be adjusted to support dynamic typing as well and with Java 7 things got even easier. In fact JITB does support dynamic code as long as it is able to infer types correct. Does this mean that I am willing to implement the full Flash Player API all by myself? Hell no.

In the next weeks I will probably create a better example that is easier to understand and get some other stuff ready so that you can try everything for yourself.

My talk at FITC San Francisco is over and I want to share some of the anouncements from today with you. At the end of my talk I was showing JITB.

What you see in the YouTube video I posted a while ago is a Java program executing a SWF. For FITC I added some more code and an OpenGL based Display List renderer. In other words: I wrote a Flash Player.

However I should rephrase that statement and say I am attempting to build a Flash Player. The current state is available in the sf2010-sprint clone of Apparat. I will merge the changes into the main Apparat branch when I am back home in Germany.

JITB is currently able to translate a subset of ActionScript code at runtime into Java bytecode and runs nearly at the same speed as native Java. This is a really huge improvement compared to standard ActionScript performance. A lot of smart people worked on the JVM and made it really fast. Apparat will allow you to leverage all this hard work in the future. I am also shooting for Java interoperability at some level so that you can call Java classes from within ActionScript. Hopefully you will be able to use JITB on your desktop machine, on a server or on an Android phone. Basically everywhere Java runs.

There are still a lot of things missing. The whole Flash API needs to be implemented. And the Display List rendering needs a proper OpenGL implementation. However I thought this might be some cool stuff to share with you in its early stages.
My hope is that more people start contributing to the project. Maybe some OpenGL guru wants to take care of the Display List rendering or someone else likes to help implement the Flash API in Java.

I also showed a Raytracer by Nico Zimmermann during my presentation and promised to put the URL on my blog so here it is. His company is called Britzpetermann and the address is http://www.britzpetermann.com/.

Update: Please do not think that this implementation is 30x faster than the Flash Player developed by Adobe. One(!) microbenchmark is never a number you should count on. I would like to make clear that I never said this.

Use the inline feature with care. Apparat does not try to optimize your code and performs nothing but dead simple inlining. This can lead to slower code due to the creation of lots of local registers. Your code gets also much bigger and will require more space in memory. I am actually not a fan of manual inlining at all. I think it makes only sense to inline code if you have a powerful optimizer available that will cleanup the whole mess.

The fun story about this example is that the inlined version is slower using the lastes Flash Player release candidate if you have only 40.000 particles. That is why I increased the number of particles to 80.000 ;). I developed the example using an old standalone player and the inlined version was nearly twice as fast. However when I watched the example in the browser with the latest release candidate the game was completely different. Kudos to Adobe for significantly improving the Flash Player performance!

• AS3-only Version
• Inline Version
• Source

The Java to SWF compiler does not compile Java sourcecode but JVM bytecode to ActionScript bytecode. This means I do not have to teach my program the Java language. It only understands JVM bytecode. This seems like an akward decision on the one hand since working on the bytecode level implies lots of problems. But it turns out that this was a really cool decision on the other hand. “Java to SWF compiler” is maybe the wrong description. “any language that compiles to JVM bytecode to SWF compiler” is maybe better.

So what does any language mean? Here is a list of JVM languages. Now you feel maybe like I did after reading that tweet. And I am really looking forward to get Scala up and running.

Some problems still exist. Threading is one issue and I will basically have to do what Scott Peterson did for Alchemy. But reflections, annotations and method overloading have to be solved as well. Some glitches may exist even after figuring everything out. Stacktraces will look pretty weird. However I think this is a really cool project.

Update: I forgot to mention something important. Java supports native code. This means you can build a library that works with OpenGL for instance. Those native methods can not be converted. There are also some other things that do not work. File access is just one of them.

Of course doing something at home influences work. I decided to write a build tool for the Hobnox AudioTool which is about 200 lines of Scala code. Cool thing is that this tool replaces a manually maintained Ant build and all project dependencies are always correct. Plus analyzing the dependencies in a more powerful way allows me to spawn the compiler in parallel. Building the AudioTool is now twice as fast and much more comfortable.

When learning a new programming language you also learn about new concepts. Functional languages in general have a different approach to nullable types. I know Scala is not the only one but let me introduce the concept in terms of ActionScript.

When you have a method that returns either a result or nothing: what do you do? Imagine you have some kind of service and a Dictionary of users. Requesting a user works by his unique id. The Dictionary is private to the class since you want to keep it read only.

function getUser(id: String): User {
  return hasUser(id) ? users[id] : null;
}

If I would now simply ask the service for an unknown user and do something like if(getUser('xyz').isLoggedIn) { trace('Hooray'); } I could probably and up with a null-reference error. No one checks for me if the user exists. So what else could we do? Write a lot of boilerplate code and prepend a check if the user is null or not each time we request one from the service. A much better approach in my opinion is to throw an error as early as possible. In this case we would rewrite the method to something like this:

function getUser(id: String): User {
  if(hasUser(id)) return users[id];
  throw new NoSuchUserError(id);
}

In this scenario we get informed about the error as early as possible. But we are stuck again. First of all ActionScript does not enforce you to catch possible exceptions. This means if you do not read the documentation of a method carefully or look into the source code of a method before calling it somewhere you will never know that it throws an exception. And what if we are actually in a scenario where we do not expect errors for non-existing objects? Think of the Dictionary object throwing each time an error when you access it and the result is null. How could I even check if an object exists in a Dictionary?

try {
  dictionary[key];
  return true;
}
catch(noSuchElementError: NoSuchElementError) {
  return false;
}

I guess you see that this can not be the solution to our problem. In a real world example you may deal with your own collection of objects instead of a Dictionary of course. So we have to get rid of exceptions and null for the optimal solution. Scala’s approach to this problem is the Option type. We always abuse null as a placeholder when we want to express that an element does not exist. The Option means that either Some or None result exists. Rewriting our getUser function using this approach would yield the following ActionScript code.

function getUser(id: String): Option {
  return hasUser(id) ? new Some(user[id]) : new None();
}

Why is this much better than the old approach? When calling the method you will always know that the method has only an optional result value. We get rid of the exception and null values. Our only problem at the moment is ActionScript. The result is now untyped. In an ideal world this method would be written as:

function getUser(id: String): Option.<User> {
  return hasUser(id) ? new Some.<User>(user[id]) : new None.<User>();
}

However we can still tackle this issue by implementing null-representations of our objects. Imagine the User class. You could rewrite the code to something like this.

function getUser(id: String): IUser {
  return hasUser(id) ? user[id] : new NullUser();
}

final class NullUser implements IUser {
  public function get isLoggedIn(): Boolean { return false; }
  public function get name(): String { return 'null'; }
}

And even if you are interested in null-reference errors you could rewrite your code to something like this:

final class NullUser implements IUser {
  public function get isLoggedIn(): Boolean {
    CONFIG::ThrowNullReferenceErrors { throw new NullReferenceError(); }
    return false;
  }
  public function get name(): String {
    CONFIG:: ThrowNullReferenceErrors { throw new NullReferenceError(); }
    return 'null';
  }
}

It is definitely a very different approach. A functional language like Scala allows you to deal much better with Options. But it makes sense to diferentiate between an uninitialized variable which is null and an optional result of a function. Unfortunately this is at the moment very painful with the lack of generics in ActionScript.

So how does this work? Because I am now working on the AudioTool backend for our highly anticipated launch I had to switch to fulltime Java development. In order to understand the JVM better I started to search for the optimizations that the HotSpot VM performs and stumbled upon a framework called Soot. Voilà. Reading about Soot made me question why nobody is working on such a framework for ActionScript. Probably because universities see Flash just as a toy. For us developers being dependent on the Flash Platform it is of course more than that.

I will quickly explain what Soot does. It reads compiled Java class files and converts the machine code they contain into a three address code representaiton. This code is then heavily optimized. TAAS is basically doing the same. I also want to note that Soot is by far more complex and complete than TAAS right now. It would be megalomania saying that TAAS is Soot for ActionScript. But you get the idea. The one framework converts Java to three address code and the other framework converts ActionScript to three address code. Maybe it is possible to connect them.

This is what I did for the Java compiler. The whole pipeline looks like this:

1. Compile Java sourcecode with the Java compiler of your choise
2. Use Soot to get the three address code representation of a compiled binary
3. Create TAAS expressions for the expressions that Soot uses
4. Connect the TAAS graph the same way the Soot graph is connected
5. Compile the TAAS graph to a SWF

This is about it. There is some glue-code involved here and there and I did not implement all Java expressions yet. I did also not bother about threads. Basically one could do what Scott has done for Alchemy. The cool thing is that we get highly optimized Java code because of the Java compiler and Soot. Then the TAAS compiler can run a second time after linking is done and perform platform specific optimizations like inlining and loop invariant code motion for certain expressions.

I already wrote a couple of classes and packed them in a SWC. Those mimic the behaviour of the Java classes like java.lang.System or java.util.LinkedList. On the Java side I implemented the Flash classes like flash.display.Sprite and flash.events.EventDispatcher. The interesting part is that all the methods those classes contain are marked native which means they have no implementation since they are native to the Flash Player.
This is also a great advantage. It is slow in Alchemy to call Flash methods and to communicate with Flash classes. The Java approach does not have this tradeoff. I also do not have to trigger the ActionScript compiler. The conversion form class to swf is entirely done using Soot and TAAS.

So far for the Java compiler. But in order to show a little bit more of the potential I wanted to have a C# compiler working as well. The great thing about advanced languages and a large community is that people have already built a lot of tools. It did not take too long to find an advanced framework that converts C# to Java. Actually this is not be the best approach. But I could not find the “Soot for C#” the evening before my FOTB session.

So I am using net2java to convert .NET code like C# or VisualBasic to Java. That code is then compiled using the Java compiler and I am using Soot again to convert the code to TAAS without much stress.

For me the most important part is to know that it is possible to compile .NET source code and Java binaries to SWF. To complete those tools one just has to implement all the missing expressions and the standard Java library in Flash. Everything could also work the other way around. Compiling from TAAS to .NET, Silverlight and Java is another option.

Strength reduction can already handle about 55 different cases at the moment. For instance it will convert code like if( x - 1 == 0 ) into if( x == 1 ). But it will also remove expressions that other optimizations could introduce. x + 0 is something that might occur during inline expansion. Or x + Number.NaN for example.

But I really like the tail recursive optimizations. TAAS can detect if a method needs to call itself or not. Take this code for example:

private function sum( i: int, value: int ): int
{
	if( i == 0 )
	{
		return value;
	}

	return sum( i - 1, value + i );
}

You see that the last statement sum( i - 1, value + i ) is tail recursive. This means we can replace the recursive call to sum() with a jump to the beginning of the method after changing the method parameters. The sum() method becomes converted to something like a simple while loop. This means also that one could have written code like this in the first place since this optimization applies only to methods that do not have to be recursive after all.

Knock yourself out with some examples:

• Before / After (Inline)
• Before / After (Inline)
• Before / After (Tail recursion)

Finally I am able to present the first results of TAAS. It was a long way to get here. I was actually not sure at all if I will manage to show something at FOTB 09. Let me explain what happens here.

In the image is the original ActionScript code on the left. The ActionScript bytecode produced by the ASC is in the middle and the compiled TAAS version is on the right. You see that I am running a loop and call a method inside that loop. It would be nice to have small helper methods like this one inlined automatically. TAAS can do this for you now.

So how does this work? First of all an SWF file is parsed and the bytecode is extracted. This bytecode is transformed into a control flow graph. This control flow graph is converted into a graph of TAAS expressions which a lot of benifits. The conversion step works like the Flash Player and kind of executes your code. The result is that all methods and local variables are typed if that is possible.

It is much easier to perform a lot of optimizations now. Like inline expansion. In order to do that I can simply compile the method that is called to TAAS and connect the vertices together in the graph after some adjustments have been made. The inline expansion step will automatically inline methods when it thinks it is useful and possible. You can only inline methods that are private or final and make no use of reflections.

Inlined methods create a lot of overhead. Usually they contribute additional local variables and the code gets bigger. So one step after the inline expansion is to clean everything up again. In this example I can remove those registers thanks to copy propagation and dead code elimination.
And it gets even better. If you take a closer look at the example, the calc method expects two paramters that are of type Number. Therefore the original method uses an Add instruction. TAAS knows about the types and sees that it will pass two integers into this method. Since it makes no sense to convert from int to Number to int it stays with integer in this case.

So after inlining the method ends up with even less local variables. The original local variable t1 is considered useless since it is only used one time. TAAS will put the code getTimer() at the position where t1 has been used and we end up with a heavily optimized method. What I like the most is that I do not have to change the way I write my ActionScript code. All happens behind the scenes automatically.
Some other optimizations are implicit. TAAS will use AddInt instead of Add if both operands are typed int. A FlowOptimizer is also involved and will invert the if expression for the loop. This can reduce the number of required jumps in a method by 50%.

I know that the output is not perfect. But this is all a work in progress and really the first result that I can share. The SWFs speak for them self.

• Before
• After

Example01

This is the old code of the already optimized attractor using the Memory API instead of a Vector.<uint>.

Example02

In this case there exists no Particle class at all and no linked list. The particle information is stored inside the memory as well. Particles are extended to a fourth value so indexing a particle can be done with a simple bitshift which is very fast.

Example03

The last example uses float instead of double values for the particles. The framerate stays the same which is really cool because the memory usage drops. Before a particle consisted of four doubles which is a total of 4 * 8b = 32b. In this example each particle takes up only 16b. There the memory difference is 0x4B0000b which is about 4.7mb in total.
And also the first version needs about 20mb on my machine which means about 12mb of RAM are not wasted. Pretty cool when thinking about devices with less memory.

By the way I just stumpled across a bug when using [Embed]. Hopefully it will be easy to fix.

• Download Examples

Hopefully you are kind enough to provide me with some feedback. I am especially interested in Exceptions that occur when reading or writing ABC files. Have fun!

• Download

Update: TDSI is now open source!

So what is the idea? TAAS is part of a framework I developed to manipulate SWF, SWC and ABC files. The main focus are of course ABC files since they contain the bytecode which gets executed.
Part of the framework are tools for control flow analysis, various bytecode analyzers and also a search-and-replace system which work on a bytecode level. There are for instance pattern matchers that search for bad code produced by the ASC and replace the match with a more performant set of instructions.

With all those weapons in my arsenal I thought it should be a walk in the park to implement the Alchemy features in a way that makes sense. So the first idea is to have the old functionality AS3C had but more robust. AS3C had a feature that was the __asm function which allowed you to inline instructions. The new framework comes with the old __asm and also another cool method: __bytecode! This will inline raw bytes. This means also you would have to know all the indices for variables you want to use from the constant pool in advance so __asm will still be your friend.

With the __bytecode method it is already possible to use all Alchemy features again. It would also be possible with the __asm method but writing plain bytes is simply more elitist. In order to make it easy for the developer I want a high-level API. Having a class with some static methods is nice of course but also slow. Alchemy is fast because those opcodes that write and read from a ByteArray are no method calls. They are low-level FlashPlayer features.

The first attempt was to write a Memory class that allows you to use the Alchemy features. This class contains raw bytecode implementations and ActionScript code. This means if you do not use the optimizer everything will still work — only 1000 times slower. When looking at the memory class there is another tool of the framework that becomes very helpful. Both the __bytecode and ActionScript stuff should not co-exist with each other. So when we inline the bytecode a dead-code-elimination will simply cleanup afterwards. Since the 0x47 byte for instance is “ReturnVoid” the ActionScript code which would follow afterwards can be dropped. That code is now unreachable.

Step two is to replace all calls to the Memory class with the correct Alchemy opcode. This was really simple and the result is a really really fast way to access a ByteArray while still maintaining a high comfort. Of course one might think now that the __bytecode method becomes useless since no methods of the Memory class are called at all. But if anyone is crazy enough to access the Memory class untyped with a runtime namespace for instance you are still happy to have the code optimized inside. In some circumstances it is simply impossible to figure out that someone called Memory.writeByte(). End of the story: your calls to a ByteArray are always optimized in the best way possible.

This is an example of the Memory.readByte() method before applying optimizations:

0x000000       GetLocal0
0x000001       PushScope
0x000002       FindPropStrict       QName(PackageNamespace("com.joa_ebert.abc.bytecode.asbridge"), "__bytecode")
0x000004       PushShort            0xd1
0x000007       PushByte             0x35
0x000009       PushByte             0x48
0x00000b       CallPropVoid         QName(PackageNamespace("com.joa_ebert.abc.bytecode.asbridge"), "__bytecode"), 3
0x00000e       GetLex               QName(PackageNamespace("flash.system"), "ApplicationDomain")
0x000010       GetProperty          QName(PackageNamespace(""), "currentDomain")
0x000012       GetProperty          QName(PackageNamespace(""), "domainMemory")
0x000014       GetLocal1
0x000015       SetProperty          QName(PackageNamespace(""), "position")
0x000017       GetLex               QName(PackageNamespace("flash.system"), "ApplicationDomain")
0x000019       GetProperty          QName(PackageNamespace(""), "currentDomain")
0x00001b       GetProperty          QName(PackageNamespace(""), "domainMemory")
0x00001d       CallProperty         QName(PackageNamespace(""), "readUnsignedByte"), 0
0x000020       ReturnValue

The same method after inlining the bytes and applying various other analysis like dead-code-elimination:

0x000000       GetLocal0
0x000001       PushScope
0x000000       GetLocal1
0x000001       GetByte
0x000002       ReturnValue

This is an example of the famous inverse square root using the Memory API:

private function invSqrt( value: Number ): Number
{
	var half: Number = 0.5 * value;
	Memory.writeFloat( value, 0 );
	Memory.writeInt( 0x5f3759df - ( Memory.readInt( 0 ) >> 1 ), 0 );
	value = Memory.readFloat( 0 );
	value = value * ( 1.5 - half * value * value );
	return value;
}

The same method before optimization in bytecode representation:

0x000000       GetLocal0
0x000001       PushScope
0x000002       PushDouble           0.5
0x000004       GetLocal1
0x000005       Multiply
0x000006       ConvertDouble
0x000007       SetLocal2
0x000008       GetLex               QName(PackageNamespace("com.joa_ebert.abc.bytecode.asbridge"), "Memory")
0x00000a       GetLocal1
0x00000b       PushByte             0x0
0x00000d       CallPropVoid         QName(PackageNamespace(""), "writeFloat"), 2
0x000010       GetLex               QName(PackageNamespace("com.joa_ebert.abc.bytecode.asbridge"), "Memory")
0x000012       PushInt              0x5f3759df
0x000014       GetLex               QName(PackageNamespace("com.joa_ebert.abc.bytecode.asbridge"), "Memory")
0x000016       PushByte             0x0
0x000018       CallProperty         QName(PackageNamespace(""), "readInt"), 1
0x00001b       PushByte             0x1
0x00001d       ShiftRight
0x00001e       Subtract
0x00001f       PushByte             0x0
0x000021       CallPropVoid         QName(PackageNamespace(""), "writeInt"), 2
0x000024       GetLex               QName(PackageNamespace("com.joa_ebert.abc.bytecode.asbridge"), "Memory")
0x000026       PushByte             0x0
0x000028       CallProperty         QName(PackageNamespace(""), "readFloat"), 1
0x00002b       ConvertDouble
0x00002c       SetLocal1
0x00002d       GetLocal1
0x00002e       PushDouble           1.5
0x000030       GetLocal2
0x000031       GetLocal1
0x000032       Multiply
0x000033       GetLocal1
0x000034       Multiply
0x000035       Subtract
0x000036       Multiply
0x000037       ConvertDouble
0x000038       SetLocal1
0x000039       GetLocal1
0x00003a       ReturnValue

The same method after inlining the Memory API:

0x000000       GetLocal0
0x000001       PushScope
0x000002       PushDouble           0.5
0x000004       GetLocal1
0x000005       Multiply
0x000006       ConvertDouble
0x000007       SetLocal2
0x00000a       GetLocal1
0x00000b       PushByte             0x0
0x000000       SetFloat
0x000012       PushInt              0x5f3759df
0x000016       PushByte             0x0
0x000000       GetInt
0x00001b       PushByte             0x1
0x00001d       ShiftRight
0x00001e       Subtract
0x00001f       PushByte             0x0
0x000000       SetInt
0x000026       PushByte             0x0
0x000000       GetFloat
0x00002b       ConvertDouble
0x00002c       SetLocal1
0x00002d       GetLocal1
0x00002e       PushDouble           1.5
0x000030       GetLocal2
0x000031       GetLocal1
0x000032       Multiply
0x000033       GetLocal1
0x000034       Multiply
0x000035       Subtract
0x000036       Multiply
0x000037       ConvertDouble
0x000038       SetLocal1
0x000039       GetLocal1
0x00003a       ReturnValue

As you can see this is blazing fast. Now the next job is to finish TAAS. Once TAAS is complete even a method like the inverse square root might be inlined and optimized much better. I did a simple test using the Lorenz attractor from before and replacing the Vector.<uint> buffer with a ByteArray gave a performance boost of about 5fps. Afterwards I tried getting rid of the Particle class completly and the framerate dropped a little bit. But imagine having 300.000 particle’s x, y and z coodrinates stored in an Array. It was still faster than the old version but not as fast as combining the power of Alchemy with simple ActionScript optimizations like linked lists.

var x: Number = 1.0;
var y: Number;

if(true)
{
	y = 3.0;
}
else
{
	y = 2.0;
}

x += y;

return x;

For the sake of simplicity I will not use local variables in the bytecode. But it will be easier to figure out what happens in the bytecode.

      PushDouble           1.0
      PushTrue
      IfTrue               L0

      PushDouble           2.0
      Jump                 L1

L0:   PushDouble           3.0
L1:   Add
      ReturnValue

This bytecode is nearly the same as the method but without local variables. This should be fairly simple to understand.

When converting bytecode to Taas, its corresponding control flow graph is converted into a control flow graph of Taas expressions. I do not want to go into much detail how the stack based code is converted but here is the graph before and after the transformation from bytecode to Taas. It looks quite similar but there is a major difference. All those constant values in the Taas graph are not values pushed on a stack but assignments to virtual variables which exist only in theory. You can see also that the Add in the bytecode has known operands and type in the Taas version. Since it is currently not know if 2.0 or 3.0 has been used there is a Φ function that says “This value is either 2.0 or 3.0 depending on the path taken at runtime”.

So as I said this code can be optimized much better than stack based code. This graph is very redundant. There are three utilities to compact and simplify the current graph of Taas expressions. The algorithm has been developed for Java bytecode but works with ActionScript very well tool. The concept is quite simple. Perform copy propagation, constant folding and dead code elimination until the graph stops changing. Applying these techniques to the Taas graph yields the following results.

Copy propagation:

Most of the theoretical variables have been eliminated using copy propagation.

Constant folding:

Known constants have been replaced. Even the if condition is no longer needed and the false branch has been removed. The dead code elimination will clean up this code afterwards.

Dead code elimination:

Dead code elimination has removed the dead Jump statement and also the value 2.0 from the Φ expression. Afterwards constant folding replaced the Φ expression with its one constant value. Then another iteration of constant folding replaced the Add expression with the constant value 4.0. Afterwards copy propagation has put the result into the Return statement and voilá.

This result may have seemed quite obvious from the beginning and you may ask who writes such code. Probably nobody. But once you start inlining methods this makes a lot of sense. A lot of preconditions are known in that case and unnecessary branches can be removed. Since inlining methods bloats up the code it is very important to compact it afterwards as much as possible. I hope you see now how interesting all of this might be in the future.